Picture display method, terminal device, storage medium and program product

Abstract

The invention discloses a picture display method, terminal equipment, a storage medium and a program product, and relates to the technical field of terminal software. In the application, when a display instruction for indicating to display a first picture is detected, terminal equipment firstly detects whether the first picture is a private picture containing private information, and if the first picture is the private picture, a facial information acquisition device acquires facial information of a user and performs face recognition. And the privacy information in the picture is controlled to be hidden under the condition of identifying that the non-target user browses the picture. Thus, in the process that the user browses the picture, protection of the privacy information in the picture can be achieved under low equipment power consumption, and the risk of privacy information leakage is reduced.

Description

Technical Field

[0001] This application relates to the field of terminal software technology, and in particular to an image display method, terminal device, storage medium, and program product. Background Technology

[0002] Users store numerous images on their devices through taking photos, downloading, and screenshotting. Some of these images may contain private information such as identification details, transaction records, and account balances. Viewing these images may pose a risk of privacy breaches. Therefore, there is an urgent need to provide an image display method to reduce the risk of privacy leaks and improve security during image viewing. Summary of the Invention

[0003] This application provides an image display method, terminal device, storage medium, and program product, which can improve the security of users' privacy information while reducing device power consumption during image browsing.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, a method for displaying an image is provided, applied to a terminal device. The method includes: responding to a first display instruction, detecting whether a first image is a privacy image, wherein the first display instruction is used to instruct the display of the first image, and the privacy image refers to an image containing privacy information; if the first image is the privacy image, activating a facial information collection device of the terminal device to collect user facial information, and performing face recognition on the collected user facial information; if it is identified that the currently collected first user facial information contains a non-target face, then displaying the first image after hiding the privacy information.

[0006] In this application, upon detecting a display command instructing the display of a first image, the terminal device first checks whether the first image is a privacy image containing privacy information. If the first image is a privacy image, the device then collects the user's facial information and performs facial recognition. Furthermore, if it detects that a non-target user is browsing the image, it controls the hiding of the privacy information in the image. In this way, during the user's image browsing process, the protection of privacy information in the image can be achieved with low device power consumption, reducing the risk of privacy information leakage.

[0007] Optionally, during the display of the first image after hiding the privacy information, the facial information collection device continuously collects the user's facial information and performs facial recognition on the collected user facial information; if the currently collected second user facial information is identified as containing a face, and the included face is the target face, then the privacy information in the first image is restored and displayed.

[0008] In this application, during the process of displaying the first image with hidden privacy information, the terminal device can continuously collect the user's facial information and perform facial recognition through a facial information collection device. This allows it to detect whether the user viewing the first image has changed, and thus decide whether to restore the display of the privacy information in the first image. This ensures that when the user viewing the first image becomes the target user, the target user can fully view the content of the first image.

[0009] Optionally, in response to a second display instruction, the terminal device detects whether the second image is the privacy image, the second display instruction being used to instruct the switching display of the second image; if the second image is not the privacy image and the second image is an image switched from the privacy image, then the second image is displayed, and the user's facial information is continuously collected through the facial information collection device; if no display instruction for any of the privacy images is detected within a first duration starting from the first moment, then the facial information collection device is turned off to stop collecting the user's facial information, the first moment being the moment when the second image begins to be displayed.

[0010] In this application, if the terminal device detects an instruction to switch to displaying a second image that does not contain privacy information while displaying a privacy image, it continues to collect user facial information for a certain period of time starting from the switch to displaying the second image. If the user does not switch to displaying a privacy image within this period of time, the terminal device can turn off the facial information collection device to stop collecting user facial information. This reduces device power consumption caused by continuous collection of user facial information. Furthermore, compared to stopping collection of user facial information immediately upon switching to the second image or upon detecting that the second image is not a privacy image, this embodiment of the application maintains user facial information collection for a certain period of time from the switch to displaying the second image, avoiding increased device power consumption caused by frequent switching of the facial information collection device when the user switches from the second image to other privacy images in a very short time.

[0011] Optionally, during the display of the first image after hiding the privacy information, in response to a first operation instruction, it is determined whether the most recently collected user facial information contains a target face. The first operation instruction is used to instruct the display of privacy information in at least one privacy area of ​​the first image, where the privacy area is an area containing privacy information. If the most recently collected user facial information contains the target face, then the privacy information in at least one of the privacy areas is displayed.

[0012] In this application, during the process of displaying a first image with hidden privacy information, if a user instruction is detected to display privacy information within at least one privacy area of ​​the first image, the terminal device can perform user authentication based on the most recently collected user facial information. If it is determined that the user's facial information contains a target face, that is, if it is determined that the target user is among the users currently viewing the first image, the terminal device can restore the display of the privacy information within the at least one privacy area. Thus, in scenarios where multiple people, including the target user, view the first image, the target user can flexibly choose to display the hidden privacy information to others.

[0013] Optionally, in response to the second operation instruction, the terminal device can determine whether the most recently collected user facial information contains a target face. The second operation instruction is used to instruct that the privacy information in the privacy image be hidden by default when displaying the privacy image. If the most recently collected user facial information contains the target face, then when a display instruction for any of the privacy images is detected, the privacy image with the privacy information hidden is displayed.

[0014] In this application, after detecting the second operation command, the terminal device can authenticate the user based on the most recently collected user facial information. Once authentication is successful, upon detecting a subsequent command to display a private image, it will no longer collect user facial information or perform face recognition; instead, it will directly hide the private information in the image by default before displaying it. This allows users to flexibly choose whether to hide the private information in the image based on their own needs.

[0015] Optionally, in this application, if the terminal device identifies that the first user's facial information contains multiple faces, then it is determined that the first user's facial information contains a non-target face; or, if it identifies that the first user's facial information contains a single face and that the face is not the target face, then it is determined that the first user's facial information contains a non-target face.

[0016] In this application, the terminal device further determines whether the face is the target face after recognizing that the first user's facial information contains a face. If the first user's facial information contains multiple faces, it is not necessary to further compare the multiple faces with the target face. It can directly determine that the first user's facial information contains a non-target face. In this way, not only can the risk of privacy information leakage in the scenario of multiple people browsing pictures be reduced, but also the power consumption of the device can be reduced.

[0017] Optionally, the first image is the first private image displayed after launching the photo album application. Alternatively, the first image is the image that is switched to be displayed after the facial information collection device is turned off.

[0018] Optionally, before detecting whether the first image is a privacy image, the first image can be identified based on a privacy recognition model to obtain an image recognition result. Then, based on the image recognition result, a mapping relationship between the image identifier of the first image and privacy indication information is generated and stored, wherein the privacy indication information is used to indicate whether the corresponding image is the privacy image. On this basis, after detecting the first display instruction, the terminal device can obtain the privacy indication information of the first image from the mapping relationship based on the image identifier of the first image. If the privacy indication information of the first image includes a first indication value, then the first image is determined to be the privacy image.

[0019] In this application, the terminal device can pre-identify whether the first image is a privacy image using a privacy recognition model, and then generate and store a mapping relationship between the image identifier and privacy indication information of the first image based on the recognition result. Based on this, after the terminal device detects the first display instruction, it can directly determine whether the first image is a privacy image based on this mapping relationship, thereby shortening the time required to detect whether the first image is a privacy image and improving image display efficiency.

[0020] Optionally, the image recognition result includes the location of one or more privacy regions, and the mapping relationship includes the location of the one or more privacy regions. Based on this, before displaying the first image after hiding the privacy information, the terminal device can obtain the location of the one or more privacy regions in the first image from the mapping relationship based on the image identifier of the first image; based on the location of the one or more privacy regions, the privacy information in the one or more privacy regions is hidden to obtain the first image after hiding the privacy information.

[0021] In this application, the image recognition result obtained by the terminal device includes the location of one or more privacy regions in the first image. Correspondingly, the terminal device can also store the location of these one or more privacy regions. Based on this, when the terminal device subsequently hides privacy information in the first image, it can accurately locate the privacy regions based on the location of these one or more privacy regions, and then selectively hide the privacy regions, while displaying the non-privacy regions as usual, thus achieving precise protection of privacy information in the image.

[0022] In a second aspect, an image display device is provided, the image display device comprising at least one module, the at least one module being configured to perform the image display method described in the first aspect above.

[0023] Thirdly, a terminal device is provided, the terminal device including a processor, the processor being configured to execute at least one program instruction or code stored in a memory to implement the image display method described in the first aspect above.

[0024] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer device, cause the computer device to perform the image display method described in the first aspect.

[0025] Fifthly, a computer program product containing instructions is provided, which, when run on a computer device, causes the computer device to perform the image display method described in the first aspect.

[0026] The technical effects achieved by the second to fifth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0028] Figure 2 A software architecture diagram of a terminal device provided in an embodiment of this application;

[0029] Figure 3 A flowchart illustrating an image display method provided in this application embodiment;

[0030] Figure 4 A flowchart illustrating the mapping relationship between image identifiers and privacy indication information is provided in this application embodiment;

[0031] Figure 5 This is a schematic diagram illustrating how to hide privacy information in an image, as provided in an embodiment of this application.

[0032] Figure 6 This is an illustration of the effect of displaying a first image in a scenario where multiple people are browsing images, as provided in an embodiment of this application.

[0033] Figure 7 An illustration showing the effect of displaying a first image in a scenario where a target user is browsing images, as provided in an embodiment of this application;

[0034] Figure 8 A flowchart illustrating another image display method provided in this application embodiment;

[0035] Figure 9 This is a diagram illustrating the display effect of the first image in a scenario where the user browsing the image changes, as provided in the embodiments of this application.

[0036] Figure 10 A flowchart illustrating the process of restoring the display of privacy information based on user actions, provided in an embodiment of this application;

[0037] Figure 11 This is a schematic diagram illustrating the restoration and display of privacy information within a specific privacy area based on user operation, as provided in an embodiment of this application.

[0038] Figure 12 This is a schematic diagram illustrating the restoration and display of privacy information in the first image based on user operation, provided in an embodiment of this application.

[0039] Figure 13 A flowchart illustrating yet another image display method provided in this application embodiment;

[0040] Figure 14 This is a schematic diagram of the structure of an image display device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0042] Before providing a detailed explanation of the embodiments of this application, let's first introduce the application scenarios involved in the embodiments of this application.

[0043] When using mobile devices such as smartphones and tablets, users may take and store photos, download images from online or social media applications, or take and save screenshots. These images stored on the device may contain private information. For example, some images may contain information about the user's ID card, credit card, or bank card. Furthermore, some images may contain private information such as transaction records, balance records, account numbers, and passwords from payment or financial applications.

[0044] For a large number of pictures stored on a terminal device, users can browse these pictures by opening the photo album. During this browsing process, other people may be present, or the user may be viewing the pictures in the album with others. In such cases, others may see private information in the pictures, leading to privacy leaks. Based on this, this application provides an image display method. In this method, when a display command indicating the display of a first image is detected, the terminal device first checks whether the first image is a private image containing private information. If the first image is a private image, a facial information collection device is activated to collect the user's facial information and perform face recognition. Furthermore, if it is determined that a non-target user is browsing the image, the private information in the image is hidden. In this way, during the user's image browsing process, privacy information in the images can be protected with low device power consumption, reducing the risk of privacy leaks.

[0045] The image display method provided in this application can be applied to terminal devices, such as smartphones, tablets, and wearable devices. This application does not impose any restrictions on the specific type of terminal device.

[0046] For example, Figure 1 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. See also... Figure 1 The terminal device 100 may include: a processor 101, an external memory interface 102, an internal memory 103, an antenna 1, an antenna 2, a mobile communication module 104, a wireless communication module 105, a display screen 106, a camera 107, a sensor module 108, a charging management module 109, a power management module 110, a battery 111, buttons 112, etc. The sensor module 108 may include a pressure sensor 108A, a gyroscope sensor 108B, an accelerometer sensor 108C, a touch sensor 108D, a time-of-flight (TOF) sensor 108E, etc.

[0047] Processor 101 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). The NPU is a neural network (NN) computing processor that, by referencing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn. The NPU enables intelligent cognitive applications of the terminal device 100, such as image recognition, face recognition, speech recognition, and text understanding. The aforementioned processing units may be independent devices or integrated into one or more processors.

[0048] The processor 101 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 101 is a cache memory. This memory can store instructions or data that the processor 101 has just used or that are used repeatedly. If the processor 101 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 101, and thus improves the efficiency of the system.

[0049] In some embodiments, the processor 101 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0050] The charging management module 109 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 109 receives charging input from the wired charger via a USB interface. In some wireless charging embodiments, the charging management module 109 receives wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 111, the charging management module 109 can also supply power to the terminal device via the power management module 110.

[0051] The power management module 110 connects the battery 111, the charging management module 109, and the processor 101. The power management module 110 receives input from the battery 111 and / or the charging management module 109, supplying power to the processor 101, internal memory 103, display screen 106, and wireless communication module 105, etc. The power management module 110 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 110 may also be located within the processor 101. In other embodiments, the power management module 110 and the charging management module 109 may be located in the same device.

[0052] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 104, wireless communication module 105, modem processor, and baseband processor.

[0053] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0054] The mobile communication module 104 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the terminal device 100. The mobile communication module 104 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 104 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 104 can be housed in the processor 101. In some embodiments, at least some functional modules of the mobile communication module 104 and at least some modules of the processor 101 can be housed in the same device.

[0055] The wireless communication module 105 can provide solutions for wireless communication applications on the terminal device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 105 can be one or more devices integrating at least one communication processing module. The wireless communication module 105 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 101. The wireless communication module 105 can also receive signals to be transmitted from processor 101, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0056] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 104, and antenna 2 is coupled to wireless communication module 105, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. Wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), and so on.

[0057] The terminal device 100 implements display functions through a GPU, a display screen 106, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 106 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 101 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0058] The display screen 106 is used to display images, videos, etc. The display screen 106 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include one or N display screens 106, where N is a positive integer greater than 1.

[0059] The terminal device 100 can perform shooting functions through an ISP, camera 107, video codec, GPU, display screen 106, and application processor.

[0060] The camera 107 is used to capture still images or videos. An object passes through the lens to generate an optical image projected onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. The focal length of the lens can be used to represent the field of view of the camera; a smaller focal length indicates a larger field of view. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In this embodiment, the terminal device 100 may include two or more cameras 107 with different focal lengths. The cameras included in the terminal device 100 may include a front-facing camera, which, in this embodiment, can be used to collect user facial information.

[0061] The external storage interface 102 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external storage card communicates with the processor 101 through the external storage interface 102 to perform data storage functions. For example, videos and other files can be saved on the external storage card.

[0062] Internal memory 103 can be used to store computer executable program code, including instructions. Internal memory 103 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback function), etc. The data storage area may store data created during the use of terminal device 100 (such as phonebook), etc. In addition, internal memory 103 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 101 executes various functional applications and data processing of terminal device 100 by running instructions stored in internal memory 103 and / or instructions stored in memory disposed in the processor.

[0063] Pressure sensor 108A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, pressure sensor 108A can be disposed on display screen 106. When a touch operation is applied to display screen 106, terminal device 100 detects the intensity of the touch operation based on pressure sensor 108A. Terminal device 100 can also calculate the touch position based on the detection signal of pressure sensor 108A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0064] The gyroscope sensor 108B can be used to determine the motion attitude of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 108B. The gyroscope sensor 108B can be used in navigation and motion-sensing game scenarios.

[0065] The accelerometer 108C can detect the magnitude of acceleration of the terminal device 100 in various directions (generally three axes). When the terminal device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of the terminal device and applied to applications such as landscape / portrait switching and pedometers.

[0066] Touch sensor 108D, also known as a "touch device," can be disposed on display screen 106. The touch sensor 108D and display screen 106 together form a touchscreen, also known as a "touchscreen." Touch sensor 108D is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 106. In other embodiments, touch sensor 108D may also be disposed on the surface of terminal device 100, in a different location than display screen 106.

[0067] The TOF sensor 108E is a depth sensor used to acquire depth information of a subject in space. Specifically, the TOF sensor 108E emits a modulated light signal. This light signal is reflected upon encountering a subject in space. The TOF sensor 108E receives the reflected light signal and calculates the time difference or phase difference between the reflected and emitted light signals to determine the depth information of the subject. In some scenarios, the TOF sensor 108E can also be used in conjunction with a camera 107 to generate a 3D image containing depth information.

[0068] It should be noted that, in some possible cases, the sensor module 108 of the terminal device 100 may also include a depth sensor such as a lidar sensor or a laser sensor for collecting depth information. In this case, the lidar sensor or the laser sensor may also be used in conjunction with the camera 107 to generate a three-dimensional image containing depth information.

[0069] Buttons 112 include a power button, volume buttons, etc. Buttons 112 can be mechanical buttons or touch buttons. Terminal device 100 can receive button input and generate key signal inputs related to user settings and function control of terminal device 100.

[0070] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0071] See Figure 2 This is a schematic diagram of the software structure of a terminal device according to an embodiment of this application. The operating system in the terminal device can be Android, Microsoft Windows, Apple iOS, or Harmony OS, etc.

[0072] In some embodiments, the operating system of a terminal device can be divided into four layers, including a hardware layer, a kernel layer, a system service layer, and an application layer, with the layers communicating with each other through software interfaces.

[0073] like Figure 2 As shown, the hardware layer includes physical devices that provide specific interactive responses, such as speakers, sensors, WLAN, modems, microphones, displays, and motors.

[0074] The kernel layer comprises the kernel abstract layer (KAL) and the driver subsystem. The KAL includes multiple kernels, such as the Linux kernel for Linux systems and the LiteOS kernel for lightweight IoT systems. A multi-kernel kernel layer can select the appropriate kernel for processing based on system requirements. The driver subsystem can include a hardware driver foundation (HDF), such as audio drivers, wireless local area network (WLAN) drivers, and modem drivers. The hardware driver foundation provides unified peripheral access capabilities and a framework for driver development and management.

[0075] The system service layer is the core set of capabilities of the operating system, providing services to applications through the framework layer. This layer may include a set of basic system capability subsystems, a set of basic software service subsystems, a set of enhanced software service subsystems, and a set of hardware service subsystems.

[0076] The system's basic capability subsystems provide fundamental capabilities for the operation, scheduling, and migration of distributed applications on operating system devices. These may include subsystems such as distributed soft bus, distributed data management, distributed task scheduling, Ark multi-language runtime, common base libraries, multi-modal input, graphics, security, artificial intelligence (AI), and user program frameworks. Among these, the Ark multi-language runtime provides runtime environments for C, C++, or JavaScript (JS) languages ​​and basic system class libraries. It can also provide a runtime environment for Java programs statically generated using the Ark compiler (i.e., the parts of the application or framework layer developed using the Java language).

[0077] The basic software services subsystem provides common, general-purpose software services to the operating system. These may include download services, multimedia services, power services, telephone services, etc.

[0078] The enhanced software service subsystem provides the operating system with differentiated, device-specific enhanced software services. It may include dedicated subsystems for smart screens, wearables, and the Internet of Things (IoT).

[0079] The hardware services subsystem provides hardware services to the operating system. This may include subsystems such as location services, biometric recognition, wearable hardware services, and IoT hardware services.

[0080] In this embodiment, the terminal device can identify private areas containing privacy information in pictures in the photo album application, perform facial recognition on collected user facial information, and hide privacy information in pictures containing privacy information through the system service layer.

[0081] The application layer includes system applications and third-party applications (or extended applications). System applications can include applications installed by default on terminal devices, such as the desktop, control bar, settings, and phone. Extended applications can be non-essential applications developed and designed by the terminal device manufacturer, such as terminal device management, device migration, note-taking, weather, and browser applications. Third-party non-system applications can be applications developed by other manufacturers that can run within the operating system, such as conferencing, games, navigation, social networking, shopping, and music applications.

[0082] In this embodiment, the terminal device can implement image rendering in the photo album application through the application layer, including rendering the original image and filling backgrounds, occluding images, or drawing text in private areas of the image. In addition, the terminal device can also implement user interaction through the application layer.

[0083] Multiple terminal devices running operating systems can achieve hardware cooperation and resource sharing through distributed soft bus, distributed device virtualization, distributed data management, and distributed task scheduling.

[0084] The image display method provided in the embodiments of this application will be explained in detail below.

[0085] Figure 3 This is a flowchart of an image display method provided in an embodiment of this application. This image display method can be applied to terminal devices. See also... Figure 3 The process may include the following steps:

[0086] S301: In response to the first display instruction, detect whether the first image is a private image.

[0087] In one possible scenario, after a user opens a photo album application, the application's interface may display thumbnails of multiple images. The user can click on one of these images to trigger a first display instruction. Upon detecting this first display instruction, the terminal device checks whether the clicked image is a private image. In this case, the clicked image becomes the first image. Here, a private image refers to an image containing private information.

[0088] In another possible scenario, while the terminal device is displaying a picture in a photo album application, the user can trigger a first display command by performing a picture switching operation, thereby instructing the terminal device to switch to displaying another picture. After detecting this first display command, the terminal device checks whether the picture to be switched to is a privacy picture containing private information. In this case, the picture to be switched to is the first picture.

[0089] It should be noted that the images in the aforementioned photo album application can include photos taken by the user through the terminal device's camera, images downloaded from the internet, images downloaded from other applications, and screenshots. Furthermore, the privacy information in the images can include characters, images, and other information that involves user privacy. For example, this could include ID photos, ID numbers, names, bank account information, etc.

[0090] In some embodiments, the terminal device may store a mapping relationship between image identifiers and privacy indication information. The privacy indication information indicates whether the image identified by the corresponding image identifier is a privacy image containing privacy information. Based on this, in response to a first display instruction, the terminal device can search for the privacy indication information corresponding to the image identifier of the first image from the mapping relationship. If the found privacy indication information corresponding to the image identifier of the first image includes a first indication value, then the first image is determined to be a privacy image. The first indication value indicates that the corresponding image is a privacy image. For example, the first indication value can be 1.

[0091] It should be noted that the image identifiers and corresponding privacy indications included in the above mapping relationship can be obtained by the terminal device after pre-identifying the images in the photo album application using a privacy recognition model. For example, the following section will use the first image as an example to describe the process of obtaining the mapping relationship between the image identifier and the privacy indication information of the first image. See [link to documentation]. Figure 4 The process may include the following steps:

[0092] S401: Based on the privacy recognition model, the first image is recognized to obtain the image recognition result.

[0093] A privacy-recognition model can be deployed in the terminal device. This model can be trained using multiple image samples. Each image sample includes pre-labeled privacy regions, which are areas containing privacy information. During training, an image sample is input into the model to be trained. The model infers the location of the corresponding image sample and outputs the predicted location of the privacy region in that image sample. Based on the predicted location and the labeled location of the privacy region, a loss function is calculated, and this loss function is used to backpropagate the model to update its parameters. Then, the next image sample is input into the updated model, and the above operation is repeated. In this way, the parameters of the model are continuously updated through multiple image samples, thereby training the privacy-recognition model.

[0094] In some embodiments, when a new image is detected in the photo album application, the terminal device can use a privacy recognition model to identify the newly added image, thereby obtaining the corresponding image recognition result. Based on this, when the terminal device detects that a first image has been added to the photo album application, it uses the privacy recognition model to identify the first image, thereby obtaining the image recognition result for the first image.

[0095] The image recognition result of the first image may include the location of one or more privacy regions in the first image. In this case, the image recognition result of the first image may also include the information type corresponding to the location of each privacy region, wherein the information type may indicate that the privacy information within the corresponding privacy region is characters or images. When the information type corresponding to a certain privacy region indicates that the privacy information within that privacy region is characters, the information type may also include the number of characters of the privacy information within that privacy region. Alternatively, if the image recognition result of the first image includes the location of one or more privacy regions in the first image, the image recognition result may also include the privacy information within each privacy region.

[0096] Optionally, the image recognition result of the first image may not include the location of the privacy region. In this case, the image recognition result of the first image may optionally include indication information to indicate that the first image does not contain a privacy region.

[0097] In other embodiments, the terminal device may also periodically recognize newly added images in the photo album application to obtain corresponding image recognition results. Based on this, after the first image is added to the photo album application, the terminal device can use a privacy recognition model to recognize at least one newly added image, including the first image, at the end of the current recognition cycle or the beginning of the next recognition cycle to obtain the image recognition result for that at least one newly added image.

[0098] In other embodiments, the terminal device can also identify newly added images in the photo album application between the current moment and the last time it entered the specified state when it detects that it has entered a specified state. Based on this, after the first image is added to the photo album application, if the terminal device detects that it has entered the specified state, it can identify the first image to obtain the identification result. The specified state can refer to an idle state or a non-busy state. For example, the terminal device can determine that it has entered an idle state when the screen is off; or, it can determine that it has entered an idle state when the terminal device detects that the CPU is in an idle state; or, it can be considered that the terminal device has entered a non-busy state when the terminal device detects that the CPU is running, but the current CPU load is less than a preset load threshold.

[0099] S402: Based on the image recognition results, generate and store the mapping relationship between the image identifier and privacy indication information of the first image.

[0100] If the image recognition result of the first image contains the location of one or more privacy regions, it indicates that there is a region containing privacy information in the first image, that is, the first image contains privacy information and is a privacy image. In this case, the terminal device can use the first indication value as the privacy indication information of the first image, and then generate and store the mapping relationship between the image identifier of the first image and the privacy indication information.

[0101] Optionally, if the image recognition result of the first image contains the location of one or more privacy regions, the terminal device may also generate and store a mapping relationship between the image identifier of the first image and the location of one or more privacy regions in the first image. Alternatively, the terminal device may directly generate and store a mapping relationship between the image identifier of the first image, privacy indication information, and the location of one or more privacy regions.

[0102] Optionally, when the image recognition result of the first image also includes the information type corresponding to each privacy region, the terminal device can also store the information type corresponding to each privacy region and the location of the corresponding privacy region.

[0103] Optionally, when the image recognition result of the first image also includes privacy information within each privacy area, the terminal device can also store the location of each privacy area corresponding to the privacy information within the corresponding privacy area.

[0104] Optionally, when the image recognition result of the first image does not include the information type and / or privacy information corresponding to the privacy region, the terminal device may also identify the privacy information in each privacy region based on the location of each privacy region using a character recognition algorithm such as optical character recognition (OCR). If a character is identified in any privacy region, the terminal device can determine the information type corresponding to that privacy region as a first type based on the recognition result. In this case, the first type indicates that the privacy information contained in the privacy region is a character. In addition, the first type can also indicate the number of characters in the privacy information. Then, the terminal device can store at least one of the first type and / or the identified characters corresponding to the location of the privacy region. If no character is identified in any privacy region, the terminal device can determine the information type corresponding to that privacy region as a second type based on the recognition result. In this case, the second type indicates that the privacy information in the privacy region is an image. Then, the terminal device can store at least one of the second type and the image in the privacy region corresponding to the location of the privacy region.

[0105] If the image recognition result of the first image does not contain the location of one or more privacy regions, or if the image recognition result of the first image contains indication information indicating that the first image does not contain privacy regions, then the terminal device can determine that the first image does not contain privacy information, that is, the first image is not a privacy image. In this case, the terminal device can use the second indication value as the privacy indication information for the first image, and then generate and store the mapping relationship between the image identifier of the first image and the privacy indication information. The second indication value is used to indicate that the corresponding image is not a privacy image. For example, the second indication value can be 0.

[0106] Table 1 illustrates the mapping relationship between an image identifier, privacy indication information, the location of a privacy region, and the information type of the privacy information contained in the terminal device. As shown in Table 1, when the privacy indication information is 1, it indicates that the image identified by the corresponding image identifier is a privacy image; when the privacy indication information is 0, it indicates that the image identified by the corresponding image identifier is not a privacy image. In this case, the location and information type of the privacy region corresponding to the image identifier are both null. When the information type corresponding to the location of the privacy region is "image," it indicates that the privacy information contained within the corresponding privacy region is an image; when the information type corresponding to the location of the privacy region is "text_n," it indicates that the privacy information contained within the corresponding privacy region consists of n characters.

[0107] Table 1. Mapping relationship between image identifiers, privacy indication information, location of privacy areas, and information type.

[0108]

[0109] It should be noted that the mapping relationship in the table above is only an example given in the embodiments of this application. In some possible cases, the mapping relationship may contain more or less content. Alternatively, the mapping relationship can be split into multiple mapping relationships with the image identifier as the index. For example, it can be split into a mapping relationship between the image identifier and the privacy indication information, and a mapping relationship between the image identifier, the location of the privacy area, and the corresponding information type.

[0110] The above mainly describes the process of generating the mapping relationship between image identifiers and privacy indication information stored in the terminal device, and the implementation process of the terminal device searching for the privacy indication information of the first image from this mapping relationship, and determining whether the first image is a privacy image based on the found privacy indication information. In some possible cases, the terminal device may fail to find the privacy indication information of the first image from the mapping relationship between image identifiers and privacy indication information. In this case, the terminal device can refer to the methods described in S401 and S402 above, use a privacy recognition model to identify the first image, and determine whether the first image is a privacy image containing privacy information based on the image recognition result. Afterwards, the terminal device can generate and store the mapping relationship between the image identifier and privacy indication information of the first image based on the image recognition result.

[0111] S302: If the first image is a private image, then activate the facial information collection device of the terminal device to collect the user's facial information and perform facial recognition on the collected user's facial information.

[0112] In this embodiment, the first image may be the first private image displayed after the user opens the photo album application, at which time the facial information collection device of the terminal device is turned off. Optionally, the first image may also be the image that the user switches to display when the facial information collection device of the terminal device is turned off. For example, if the image currently displayed by the terminal device does not contain private information, and the terminal device has turned off the facial information collection device, the first image is the next image to be displayed as indicated by the user. In this case, after determining that the first image is a private image, the terminal device activates the facial information collection device and begins to collect the user's facial information through the facial information collection device.

[0113] It should be noted that the facial information acquisition device of the terminal device may include the terminal device's camera and / or depth sensor. The camera may be a front-facing camera; the depth sensor may refer to a sensor used to acquire depth information, for example, the depth sensor may include at least one of a TOF sensor, a LiDAR sensor, and a laser sensor.

[0114] When the facial information acquisition device is a camera on the terminal device, the user's facial information acquired by the terminal device through the camera can be a facial image of the user. When the facial information acquisition device is a depth sensor such as a TOF sensor, LiDAR sensor, or laser sensor on the terminal device, the user's facial information acquired by the terminal device through the depth sensor can be the depth information of the user's face. When the facial information acquisition device includes both a camera and a depth sensor, the user's facial information acquired through the camera and depth sensor is a three-dimensional image of the user's face containing depth information.

[0115] After collecting the user's facial information, the terminal device can perform facial recognition on the collected user's facial information.

[0116] Specifically, the terminal device can detect how many faces are included in the currently collected first user facial information. If the first user facial information contains multiple faces, it can be determined that the first user facial information contains faces that are not the target face, that is, the first user facial information contains non-target faces. In this case, the terminal device can execute S303. If the first user facial information contains only one face, the terminal device can further detect whether the face included in the first user facial information is the target face. If it is not the target face, it is determined that the first user facial information contains a non-target face, and the terminal device can execute S303. Optionally, if the face included in the first user facial information is the target face, the terminal device can execute S304.

[0117] It should be noted that the terminal device may store information about target faces that the user has pre-defined permission to view private information in private images. This target face can be the face of the target user, for example, the face of the terminal device's owner. Therefore, when detecting whether a face in the first user's facial information is the target face, the terminal device can perform feature comparison between the information of the face in the first user's facial information and the stored target face information to determine whether the face in the first user's facial information is the target face.

[0118] Optionally, in some embodiments, after determining that the first image is a private image, but before completing face recognition, the terminal device can first hide the private information in the first image and then display the first image with the private information hidden. Afterwards, once face recognition is complete, the terminal device can further determine whether to execute S303 or S304 based on the face recognition result; that is, to decide whether to continue displaying the first image with the private information hidden or to restore the display of the private information in the first image. This not only ensures timely response to the user's display commands but also guarantees that the private information in the first image is not leaked when displaying the first image during the facial information collection and face recognition process.

[0119] Specifically, in one example, as described above, the terminal device identifies the location of privacy regions in the first image using a privacy recognition model and stores the mapping relationship between the image identifier of the first image and the location of the privacy regions. Based on this, the terminal device can obtain the location of one or more privacy regions in the first image from this mapping relationship based on the image identifier of the first image. Then, the terminal device can perform privacy information hiding processing on one or more privacy regions based on their locations, thereby obtaining the first image with the privacy information hidden.

[0120] The terminal device can determine one or more privacy regions in the first image based on their locations. Then, each privacy region is occluded to hide the privacy information within it. For example, the terminal device can use a mosaic algorithm to add mosaic effects to the privacy regions, overlay a solid-color image on them, or apply Gaussian blur to the regions, thus blurring the privacy regions and making the privacy information invisible, and so on.

[0121] Optionally, if the terminal device also stores the information type corresponding to the location of each privacy region, then when the information type corresponding to any privacy region is a character, the terminal device can also draw a privacy character in the occluded privacy region to indicate that the privacy information contained in the privacy region is a character. Optionally, if the information type corresponding to the privacy region also includes a number of characters, then the number of privacy characters drawn by the terminal device in the privacy region can be equal to the number of characters included in the information type.

[0122] For example, see Figure 5The left image shows a photo of an ID card. This first image includes five privacy areas: privacy area 50 contains the user's name, privacy area 51 contains the user's date of birth, privacy area 52 contains the user's address, privacy area 53 contains the user's ID number, and privacy area 54 contains the user's photo. Based on this, the terminal device can occlude each privacy area. Then, in privacy areas 51 to 53, a corresponding number of privacy characters "*" are drawn according to the number of characters contained within. The processed first image is shown below. Figure 5 As shown in the right figure.

[0123] In another example, if the location of the privacy area in the first image is not stored in the terminal device, the terminal device can also directly obscure the first image to hide the privacy information in the first image.

[0124] S303: If it is identified that the first user's facial information currently collected contains a non-target face, then the first image after hiding the privacy information is displayed.

[0125] In one possible scenario, if the terminal device has already hidden the privacy information in the first image before completing face recognition, then after determining that the first user's facial information contains a non-target face, that is, after determining that there is a non-target user among the users currently viewing the first image, the terminal device can continue to display the first image with the privacy information hidden.

[0126] In another possible scenario, if the terminal device does not display the first image before completing face recognition, then after determining that the first user's facial information contains a non-target face, the terminal device can hide the privacy information in the first image to obtain a first image with the privacy information hidden, and then display the first image with the privacy information hidden. The implementation process of the terminal device hiding the privacy information in the first image can be found in the relevant description in S302, and will not be repeated here.

[0127] Figure 6 This is a rendering of a scenario in which the first image is displayed, as shown in an embodiment of this application. Figure 6 As shown, when multiple people view the photo album at the same time, the terminal device will recognize multiple faces by collecting the users' facial information. Accordingly, when displaying the first picture, the terminal device hides the privacy information in the first picture.

[0128] Figure 7 This is a rendering showing the effect of displaying the first image in another scenario as illustrated in an embodiment of this application. Figure 7As shown, when only one person is viewing the album and this person is not the owner of the terminal device, the terminal device recognizes a face by collecting the user's facial information, and determines that the face is not the target face by comparison. Accordingly, the terminal device hides the privacy information in the first image when displaying the first image.

[0129] S304: If the currently collected first user facial information is identified to contain a face and the face is the target face, then the first image is displayed.

[0130] In one possible scenario, if the terminal device has already hidden the privacy information in the first image before displaying it, then after determining that the first user's facial information contains a face and that the face is the target face, that is, after determining that the user currently viewing the first image is the target user, the terminal device can restore the display of the privacy information in the first image, that is, fully display the first image.

[0131] In another possible scenario, if the terminal device does not display the first image before completing face recognition, then after determining that the first user's facial information contains a face and that the face is the target face, the terminal device directly displays the first image. That is, it does not hide the privacy information in the first image and displays the content of the first image in its entirety.

[0132] In this embodiment, when a display command for indicating the display of a first image is detected, the terminal device first checks whether the first image is a privacy image containing privacy information. If the first image is a privacy image, the facial information collection device is then activated to collect the user's facial information and perform face recognition. Furthermore, if it is determined that a non-target user is browsing the image, the privacy information in the image is hidden. In this way, while improving the security of privacy information in the image, the device's power consumption is reduced during the user's image browsing process.

[0133] In addition, in the embodiments of this application, when the terminal device recognizes multiple faces, it is not necessary to compare whether the target face is included. Instead, it can directly display the image after hiding the privacy information. This not only reduces the risk of privacy information leakage in the scenario of multiple people browsing images, but also reduces the power consumption of the device.

[0134] Finally, in this embodiment, when the terminal device hides the privacy information in the privacy image, it can accurately locate the privacy area based on the identified location of the privacy area, and then selectively occlude the privacy area, while displaying the non-privacy area as usual, thus achieving precise protection of the privacy information in the image.

[0135] Based on the method described in the above embodiments, if the terminal device executes S303, the user browsing the first image may change during the process of displaying the first image after hiding the privacy information. In this case, the terminal device can also execute... Figure 8 S801 and S802 are shown.

[0136] S801: During the process of displaying the first image after hiding privacy information, the user's facial information is continuously collected through the facial information collection device, and facial recognition is performed on the collected user facial information.

[0137] During the display of the first image after hiding privacy information, the user viewing the first image may change. In this case, the terminal device can continuously collect the user's facial information through the facial information collection device from the moment the first image after hiding privacy information is displayed, and perform facial recognition on the collected user's facial information to detect whether the person viewing the first image has changed.

[0138] In this embodiment, after executing S302, the terminal device can keep the facial information collection device constantly enabled. Based on this, in one possible implementation, the terminal device can collect the user's facial information via the facial information collection device at preset time intervals, starting from the moment the first image after hiding privacy information is displayed, and perform facial recognition on the collected user facial information. The preset time interval is the interval at which the facial information collection device collects information.

[0139] In another possible implementation, the terminal device can also collect user facial information via a facial information collection device at preset time intervals, starting from the moment the first image after hiding the privacy information is displayed. Starting with the first collected user facial information, the terminal device can perform face recognition every m user facial information entries. For example, taking m = 2, starting from the moment the first image after hiding the privacy information is displayed, the terminal device performs face recognition on the first user facial information collected by the facial information collection device. No face recognition is performed on the second and third user facial information entries collected by the facial information collection device. Then, face recognition is performed again on the fourth collected user facial information entry, and so on, with the terminal device performing face recognition every two user facial information entries.

[0140] It should be noted that the introduction of the facial information collection device and the process of the terminal device performing facial recognition on the user's facial information can be referred to S302 in the aforementioned embodiment, and will not be repeated here.

[0141] S802: If it is identified that the currently collected second user facial information contains a face, and the face contained is the target face, then the privacy information in the first image is restored and displayed.

[0142] During the process of continuously collecting user facial information through a facial information collection device and performing facial recognition on the collected user facial information, if at some point the terminal device recognizes that the currently collected second user facial information contains a face and that face is the target face, it means that the user currently browsing the image no longer includes anyone other than the target user. In this case, the terminal device can restore the display of the privacy information in the first image.

[0143] Of course, if the terminal device recognizes that the currently collected second user facial information contains a face that is not the target face, or if the second user facial information contains multiple faces, it means that there are still non-target users among the users currently browsing the image. In this case, the terminal device can continue to display the first image after hiding the privacy information.

[0144] The above-described S801 and S802 mainly describe how, during the process of displaying the first image after hiding the privacy information, the user browsing the first image is detected to see if there is any change, and then a decision is made on whether to restore the display of the privacy information in the first image. In some possible cases, if the terminal device executes S304 in the aforementioned embodiment, during the process of fully displaying the first image, the terminal device can also refer to the method described in S801 to continuously collect the user's facial information through the facial information collection device and perform face recognition on the collected user's facial information. If at some point the terminal device recognizes that the currently collected third user facial information contains a non-target face, for example, it contains multiple faces, or it contains one face that is not the target face, then the terminal device can display the first image after hiding the privacy information.

[0145] Figure 9 This is a schematic diagram illustrating how, during the process of displaying a first image after hiding privacy information, the display state of the hidden information is switched based on the perceived user during an embodiment of this application. Figure 9 As shown in the left-middle image, when the terminal device detects the first display command, multiple people are simultaneously viewing the image. Therefore, the first user facial information collected by the terminal device contains multiple faces. In this case, the terminal device hides the privacy information in the first image before displaying it. During the process of displaying the first image with the privacy information hidden, the terminal device continuously collects user facial information. At a certain moment, such as... Figure 9As shown in the right image, the user browsing the first image has become the target user alone. Correspondingly, the facial information of the second user collected by the terminal device contains a face that is the target face. In this case, the terminal device restores the display of the hidden information in the first image.

[0146] In this embodiment, during the process of displaying the first image with hidden privacy information, the terminal device can continuously collect the user's facial information and perform facial recognition through a facial information collection device to detect whether the user viewing the first image has changed, and then decide whether to restore the display of the privacy information in the first image. This ensures that when the user viewing the first image becomes the target user, the target user can fully view the content of the first image.

[0147] In addition, while displaying the complete first image, the terminal device can continuously collect the user's facial information and perform facial recognition through a facial information collection device. This allows it to detect whether the user viewing the first image has changed, and thus decide whether to hide the private information in the first image. This ensures that if the user viewing the first image changes from a target user to a non-target user, or if a non-target user is added, the private information in the first image can be hidden promptly, reducing the risk of privacy leaks.

[0148] As can be seen from the above embodiments, during the process of displaying a privacy image with hidden privacy information or displaying a complete privacy image, the terminal device can continuously collect the user's facial information through a facial information collection device to detect whether the user browsing the image has changed. Furthermore, during the display of the privacy image, the user may switch to displaying other images; in this case, the terminal device can also perform... Figure 8 S803 to S805 are shown.

[0149] S803: In response to the second display instruction, detect whether the second image is a private image. The second display instruction is used to instruct the switching display of the second image.

[0150] In this embodiment, after viewing the currently displayed image, a user may want to switch to another image. In this case, the user can trigger a second display instruction through an image switching operation, thereby instructing the terminal device to switch to displaying another image. For example, the user can swipe the currently displayed image to trigger the second display instruction, thereby instructing the terminal device to switch to displaying the previous or next image. Based on this, after detecting the second display instruction, the terminal device checks whether the image to be switched to is a privacy image containing privacy information. At this time, the image to be switched to indicated by the second display instruction is the second image. The implementation method of the terminal device detecting whether the image to be switched to is a privacy image containing privacy information can refer to the relevant implementation method described in S301 of the aforementioned embodiment, and will not be repeated here.

[0151] S804: If the second image is not a private image and is an image switched from a private image, then the second image is displayed, and the user's facial information is continuously collected through the facial information collection device.

[0152] If it is determined that the second image is not a private image, the terminal device can display the complete second image.

[0153] It should be noted that, as described above, during the display of the privacy image, the terminal device can continuously collect the user's facial information through a facial information collection device to detect whether the user viewing the privacy image has changed. Based on this, if the second image is a switched image from a currently displayed privacy image, for example, a switched image from a currently displayed first image, the terminal device can continue to collect the user's facial information through the facial information collection device after switching to the second image. Optionally, during the collection of user facial information, facial recognition of the collected user facial information can be stopped. In addition, the terminal device can set a first timer from the moment the second image begins to be displayed, that is, from the first moment, and the duration of the first timer is a first duration. For example, the first duration is 2 seconds.

[0154] S805: If no display instruction for any privacy image is detected within a first duration starting from the first moment, the facial information collection device is turned off to stop collecting user facial information. The first moment is the moment when the second image is started to be displayed.

[0155] If the terminal device does not detect any display instruction for any private image during the first timer's operation, it will stop collecting user facial information through the facial information collection device when the first timer expires, thereby reducing device power consumption.

[0156] In one possible scenario, during the execution of the first timer, the terminal device does not detect any display instructions for any other image; that is, the terminal device is continuously displaying the second image. In this case, when the first timer reaches its duration, the terminal device can turn off the facial information collection device to stop collecting the user's facial information.

[0157] In another possible scenario, during the first timer's execution, the terminal device detects a display command for another image, but the display command indicates that the other image to be displayed is not a private image. In this case, the terminal device switches to displaying the other image based on the display command. When the first timer reaches its duration, the terminal device disables the facial information collection device to stop collecting the user's facial information.

[0158] Optionally, during the execution of the first timer, if the terminal device detects a display instruction for any privacy image, the terminal device can refer to the method for displaying the first image described in the foregoing embodiments, collect the user's facial information through a facial information collection device, and perform face recognition. Then, based on whether the collected user facial information contains a non-target face, it determines how to display the corresponding privacy image according to the display instruction, and continues to collect user facial information during the display of the privacy image.

[0159] In this embodiment, if the terminal device detects an instruction to switch to displaying a second image that does not contain privacy information while displaying a privacy image, it continues to collect user facial information for a certain period of time starting from the switch to displaying the second image. If the user does not switch to displaying a privacy image within this period, the terminal device can turn off the facial information collection device to stop collecting user facial information. This reduces device power consumption caused by continuous collection of user facial information. Furthermore, compared to stopping facial information collection immediately upon switching to the second image or upon detecting that the second image is not a privacy image, this embodiment maintains facial information collection for a certain period of time from the switch to displaying the second image, avoiding increased device power consumption caused by frequent switching of the facial information collection device when the user switches from the second image to other privacy images in a very short time.

[0160] Based on the foregoing embodiments, in one possible scenario, during the process of the terminal device displaying the first image after hiding privacy information, if multiple people are viewing the first image after hiding privacy information, and these multiple people may include the target user, and the target user may want to show the privacy information in the first image to others, in this case, the terminal device can... Figure 10 The steps S1001 and S1002 shown herein are for displaying the privacy information in the first image to others.

[0161] S1001: During the process of displaying the first image after hiding privacy information, in response to the first operation instruction, it is determined whether the most recently collected user facial information contains a target face, and the first operation instruction is used to instruct the display of privacy information in at least one privacy area in the first image.

[0162] In one possible implementation, the first image includes one or more privacy areas. When a user is viewing the first image after hiding privacy information, if they want to view the privacy information within a target privacy area in the first image, they can click on the target privacy area to trigger a first operation instruction, thereby instructing the terminal device to display the privacy information within the target privacy area.

[0163] In another possible implementation, while displaying the first image with hidden privacy information, the terminal device can also display an unhide control on the current display interface. If a user wants to view the privacy information in the first image, they can click the unhide control to trigger a first operation command, instructing the terminal device to display the privacy information within the first image. Alternatively, the user can perform a preset first gesture operation on the interface displaying the first image to trigger the first operation command, thereby instructing the terminal device to display the privacy information within the first image.

[0164] After detecting the first operation command, the terminal device can determine whether the most recently collected user facial information contains the target face, thereby determining whether the users browsing the first image include the target user.

[0165] The terminal device can acquire the user's facial information from the most recent collection and extract information about one or more faces contained in the user's facial information. Then, it compares the information of each extracted face with the information of the target face to determine whether the user's facial information contains the target face.

[0166] Optionally, for the most recently collected user facial information, if the terminal device has already performed face recognition on the user's facial information and stored the face recognition result when it collected the user's facial information, the terminal device can also directly obtain the face recognition result of the most recently collected user facial information and determine whether the user's facial information contains the target face based on the face recognition result.

[0167] It should be noted that during the display of the first image after hiding privacy information, if the terminal device is continuously collecting user facial information through the facial information collection device, then if the current moment is the time when the user's facial information is being collected, the most recently collected user facial information will be the user facial information collected at the current moment; if the current moment is not the time when the user's facial information is being collected, the most recently collected user facial information will be the user facial information collected before the current moment. Optionally, during the display of the first image after hiding privacy information, if the terminal device is not continuously collecting user facial information through the facial information collection device, then the most recently collected user facial information can be the first user facial information or the user facial information collected by the terminal device when it activates the facial information collection device at the current moment.

[0168] S1002: If the target face is included in the most recently collected user facial information, display privacy information in at least one privacy area.

[0169] If the first operation instruction is used to instruct the terminal device to display privacy information within the target privacy area, then after determining that the target face is included in the most recently collected user facial information, the terminal device will restore the display of privacy information within the target privacy area, while privacy information in other privacy areas of the first image other than the target privacy area can remain hidden.

[0170] For example, see Figure 11 In the left image, if a user wants to view the privacy information in privacy area 1100 of the first image, the user can click on privacy area 1100. After detecting this operation command, the terminal device determines that the most recently collected user facial information contains the target face, that is, the user currently viewing the first image has a target user. At this time, the terminal device can restore the display of the privacy information in privacy area 1100, while the privacy information in other privacy areas remains hidden, such as... Figure 11 As shown in the right figure.

[0171] Optionally, after the terminal device restores the display of the privacy information within the target privacy area, if the user subsequently wants to restore the privacy information within the target privacy area to a hidden state, the user can click on the target privacy area again. The terminal device can hide the privacy information within the target privacy area after detecting the user's click instruction.

[0172] If the first operation instruction is used to instruct the terminal device to display all the privacy information in the first image, then after determining that the target face is included in the most recently collected user facial information, the terminal device displays the complete first image. At this time, the privacy information in each privacy area of ​​the first image will be restored to display.

[0173] For example, see Figure 12 The left image shows a control 120 that can unhide the first image. When a user wants to view the privacy information in the first image, they can click this control 120. After detecting this operation command, the terminal device determines that the most recently collected user facial information contains the target face, meaning the user currently viewing the first image is the target user. At this point, the terminal device can restore the display of the privacy information in each privacy area, i.e., such as... Figure 12 The right image in the image shows the complete first image.

[0174] Optionally, while the terminal device is displaying the first image in its entirety, it can also display a hidden control on the interface displaying the first image. Subsequently, if the user wants to restore the privacy information in the first image to a hidden state, the user can click the hidden control. After detecting this operation command, the terminal device can display the first image with the privacy information hidden. Alternatively, the user can perform a preset second gesture operation on the interface displaying the first image, and after detecting this operation command, the terminal device can display the first image with the privacy information hidden.

[0175] It should be noted that if the terminal device is continuously collecting user facial information and performing face recognition through a facial information collection device while displaying the first image after hiding the privacy information, the terminal device can continue to collect user facial information through the facial information collection device after the privacy information in at least one privacy area in the first image is restored to display. However, it can stop performing face recognition until it detects an instruction to switch to displaying other images or until it detects that the privacy information in at least one privacy area in the first image is restored to be hidden, and then resume the operation in the corresponding scenario.

[0176] In this embodiment, during the display of a first image with hidden privacy information, if a user instruction is detected to display privacy information within at least one privacy area of ​​the first image, the terminal device can perform user authentication based on the most recently collected user facial information. If it is determined that the user's facial information contains a target face, that is, if it is determined that the target user is among the users currently viewing the first image, the terminal device can restore the display of the privacy information within the at least one privacy area. Thus, in scenarios where multiple people, including the target user, are viewing the first image, the target user can flexibly choose to display the hidden privacy information to others.

[0177] Based on the foregoing embodiments, in some possible scenarios, for private images containing privacy information in a photo album application, users may want to hide the privacy information by default whenever any user views the private image. Based on this, this application also provides an image display method, see [link to relevant documentation]. Figure 13 The process may include the following steps:

[0178] S1301: In response to the second operation instruction, determine whether the most recently collected user facial information contains a target face. The second operation instruction is used to instruct that the privacy information in the privacy image be hidden by default when displaying the privacy image.

[0179] In one possible implementation, when the terminal device is currently displaying an image—for example, if a user wants to directly hide private information in all private images within the photo album application—the user can directly perform a preset third gesture operation on the interface displaying the first image to trigger the second operation command. Alternatively, the terminal device can display a globally hidden control on the image display interface. Based on this, the user can click the globally hidden control on the interface displaying the first image to trigger the second operation command.

[0180] In another possible implementation, if the terminal device is currently displaying an image (e.g., the first image is displayed), and the user wants to directly hide the privacy information in all private images within the photo album application, the user can return to the photo album application interface and perform a preset third gesture to trigger the second operation command. Alternatively, the terminal device can display a globally hidden control within the photo album application interface. Based on this, after returning to the photo album application interface, the user can click on the globally hidden control to trigger the second operation command.

[0181] After detecting the second operation command, the terminal device can determine whether the most recently collected user facial information contains the target face, thereby determining whether the user who triggered the second operation command is the target user with global hiding permissions. The implementation method for the terminal device to determine whether the most recently collected user facial information contains the target face can be referred to the description in S1001 above.

[0182] It should be noted that in some possible scenarios, when the terminal device is currently displaying the photo album application interface, if the user wants to directly hide the private information in individual private photos within the photo album application, the user can perform a preset third gesture operation within the photo album application interface to trigger the second operation command. Alternatively, if the terminal device displays a global hide control within the photo album application interface, the user can directly click on the global hide control to trigger the second operation command. After detecting the second operation command, the terminal device can collect the user's facial information through a facial information collection device and perform facial recognition on the collected user facial information to determine whether the user's facial information contains the target face.

[0183] S1302: If the target face is included in the most recently collected user facial information, then when a display instruction for any privacy image is detected, the privacy image with the privacy information hidden will be displayed.

[0184] In one possible implementation, if the user triggers the second operation command within the display interface of the first image, after determining that the target face is included in the most recently collected user facial information, the terminal device can hide the privacy information in the currently displayed first image if the privacy information is not hidden. Subsequently, when the terminal device detects a display command for any image, if the image is a privacy image, the terminal device will no longer collect user facial information through the facial information collection device, nor will it perform facial recognition; instead, it will directly display the image with the privacy information hidden.

[0185] It should be noted that if the terminal device is continuously collecting the user's facial information through the facial information collection device while displaying the first image, and after determining that the target face is included in the most recently collected facial information, the terminal device can turn off the facial information collection device to stop collecting the user's facial information because the facial information collection device is in the on state. If the terminal device's facial information collection device is in the off state, the terminal device can keep the facial information collection device in the off state.

[0186] In another possible implementation, if the user triggers the second operation command in the interface of the photo album application, after determining that the target face is included in the most recently collected user facial information, when the terminal device detects a display command for any image, if the image is a private image, the terminal device will not collect the user's facial information or perform facial recognition, but will directly display the image after hiding the private information.

[0187] Optionally, after determining that the target face is included in the most recently collected user facial information, if the facial information collection device of the terminal device is in an on state, the terminal device can directly turn off the facial information collection device to stop the collection of user facial information. If the facial information collection device of the terminal device is in a off state, then the facial information collection device remains in a off state.

[0188] In some embodiments, after the terminal device enables the global hiding function through the above method based on the second operation instruction, the user can also cancel the global hiding function by triggering a third operation instruction when the global hiding function is no longer needed.

[0189] Specifically, users can refer to the methods for triggering the second operation command described above. On the terminal device's screen displaying an image, the user can trigger the third operation command by performing a preset fourth gesture or clicking the control to unhide the image globally. Alternatively, the user can trigger the third operation command on the terminal device's screen displaying a photo album application by performing a preset fourth gesture or clicking the control to unhide the image globally. Correspondingly, after detecting the third operation command, the terminal device can perform user authentication based on the most recently collected user facial information. If the most recently collected user facial information contains the target face, the terminal device can cancel the global hiding function and continue displaying the image according to the implementation methods described in the aforementioned embodiments.

[0190] The above embodiments mainly describe the process of enabling a user to enable the global hiding function to directly hide private information in private images in a photo album application. In some possible scenarios, users may not need to hide the private information in private images in a photo album application. In this case, users can refer to the previously described method of triggering the fourth operation command, and trigger the fourth operation command by performing a preset fifth gesture operation or clicking the globally displayed control in the interface of a certain image displayed on the terminal device or in the interface of the photo album application. Accordingly, after detecting the fourth operation command, the terminal device can determine whether the most recently collected user facial information contains the target face. If the most recently collected user facial information contains the target face, then when the terminal device detects a display command for any image, it will no longer check whether the image contains private information and will directly display the image. Furthermore, if the terminal device detects the fourth operation command in the interface of displaying a private image with hidden private information, the terminal device can also restore the display of the private information in the private image. Of course, after enabling the global display function, users can also cancel the global display function by triggering the fifth operation command. Specifically, users can refer to the aforementioned method of triggering the second operation command. On the terminal device's interface displaying an image or in the photo album application's interface, the user can trigger the fifth operation command by performing a preset sixth gesture or clicking the control to cancel global display. Correspondingly, after detecting the fifth operation command, the terminal device determines whether the most recently collected user facial information contains the target face. If the most recently collected user facial information contains the target face, the terminal device can cancel the global display function and continue displaying the image according to the implementation methods described in the aforementioned embodiments.

[0191] In this embodiment, after detecting the second operation command, the terminal device can authenticate the user based on the most recently collected user facial information. Once authentication is successful, upon detecting a subsequent command to display a private image, it will no longer collect user facial information or perform face recognition; instead, it will directly hide the private information in the image before displaying it. This allows users to flexibly choose whether to hide the private information in the image based on their own needs.

[0192] The image display device provided in the embodiments of this application will be described next.

[0193] Figure 14 This is a schematic diagram of the structure of an image display device provided in an embodiment of this application. This image display device can be deployed in a terminal device, such as... Figure 14 As shown, the image display device 1400 includes a detection module 1401, a face recognition module 1402, and a display module 1403.

[0194] The detection module 1401 is used to execute S301 in the aforementioned embodiment; the face recognition module 1402 is used to execute S302 in the aforementioned embodiment; and the display module 1403 is used to execute S303 or S304 in the aforementioned embodiment.

[0195] Optionally, during the process of displaying the first image after hiding the privacy information by the display module 1403, the face recognition module 1402 is also used to continuously collect the user's facial information through the facial information collection device and perform face recognition on the collected user facial information; the display module 1403 is also used to restore the display of the privacy information in the first image if it is recognized that the currently collected second user facial information contains a face and the face contained is the target face.

[0196] Optionally, the image display device further includes a control module. In this regard, the detection module 1401 is further configured to, in response to a second display instruction, detect whether the second image is a private image, the second display instruction being used to instruct the switching display of the second image; the display module 1403 is further configured to, if the second image is not a private image and the second image is an image switched from a private image, display the second image and trigger the control module to control the facial information collection device to continuously collect user facial information; the control module is further configured to, if no display instruction for any private image is detected within a first time period starting from the first moment, control the facial information collection device to shut down to stop collecting user facial information, the first moment being the moment when the second image begins to be displayed.

[0197] Optionally, during the process of displaying the first image with hidden privacy information by display module 1403, face recognition module 1402 is further configured to respond to a first operation instruction to determine whether the most recently collected user facial information contains a target face. The first operation instruction is configured to instruct the display of privacy information in at least one privacy area of ​​the first image, where the privacy area is the area containing privacy information. Display module 1403 is further configured to display the privacy information in at least one privacy area if the most recently collected user facial information contains a target face.

[0198] Optionally, after the display module 1403 displays the first image with the privacy information hidden, the face recognition module 1402 is further configured to respond to the second operation instruction to determine whether the most recently collected user facial information contains a target face. The second operation instruction is used to instruct that the privacy information in the privacy image be hidden by default when displaying the privacy image. The display module 1403 is further configured to, if the most recently collected user facial information contains a target face, display the privacy image with the privacy information hidden when a display instruction for any privacy image is detected.

[0199] Optionally, the face recognition module 1402 is specifically used to determine that the first user's face information contains a non-target face if it is recognized that the first user's face information contains multiple faces; or, if it is recognized that the first user's face information contains a single face and the face contained therein is not the target face, then it is determined that the first user's face information contains a non-target face.

[0200] Optionally, the first image is the first private image displayed after launching the photo album application.

[0201] Optionally, the image display device 1400 further includes a privacy recognition module and a storage module. The privacy recognition module is used to recognize the first image based on a privacy recognition model to obtain an image recognition result. The storage module is used to generate and store a mapping relationship between the image identifier of the first image and privacy indication information based on the image recognition result. The privacy indication information is used to indicate whether the corresponding image is a privacy image. Based on this, the detection module 1401 is specifically used to obtain the privacy indication information of the first image from the mapping relationship based on the image identifier of the first image. If the privacy indication information of the first image includes a first indication value, then the first image is determined to be a privacy image.

[0202] Optionally, the image recognition result includes the location of one or more privacy regions in the first image, and the mapping relationship also includes the location of one or more privacy regions. The image display device is further used to obtain the location of one or more privacy regions from the mapping relationship based on the image identifier of the first image; and to hide the privacy information in one or more privacy regions based on the location of the one or more privacy regions to obtain the first image after hiding the privacy information.

[0203] In this embodiment, when a display command for indicating the display of a first image is detected, the terminal device first checks whether the first image is a privacy image containing privacy information. If the first image is a privacy image, the facial information collection device is then activated to collect the user's facial information and perform facial recognition. Furthermore, if it is determined that the user is not the target user browsing the image, the privacy information in the image is hidden. In this way, the security of privacy information in the image can be improved with lower device power consumption during the user's image browsing process.

[0204] It should be noted that the module division in the image display device provided in the above embodiments is illustrative and only represents one logical functional division. In actual implementation, other division methods may also be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0205] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal device) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] Furthermore, the image display device and image display method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0207] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0208] In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be referenced mutually. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. In the textual description of the embodiments of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. In this application, "first," "second," and various numerical designations are only for ease of description and are not used to limit the scope of the embodiments of this application. For example, they are used to distinguish different messages, rather than to describe a specific order or sequence.

[0209] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The order of the process numbers does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0210] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for displaying an image, characterized in that, Applied to a terminal device, the method includes: In response to a first display instruction, it is detected whether the first image is a private image. The first display instruction is used to instruct the display of the first image, and the private image refers to an image containing private information. If the first image is the privacy image, then the facial information collection device of the terminal device is activated to collect the user's facial information, and facial recognition is performed on the collected user's facial information. If the currently collected first user facial information is identified to contain a non-target face, then the first image after hiding the privacy information is displayed.

2. The method according to claim 1, characterized in that, The method further includes: During the process of displaying the first image after hiding privacy information, the facial information collection device continuously collects the user's facial information and performs facial recognition on the collected user facial information. If the currently collected second user facial information is identified to contain a face, and the face contained is the target face, then the privacy information in the first image is restored and displayed.

3. The method according to claim 2, characterized in that, The method further includes: In response to a second display instruction, it is detected whether the second image is the privacy image, wherein the second display instruction is used to instruct the display of the second image. If the second image is not the privacy image and the second image is an image switched from the privacy image, then the second image is displayed, and the user's facial information is continuously collected through the facial information collection device; If no display instruction for any of the privacy images is detected within a first time period starting from the first moment, the facial information collection device is turned off to stop collecting user facial information. The first moment is the moment when the second image begins to be displayed.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: During the process of displaying the first image after hiding privacy information, in response to the first operation instruction, it is determined whether the most recently collected user facial information contains a target face. The first operation instruction is used to instruct the display of privacy information in at least one privacy area in the first image, and the privacy area is the area containing privacy information. If the target face is included in the most recently collected user facial information, then at least one privacy information in the privacy area is displayed.

5. The method according to any one of claims 1 to 4, characterized in that, After displaying the first image with the hidden privacy information, the method further includes: In response to a second operation instruction, it is determined whether the most recently collected user facial information contains a target face. The second operation instruction is used to instruct that the privacy information in the privacy image be hidden by default when the privacy image is displayed. If the target face is included in the most recently collected user facial information, then when a display instruction for any of the privacy images is detected, the privacy image with the privacy information hidden is displayed.

6. The method according to any one of claims 1 to 5, characterized in that, If multiple faces are identified in the first user's facial information, then it is determined that the first user's facial information contains a non-target face; or, If the first user's facial information is identified as containing a face that is not the target face, then it is determined that the first user's facial information contains a non-target face.

7. The method according to any one of claims 1 to 6, characterized in that, The first image is the first private image displayed after launching the photo album application.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Based on the privacy recognition model, the first image is recognized to obtain the image recognition result; Based on the image recognition results, a mapping relationship between the image identifier of the first image and privacy indication information is generated and stored, wherein the privacy indication information is used to indicate whether the corresponding image is the privacy image; The detection of whether the first image is a private image includes: Based on the image identifier of the first image, the privacy indication information of the first image is obtained from the mapping relationship; If the privacy indication information of the first image includes a first indication value, then the first image is determined to be the privacy image.

9. The method according to claim 8, characterized in that, The image recognition result includes the location of one or more privacy regions in the first image, and the mapping relationship also includes the location of the one or more privacy regions. Before displaying the first image after hiding the privacy information, the method further includes: Based on the image identifier of the first image, the location of the one or more privacy regions is obtained from the mapping relationship; Based on the location of the one or more privacy regions, the privacy information in the one or more privacy regions is hidden to obtain the first image after the privacy information is hidden.

10. A terminal device, characterized in that, The terminal device includes a processor, which is configured to execute at least one program instruction or code stored in a memory to implement the image display method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer device, cause the computer device to perform the image display method according to any one of claims 1 to 9.

12. A computer program product containing instructions, characterized in that, When the instructions are executed by a computer device, the computer device performs the image display method as described in any one of claims 1 to 9.

Images