Apparatus and method for processing visual content

Abstract

The present disclosure relates to an apparatus and method for processing visual content to determine whether target information displayed on one or more reflective surfaces present in the visual content is to be deleted. And if it is determined that the target information is displayed on the one or more reflecting surfaces, providing prompt information to a user. The prompt information indicates that the target information is displayed on the one or more reflecting surfaces, and requests the user to confirm whether to delete the target information by using an editing operation. Therefore, the existence of the target information in the visual content can be notified to the user in time, so that the user can decide whether to reserve the target information or delete the target information from the visual content. In this way, the risk that visual information leaks through one or more reflecting surfaces existing in the visual content can be reduced.

Description

Technical Field

[0001] The present disclosure relates generally to the field of data processing, and more particularly to an apparatus and method for processing visual content to determine whether to delete target information displayed on one or more reflective surfaces present in the visual content. Background Art

[0002] The advent of the Internet has greatly improved the speed and efficiency of information sharing. In recent years, visual information sharing has become one of the most popular Internet services, such as sharing photos or videos through social networking sites or mobile applications, or sharing videos during video calls. With the popularity of visual information sharing, information security issues have become increasingly prominent. For example, video conferencing has become the number one cause of data breaches worldwide in recent years.

[0003] A special type of information leakage risk is caused by visual reflections. Reflections, or in other words, reflective surfaces, are often present in our daily life environment. Some examples of reflective surfaces include mirrors, glasses, windows, sunglasses, screens, polished metal or ceramic surfaces, and even water spots on the floor. Reflective surfaces may accidentally capture some visual information from surrounding objects and cause unexpected information leakage. For example, the glasses of a user participating in a video conference may reflect objects displayed on his computer screen, which may include trade secrets, personal ID / bank cards or hard copies of business contracts, or may reflect the user typing passwords on the keyboard. As another example, a mirror displayed in a selfie photo may reflect other people's faces or product prototypes that should not be exposed to the public, or a window presented in the camera's field of view may reflect structures and ongoing activities in the office. In all these and other cases, when certain visual information is shared between users, reflective surfaces pose a significant risk of information leakage, which in turn may cause great personal or commercial damage to the users (for example, the service provider may lose the trust of users and eventually lose its market share).

[0004] This problem becomes more serious when considering more common reflection situations that are less noticeable to the user and harder to avoid. For example, reflections off the outer surface of a coffee cup or shiny plastic bag might be able to tell if a person / object is nearby. In extreme cases, even diffuse reflections off a white wall can be used to reconstruct a light field.

[0005] Therefore, a technical solution is needed that enables people to reduce the risk of leaking information through visual reflections. Summary of the invention

[0006] The present invention summary briefly introduces some concepts, which will be further described in the detailed description. The present invention summary is not intended to identify the key features of the present disclosure, nor is it intended to limit the scope of the present disclosure.

[0007] The purpose of the present disclosure is to provide a technical solution that can reduce the risk of information leakage through visible reflections present in visual content.

[0008] The above objects are achieved by the features of the independent claims in the attached claims. Further embodiments and examples are apparent from the dependent claims, the description and the drawings.

[0009] According to a first aspect, a device for processing visual content is provided. The device includes a memory and a processor coupled to the memory. The memory stores processor-executable instructions, which, when executed by the processor, cause the processor to operate as follows. First, the processor receives the visual content. Then, the processor determines whether there is a reflective surface in the visual content. When the reflective surface exists in the visual content, the processor determines whether target information is displayed on the reflective surface. When the target information is displayed on the reflective surface, the processor provides prompt information to the user, the prompt information including: (i) an indication that the target information is displayed on the reflective surface; (ii) a request to delete the target information by using an editing operation. Afterwards, the processor receives user input from the user, the user input indicating whether to delete the target information by using the editing operation. The device configured in this way can promptly notify the user of the existence of the target information displayed on the reflective surface so that the user can decide to keep the target information or delete the target information from the visual content.

[0010] In an embodiment of the first aspect, the processor is further configured to obtain a preview version of the visual content without the target information by using the editing operation. Then, the processor is configured to provide the preview version of the visual content together with the prompt information to the user. By using this preview version of the visual content, the user can check the result of applying the editing operation to the visual content before making a final decision on the target information, which may be beneficial in certain usage scenarios.

[0011] In an embodiment of the first aspect, the memory further stores a target information type set and a risk level set, each risk level being associated with an information type in the target information type set. In this embodiment, when the target information is displayed on the reflective surface, the processor is further configured to perform the following operations. The processor searches the target information type set to which the target information belongs. Then, the processor searches the risk level set to which the target information type is associated. Next, the processor provides the target information type and the risk level together with the prompt information to the user. By knowing the type and risk level of the target information displayed on the reflective surface, the user can appropriately decide whether to delete the target information.

[0012] In an embodiment of the first aspect, the target information type set includes: personal data, user activity related information, and company related unique identification symbols. Therefore, the device according to the first aspect can analyze various information displayed on the reflective surface, which can make its use more flexible.

[0013] In an embodiment of the first aspect, the processor is further configured to: enlarge a segment of the visual content including the reflective surface; and provide the enlarged segment of the visual content together with the prompt information to the user. In this way, the device according to the first aspect can support the user to check the target information displayed on the reflective surface in minute details.

[0014] In an embodiment of the first aspect, the processor is used to determine whether the reflective surface exists in the visual content and whether the target information is displayed on the reflective surface by using a machine learning algorithm. By using a machine learning algorithm, the device according to the first aspect can operate more efficiently.

[0015] In an embodiment of the first aspect, the editing operation includes at least one of the following operations: an operation of coating the reflective surface in the visual content; an operation of blurring the reflective surface in the visual content; an operation of cutting out the segment including the reflective surface in the visual content; an operation of replacing the target information with public information on the reflective surface in the visual content. This can make the device according to the first aspect more flexible in use. For example, when the reflective surface displaying the target information in the image is located at the boundary of the image, the cutting operation can be used. When there are no strict requirements on the quality of the visual content (such as the quality of the image), the blurring and coloring operations can be used. When the quality of the visual content is very important, the target information (for example, a new company logo reflected in the mirror shown in the image) can be replaced with public information (for example, an old company logo that is already known to the public). Therefore, each of the editing operations listed above may be beneficial in a specific usage scenario.

[0016] According to a second aspect, a method for processing visual content is provided. The method starts with a step of receiving the visual content. Then, the method proceeds to a step of determining whether there is a reflective surface in the visual content. When the reflective surface exists in the visual content, the method proceeds to a step of determining whether the target information is displayed on the reflective surface. When the target information is displayed on the reflective surface, the method proceeds to a step of providing prompt information to the user, the prompt information comprising: (i) an indication that the target information is displayed on the reflective surface; (ii) a request to delete the target information by using an editing operation. Thereafter, the method proceeds to a step of receiving user input from the user, the user input indicating whether to delete the target information by using the editing operation. In this way, the user can be notified of the existence of the target information displayed on the reflective surface in a timely manner so that the user can decide whether to retain the target information or delete the target information from the visual content.

[0017] In an embodiment of the second aspect, the method further comprises the following steps: obtaining a preview version of the visual content without the target information by using the editing operation; and providing the preview version of the visual content together with the prompt information to the user. By using such a preview version of the visual content, the user can check the result of applying the editing operation to the visual content before making a final decision on the target information, which may be beneficial in certain usage scenarios.

[0018] In an embodiment of the second aspect, the method further comprises, after determining that the target information is displayed on the reflective surface, performing the following steps: searching a target information type to which the target information belongs in a pre-stored target information type set; searching a risk level associated with the target information type in a pre-stored risk level set; and providing the target information type and the risk level together with the prompt information to the user. By knowing the type and risk level of the target information displayed on the reflective surface, the user can appropriately decide whether to delete the target information.

[0019] In an embodiment of the second aspect, the target information type set includes: personal data, user activity related information, and company related unique identification symbols. Therefore, the method according to the second aspect can be used to analyze various information displayed on the reflective surface, which can make it more flexible in use.

[0020] In an embodiment of the second aspect, the method further comprises the following steps: amplifying a segment of the visual content including the reflective surface; and providing the amplified segment of the visual content together with the prompt information to the user. The amplification can support the user to examine the target information displayed on the reflective surface in minute details.

[0021] In an embodiment of the second aspect, the determining whether the reflective surface exists in the visual content and the determining whether the target information is displayed on the reflective surface are performed by using a machine learning algorithm. By using a machine learning algorithm, the method according to the second aspect can be performed more efficiently.

[0022] In an embodiment of the second aspect, the editing operation includes at least one of the following operations: an operation of coating the reflective surface in the visual content; an operation of blurring the reflective surface in the visual content; an operation of cutting out the segment including the reflective surface in the visual content; an operation of replacing the target information with the public information on the reflective surface in the visual content. This can make the method according to the second aspect more flexible in use. For example, when the reflective surface displaying the target information in the image is located at the boundary of the image, the cutting operation can be used. When there are no strict requirements on the quality of the visual content (such as the quality of the image), the blurring and coloring operations can be used. When the quality of the visual content is very important, the target information (for example, a new company logo reflected in the mirror shown in the image) can be replaced with public information (for example, an old company logo that is already known to the public). Therefore, each of the editing operations listed above may be beneficial in a specific usage scenario.

[0023] According to a third aspect, a computer program product is provided. The computer program product comprises a computer readable storage medium storing computer code. The computer code is executed by at least one processor, causing the at least one processor to perform the method according to the second aspect. By using such a computer program product, the implementation of the method according to the second aspect in any data processing device, such as the device according to the first aspect, can be simplified.

[0024] Other features and advantages of the present disclosure will be apparent by reading the following detailed description and reviewing the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present disclosure is explained below with reference to the accompanying drawings, in which:

[0026] Figure 1 A block diagram of an apparatus for processing visual content provided by an exemplary embodiment is shown;

[0027] Figure 2An exemplary embodiment provides a method for operating Figure 1 A flowchart of a method of an apparatus;

[0028] Figure 3 schematically illustrates visual content including reflections provided by an exemplary embodiment;

[0029] Figure 4 Schematically shows the Figure 2 Method of processing Figure 3 The prompt information received by the user during the visual content;

[0030] Figure 5 Schematically shows the Figure 2 The method is Figure 3 The processing result of the visual content;

[0031] Figure 6 A block diagram of a wireless communication system provided by an exemplary embodiment is shown. DETAILED DESCRIPTION

[0032] Various embodiments of the present disclosure are further described in detail in conjunction with the accompanying drawings. However, the present disclosure can be embodied in many other forms and should not be construed as being limited to any specific structure or function discussed in the following description. On the contrary, these embodiments are provided to make the description of the present disclosure detailed and complete.

[0033] Based on the detailed description, it is apparent to those skilled in the art that the scope of the present disclosure includes any embodiment of the present disclosure disclosed herein, whether the embodiment is implemented independently or in combination with any other embodiment of the present disclosure. For example, the apparatus and method disclosed herein can be implemented in practice by using any number of embodiments provided herein. In addition, it should be understood that any embodiment of the present disclosure can be implemented using one or more elements or steps set forth in the appended claims.

[0034] Any embodiment described herein as an "exemplary embodiment" is not necessarily to be construed as preferred or having advantages over other embodiments, unless otherwise stated.

[0035] According to exemplary embodiments disclosed herein, visual content may refer to pictures, photographs, images, image frames, schematic diagrams (e.g., flow charts, block diagrams, circuit diagrams), charts (e.g., bar charts, line charts, maps, etc.), infographics, videos (e.g., video clips, online videos), video frames, screenshots, memos, slide layouts, or any combination thereof. In the case of using video, the video may be decomposed into video frames, each of which may be processed separately according to aspects of the present disclosure, as will be discussed in detail below.

[0036] As used in the exemplary embodiments disclosed herein, a reflective surface may refer to a surface that is capable of reflecting light, thereby forming a reflection that may include visual information. Some examples of reflective surfaces may include, but are not limited to, water surfaces, eye surfaces, white paper surfaces, colored object surfaces, metal surfaces, mirror surfaces, glass surfaces, and the like. It should be noted that the present disclosure is not limited to a certain type of reflection formed by a reflective surface. In other words, all types of reflections (e.g., regular reflections and irregular reflections) that can be used to partially or fully restore visual information displayed on a reflective surface are within the scope of consideration. When the scene to be imaged / captured by a camera includes one or more reflective surfaces, the visual information displayed on the one or more reflective surfaces will be included in the obtained visual content (e.g., the obtained image).

[0037] According to exemplary embodiments disclosed herein, target information may refer to one or more specific types of visual information displayed on a reflective surface. The target information may be defined by a user and may include, but is not limited to, restricted information, secret information, and confidential information. Therefore, it is highly desirable to avoid inadvertent disclosure of target information in any visual content. However, to the best of the author's knowledge, in research and industrial products related to visual information sharing services, the problem of information leakage occurring through reflective / reflective surfaces displayed in visual content has not yet been addressed.

[0038] Therefore, the exemplary embodiments disclosed herein provide a technical solution that can mitigate or even eliminate the above-mentioned shortcomings specific to the prior art. Specifically, the technical solution involves providing prompt information to a user in response to detecting target information on one or more reflective surfaces present in the visual content. The prompt information indicates that the target information is displayed on the one or more reflective surfaces, and requests the user to confirm whether to delete the target information by using an editing operation. Therefore, the user can be promptly notified of the existence of the target information in the visual content, so that the user can decide whether to retain the target information or delete the target information from the visual content. In this way, the risk of visual information leaking through one or more reflective surfaces present in the visual content can be reduced.

[0039] Figure 1A block diagram of an apparatus 100 for processing visual content provided by an exemplary embodiment is shown. The apparatus 100 may be implemented as a single device or may be a part of a user equipment (UE). A UE may refer to an electronic computing device for performing wireless communication. UE can be implemented as a mobile station, a mobile terminal, a mobile subscriber unit, a mobile phone, a cellular phone, a smart phone, a cordless phone, a personal digital assistant (PDA), a wireless communication device, a desktop computer, a laptop computer, a tablet computer, a gaming device, a netbook, a smartbook, an ultrabook, a medical mobile device or apparatus, a biometric sensor, a wearable device (e.g., a smart watch, smart glasses, a smart wristband, etc.), an entertainment device (e.g., an audio player, a video player, etc.), a vehicle component or sensor (e.g., a driver assistance system), a smart meter / sensor, an unmanned vehicle (e.g., an industrial robot, a quadcopter, etc.) and its components (e.g., an autonomous vehicle computer), industrial manufacturing equipment, a global positioning system (GPS) device, an Internet of Things (IoT) device, an industrial IoT (IIoT) device, a machine-type communication (MTC) device, a set of massive IoT (MIoT) or massive MTC (mMTC) devices / sensors, or any other suitable device for supporting wireless communication. In some embodiments, a UE may refer to at least two collocated and interconnected UEs as defined above.

[0040] like Figure 1 As shown, the apparatus 100 includes a processor 102 and a memory 104. The memory 104 stores processor executable instructions 106, which, when executed by the processor 102, cause the processor 102 to perform various aspects of the present disclosure, as will be described in more detail below. It should be noted that Figure 1 The number, arrangement, and interconnection of the structural elements constituting the apparatus 100 shown are not intended to limit the present disclosure, but are merely intended to provide a general idea of ​​how the structural elements may be implemented within the apparatus 100. For example, the processor 102 may be replaced with multiple processors, and the memory 104 may be replaced with multiple removable and / or fixed storage devices, depending on the specific application. In addition, when the apparatus 100 is implemented as a single device, it may also include a transceiver ( Figure 1The transceiver is not shown in the figure, and is used to perform, for example, wireless communication with the UE (for example, to send the result of processing the visual content to the UE). The transceiver itself can be implemented as two separate devices, one for receiving operations and the other for sending operations. Regardless of the implementation, the transceiver is intended to be able to perform different operations required to perform data reception and transmission, such as signal modulation / demodulation, encoding / decoding, etc.

[0041] The processor 102 may be implemented as a CPU, a general-purpose processor, a single-purpose processor, a microcontroller, a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a complex programmable logic device, etc. It should also be noted that the processor 102 may be implemented as any combination of one or more of the above. For example, the processor 102 may be a combination of two or more microprocessors.

[0042] The memory 104 may be implemented as a classic non-volatile memory or a volatile memory used in modern electronic computers. For example, the non-volatile memory may include a read-only memory (ROM), a ferroelectric random access memory (RAM), a programmable ROM (PROM), an electrically erasable PROM (EEPROM), a solid state drive (SSD), a flash memory, a magnetic disk storage (such as a hard disk drive and a magnetic tape), an optical disk storage (such as a CD, a DVD, and a Blu-ray disc), etc. As for the volatile memory, examples thereof include dynamic RAM, synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), static RAM, etc.

[0043] The processor executable instructions 106 stored in the memory 104 may be configured as computer executable program code that causes the processor 102 to perform various aspects of the present disclosure. The computer executable program code for performing the operations or steps of various aspects of the present disclosure may be written in any combination of one or more programming languages ​​such as Java, C++, etc. In some examples, the computer executable program code may be in the form of a high-level language or in a precompiled form and dynamically generated by an interpreter (also pre-stored in the memory 104).

[0044] Figure 2 A flowchart of a method 200 for operating the apparatus 100 provided by an exemplary embodiment is shown. If the apparatus 100 is part of a UE, the method 200 is performed locally in the UE. If the apparatus 100 is implemented as a separate device (e.g., another UE or a remote server) for performing wired or wireless communication with the UE, the method 200 is performed remotely relative to the UE.

[0045] The method 200 starts from step S202, in which the processor 102 receives visual content. For example, the visual content may be an image just captured by a camera of a UE (e.g., a smartphone), or an image downloaded from an image database stored locally in the UE or remotely in another UE or a server. As mentioned above, other types of visual content may also exist, such as video clips, schematic diagrams, etc.

[0046] Then, the method 200 proceeds to step S204, in which the processor 102 determines whether there is a reflective surface in the visual content. The processor 102 can perform step S204 by using a machine learning algorithm, such as a neural network (e.g., a region-based convolutional neural network (R-CNN), Fast R-CNN, Faster R-CNN, You Only Look Once (YOLO) framework, etc.). Standard computer vision pipelines for object detection, such as instance-level object segmentation, which can be based on Mask R-CNN, U-Net framework, can also be used. Alternatively or additionally, the processor 102 can perform step S204 by using hand-crafted features and rules from human experts. In addition, contextual information can be expressed as auxiliary tasks to improve the determination in step S204 (e.g., face detection helps glasses detection, thereby helping to detect reflections provided by glasses).

[0047] When a reflective surface is present in the visual content, the method 200 proceeds to step S206, in which the processor 102 determines whether target information is displayed on the reflective surface. As previously described, the target information may be restricted information, confidential information, or any other information that should not be disclosed (at least temporarily). The target information may be user-defined. Thus, the user may decide which object or concept displayed on the reflective surface to delete. For example, if the user does not want others to know that he / she has a cat, he / she may indicate the cat's reflection as the target information. Step 206 may also be performed by the processor 102 with the aid of a suitable machine learning algorithm, such as a suitably trained neural network (e.g., CNN).

[0048] When the target information is displayed on the reflective surface, the method 200 proceeds to step S208, in which the processor 102 provides (e.g., outputs, displays, or transmits) prompt information to the user. For example, the prompt information can be output to the user as a notification or message via a dedicated mobile application installed on a smart phone (the mobile application can also be used by the user to select the visual content to be processed and define the target information). The prompt information includes: (i) an indication that the target information is displayed on the reflective surface; (ii) a request to delete the target information by using an editing operation. The editing operation can be: an operation of painting on the reflective surface in the visual content; an operation of blurring the reflective surface in the visual content; an operation of cutting out a segment of the visual content that includes the reflective surface; an operation of replacing the target information with public information on the reflective surface in the visual content, or any combination of these operations. These editing operations can be used according to the requirements for the naturalness, or in other words, the quality, of the visual content to be processed. Therefore, if necessary, the prompt information can include a list of the editing operations indicated above. In addition, different visual generation algorithms can be used to use visual content with one or more reflections as input data to obtain visual content without one or more reflections (i.e., one or more reflections are removed / minimized) as output data. These algorithms can also be trained taking into account the naturalness of the visual content (e.g., through adversarial loss) so that the processed visual content is perceptually realistic.

[0049] The method 200 ends with step S210, in which the processor 102 receives a user input from the user. The user input indicates whether the target information is to be deleted using an editing operation. If the prompt information also indicates some or all of the above-mentioned editing operations, the user input may also indicate which of these editing operations is to be used (of course, if the user has decided to delete the target information from the visual content). After the visual content is processed by using the editing operation, it may be ready for continued use or future use (for example, the visual content may be stored locally in the UE's photo album, or may be uploaded to a remote server or shared with other users via a wireless network).

[0050] In one embodiment, the method 200 may include an additional step (before step S208), in which the processor 102 obtains a preview version of the visual content without the target information by using the editing operation. In this regard, the processor 102 may be configured to select one of the above editing operations based on a predefined (e.g., by a user through a dedicated mobile application) requirement for the quality of the visual content (e.g., in the form of "high quality", "medium quality" or "low quality"). When such a preview version is obtained, it may be provided to the user together with the prompt information in step S208.

[0051] In an alternative or additional embodiment, method 200 may include an additional step (before step S208) in which processor 102 magnifies the segment of the visual content that includes the reflective surface. Processor 102 may then provide the magnified segment of the visual content to the user along with the prompt information in step S208. When small objects are displayed on the reflective surface, such as jewelry, small font text, etc., magnification may be required.

[0052] In one embodiment, the method 200 may include an additional step (before step S208), in which the processor 102 determines the type and risk level of the target information displayed on the reflective surface. In this regard, the memory 104 may also store a set of target information types and a set of risk levels, each risk level being associated with an information type in the set of target information types. Both sets may be defined by the user. Some examples of target information types include, but are not limited to, personal data, user activity related information, and company-related unique identification symbols. Personal data, also referred to as personally identifiable information, may refer to any information associated with an identifiable person, such as an identification number (e.g., social security number, bank card number, etc.), an identification card (e.g., bank card, access card, access card, driver's license, etc.), location data (e.g., home address, current location data, etc.), name, online identifier (e.g., IP address, nickname, etc.), health information, an individual's economic, cultural or social identity, or any combination thereof. User activity related information may refer to any scene and activity involving one or more users of interest (e.g., movie scenes, closed training of athletes, etc.). Company-related unique identification symbols may include company logos, brands, corporate identification codes, etc. For risk levels, they can be represented by using a specific color coding pattern (for example, the colors "red", "orange" and "green" can correspond to high risk level, medium risk level and low risk level, respectively) and / or character coding (for example, the letters "A", "B" and "C" can correspond to high risk level, medium risk level and low risk level, respectively). First, the processor 102 searches the target information type set for the target information type. Then, the processor 102 searches the risk level associated with the target information type in the risk level set. Subsequently, the processor 102 can provide the target information type and the risk level together with the prompt information to the user in step S208. It should be noted that if two or more target information types are displayed on the reflective surface, the processor 102 can sum their corresponding risk levels to obtain a combined risk level, which can then be provided to the user (for example, by using a specific pattern of color coding or character coding). If the combined risk level is higher than a threshold (for example, risk level "A"), the user can immediately decide to delete the target information.

[0053] Figure 3Schematically illustrates visual content including reflections provided by an exemplary embodiment. More specifically, the visual content is displayed on a touch screen of a smartphone 300 as an image 302, wherein a user 304 is shown as viewing himself in a mirror 306. The image 302 is captured (e.g., by another user) so that the mirror 306 reflects not only the user 304, but also his / her cat. Assume that the user 304 wants to share the image 302 with his / her friends, but does not want them to know his / her cat. Therefore, the reflection of the cat can be regarded as target information. It is also assumed that the image 302 is opened in a dedicated mobile application, and the user 304 uses the dedicated mobile application to select the image to be processed and (at least) indicate the target information to be detected on the image. Furthermore, the mobile application is used to initiate the execution of the method 200 by requesting the user 304 to confirm the processing of the image 302. If the user 304 presses "yes", the mobile application will cause the device 100 to start the method 200. The apparatus 100 may be integrated in the smartphone 300 (i.e., the image 302 may be processed locally in the smartphone 300), or may be a remote server accessed by the smartphone 300 via a wireless network (i.e., the image 302 may be sent to the remote server to be processed according to the method 200). Those skilled in the art will recognize that the software components (e.g., buttons “yes” and “no”) and their arrangement on the touch screen of the smartphone 300 are for illustrative purposes only and should not be construed as any limitation of the present disclosure.

[0054] Figure 4 The prompt information received by the user 304 during processing of the visual content (i.e., the image 302) according to the method 200 is schematically shown. More specifically, it is assumed that the processor 102 of the device 100 has executed steps S202 to S206 of the method 200 and found the reflection of the cat in the mirror 306. Subsequently, in step S208 of the method 200, the processor 102 provides the prompt information to the user 304 (e.g., sends it to the smartphone 300), so that the prompt information is displayed on the touch screen of the smartphone 300. Figure 4 As shown, the prompt information includes a text indication "target information ('cat') detected!" and a text request "Do you want to delete the target information?" It should be noted that, instead of the text indication, or in addition to the text indication, the prompt information may also include a graphic element overlaid on the image 302 to highlight the target information (i.e., the cat's reflection). Such a graphic element may be composed of different straight lines or curves (e.g., solid lines, dashed lines, etc.) surrounding the target information (see Figure 4), different arrow pointers, etc. If the user 304 decides to delete the reflection of the cat in the mirror 306, he / she should press the button "Yes" on the touch screen of the smartphone 300; otherwise, he / she should press the button "No". The processor 102 of the device 100 will receive the result of pressing the button "Yes" or "No" as the user input in step S210 of the method 200. It should also be noted that the prompt information can also include a list of editing operations available for the image 302; the list can be formed based on the above-mentioned editing operations and the image quality requirements that can be predefined by the user 304 in the mobile application.

[0055] Figure 5 The result of processing visual content (i.e., image 302) according to method 200 is schematically shown. Assume that user 304 responds to Figure 4 In response to the prompt information shown, the button "Yes" is pressed, and the operation of replacing the target information with the public information on the reflective surface is selected to make the image 302 more realistic. In response to this user input, the processor 102 of the device 100 will replace the cat's reflection with the reflection of any other public object. Such a collection of objects can be pre-stored in the memory 104 of the device 100. For example, in Figure 5 In the example, the cat's reflection is replaced by the reflection of the potted flower. It should be noted that in step S208 of method 200, the result of processing image 302 may have been reported to the user as a preview version of image 302, so that the user can evaluate the efficiency of the editing operation.

[0056] Figure 6A block diagram of a wireless communication system 600 provided by an exemplary embodiment is shown. System 600 includes a smartphone 602, which is assumed to be implemented in the same or similar manner as smartphone 300. In other words, a similar dedicated mobile application is installed on smartphone 602. System 600 also includes a cloud server 604, which is assumed to be implemented as device 100. When it is necessary to process visual content according to method 200, smartphone 602 accesses cloud server 604 via wireless network 606 (e.g., the Internet). Imagine that smartphone 602 has an image similar to image 302, which is to be examined to determine the presence of target information. Then, smartphone 602 uploads the image to cloud server 604, where method 200 is initiated for the image. If cloud server 604 detects that the image includes one or more reflective surfaces that display target information, cloud server 604 will report this to smartphone 602 via wireless network 606, so that the user of smartphone 602 can decide whether to delete the target information from the image. If the user confirms that the target information is to be deleted, cloud server 604 will apply appropriate (based on the requirements for image quality, if any) editing operations to the image. Afterwards, the cloud server 604 provides the processed or edited image back to the smartphone 602 so that the user can transfer it to different devices of his / her friends, such as a desktop computer 608, a smart watch 610, and another smartphone 612. If necessary, the smartphone 602 can communicate with the cloud server 604 through a wireless network 606, and communicate with each of the desktop computer 608, the smart watch 610, and the smartphone 612 through different wireless or wired networks (e.g., a Wi-Fi network, a local area network (LAN), etc.).

[0057] It should also be noted that if a user uses smartphone 602 to share a video (e.g., a live stream) with each of desktop computer 608, smart watch 610, and smartphone 612 via wireless network 606, a mobile application installed on smartphone 602 can be used to decompose the video into a set of video frames (in real time) and direct each video frame in the set of video frames to cloud server 604 for processing according to method 200.

[0058] In another embodiment, assume that a teacher uses a desktop computer 608 to transmit a video lecture to students who are users of, for example, smartphones 602 and 612. In this case, a dedicated mobile application similar to that used in smartphone 300 is installed on the desktop computer 608. Assuming that the video lecture is transmitted via a communication platform for simultaneously displaying all participants (e.g., Microsoft Teams, Skype, Zoom, etc.), the mobile application can (continuously or periodically) obtain screenshots and direct the screenshots to the cloud server 604 for processing according to method 200. For example, if one of the users of smartphones 602 and 612 wears glasses, there may be reflections on the surface of their glasses. If the cloud server 604 determines that the reflections present in one or more screenshots are not related to the information of the video lecture (i.e., in this case, the target information is any information different from the information transmitted by the teacher), a prompt information can be provided to the teacher, indicating that the user of, for example, smartphone 602 is not paying attention to the video lecture. It should be noted that in this case, the cloud server 604 cannot inform the teacher of the exact content of one or more reflections to ensure privacy.

[0059] Those skilled in the art will recognize that each step or operation of method 200, or any combination of steps or operations, can be implemented by various means such as hardware, firmware and / or software. For example, one or more of the above steps or operations can be implemented by processor executable instructions, data structures, program modules and other suitable data representations. In addition, the processor executable instructions that implement the above steps or operations can be stored on a corresponding data carrier and executed by processor 102. The data carrier can be implemented as any computer-readable storage medium configured to be read by the at least one processor to execute processor executable instructions. Such computer-readable storage media may include volatile and non-volatile media, removable and non-removable media. As an example, rather than a limitation, computer-readable media include media implemented in any method or technology suitable for storing information. In more detail, actual examples of computer-readable media include, but are not limited to, information transfer media, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (digital versatile disc, DVD), holographic media or other optical disk storage, magnetic tape, cassette, magnetic disk storage and other magnetic storage devices.

[0060] Although exemplary embodiments of the present disclosure are described herein, it should be noted that any various changes and modifications may be made in the embodiments of the present disclosure without departing from the scope of legal protection defined by the appended claims. In the appended claims, the word "comprising" does not exclude other element operations, and the term "a" or "an" does not exclude a plurality. The listing of certain measures in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A device for processing visual content, characterized in that include: A memory storing processor executable instructions; A processor, coupled to the memory, configured to: receiving the visual content; determining whether a reflective surface exists in the visual content; When the reflective surface exists in the visual content, determining whether target information is displayed on the reflective surface; When the target information is displayed on the reflective surface, prompt information is provided to the user, the prompt information including: (i) an indication that the target information is displayed on the reflective surface; (ii) a request to delete the target information by using an editing operation; A user input is received from the user, the user input indicating whether the target information is to be deleted by using the editing operation.

2. The device according to claim 1, characterized in that The processor is further configured to: Before providing the prompt information to the user, obtaining a preview version of the visual content without the target information by using the editing operation; The preview version of the visual content is provided to the user along with the prompt information.

3. The device according to claim 1 or 2, characterized in that: The memory further stores a set of target information types and a set of risk levels, each risk level being associated with an information type in the set of target information types, and wherein the processor is further configured to, when the target information is displayed on the reflective surface: Searching the target information type to which the target information belongs in the target information type set; searching the risk level set for the risk level associated with the target information type; The target information type and the risk level are provided to the user together with the prompt information.

4. The device according to claim 3, characterized in that The target information type set includes: Personal data; Information about user activities; A unique identifier associated with a company.

5. The device according to any one of claims 1 to 4, characterized in that The processor is further configured to: enlarging a segment of the visual content that includes the reflective surface; The enlarged segment of the visual content is provided to the user together with the prompt information.

6. The device according to any one of claims 1 to 5, characterized in that The processor is used to determine whether the reflective surface exists in the visual content and whether the target information is displayed on the reflective surface by using a machine learning algorithm.

7. The device according to any one of claims 1 to 6, characterized in that The editing operation includes at least one of the following operations: Performing a coating operation on the reflective surface in the visual content; an operation of blurring the reflective surface in the visual content; An operation of cutting out the segment including the reflective surface in the visual content; An operation of replacing the target information with the common information on the reflective surface in the visual content.

8. A method for processing visual content, characterized in that include: receiving the visual content; determining whether a reflective surface exists in the visual content; When the reflective surface exists in the visual content, determining whether target information is displayed on the reflective surface; When the target information is displayed on the reflective surface, prompt information is provided to the user, the prompt information including: (i) an indication that the target information is displayed on the reflective surface; (ii) a request to delete the target information by using an editing operation; A user input is received from the user, the user input indicating whether the target information is to be deleted by using the editing operation.

9. The method according to claim 8, characterized in that Also includes: Before providing the prompt information to the user, obtaining a preview version of the visual content without the target information by using the editing operation; The preview version of the visual content is provided to the user along with the prompt information.

10. The method according to claim 8 or 9, characterized in that: The method further comprises, after determining that the target information is displayed on the reflective surface: Searching for the target information type to which the target information belongs in a pre-stored target information type set; Searching for a risk level associated with the target information type in a pre-stored risk level set; The target information type and the risk level are provided to the user together with the prompt information.

11. The method according to claim 10, characterized in that The target information type set includes: Personal data; Information about user activities; A unique identifier associated with a company.

12. The method according to any one of claims 8 to 11, characterized in that Also includes: enlarging a segment of the visual content that includes the reflective surface; The enlarged segment of the visual content is provided to the user together with the prompt information.

13. The method according to any one of claims 8 to 12, characterized in that The determining whether the reflective surface exists in the visual content and the determining whether the target information is displayed on the reflective surface are performed by using a machine learning algorithm.

14. The method according to any one of claims 8 to 13, characterized in that The editing operation includes at least one of the following operations: Performing a coating operation on the reflective surface in the visual content; an operation of blurring the reflective surface in the visual content; An operation of cutting out the segment including the reflective surface in the visual content; An operation of replacing the target information with the common information on the reflective surface in the visual content.

15. A computer program product, characterized in that The method comprises a computer-readable storage medium, wherein the computer-readable storage medium stores computer code, which, when executed by at least one processor, causes the at least one processor to perform the method according to any one of claims 8 to 14.

Images