Method and system for privacy violation detection of stored images

Abstract

Methods and systems for detecting privacy violations in image files. A policy to be used by a host image application is obtained, and a file system is monitored for digital images modified by a monitored image application. Next, it is determined that a digital image file includes at least some content that violates a defined setting for the host image application, and an action is taken based on the determination that the digital image includes at least some content that violates the defined setting for the host image application.

Description

Technical Field

[0001] This application generally relates to sensitive information in digital images, and more specifically, to identifying sensitive information in digital images and cleaning digital images. Background Technology

[0002] Digital images may contain sensitive information. In some cases, the sensitive information displayed in digital images is confidential company information.

[0003] In other cases, sensitive information is hidden from the user. Electronic devices (such as smartphones) are often equipped with camera apps that add metadata to every photo taken. For example, a GPS-enabled camera app might include the precise location coordinates and time of capture in a digital image. The user may not be aware that the digital image contains such sensitive information.

[0004] Users may be sharing sensitive information without realizing it when sharing digital images with parties outside of an organization. Ensuring a certain level of privacy for digital images would be beneficial. Attached Figure Description

[0005] As an example, reference will be made herein to the accompanying drawings illustrating exemplary embodiments of this application, wherein:

[0006] Figure 1 This is a schematic diagram of the operating environment of the illustrated example embodiment.

[0007] Figure 2 It is a diagram. Figure 1 A block diagram of components in an example embodiment of the China Mobile computing system.

[0008] Figure 3 A flowchart illustrating an example method for removing sensitive information from a digital image is shown.

[0009] Figure 4 A flowchart illustrating an example method for detecting privacy violations in digital image files is shown.

[0010] Figure 5 This is an illustration of the front view of an example electronic device when it displays the option to share digital images.

[0011] Figure 6 This is an illustration of the front view of an example electronic device when displaying a list of applications, and

[0012] Figure 7 These are illustrations of example digital images containing sensitive information, including depictions of a stack of papers and a smartphone.

[0013] Figure 8 yes Figure 7The image is a diagram of the cleaned digital image.

[0014] Figure 9 These are illustrations of example digital images containing sensitive information, including depictions of computer monitors displaying text.

[0015] Figure 10 yes Figure 9 The image is a diagram of the cleaned digital image.

[0016] Similar reference numerals can be used to represent similar components in different figures. Detailed Implementation

[0017] In a first aspect, this application describes a computer-implemented method for detecting privacy violations in image files. The method includes: obtaining a policy to be used by a main image application; monitoring a file system for digital image files modified by the monitored image application; determining that the digital image file contains at least some content that violates defined settings for the main image application; and taking action in response to determining that the digital image file contains at least some content that violates the defined settings for the main image application.

[0018] In some embodiments, monitoring the file system for digital image files modified by the monitored image application includes: continuously monitoring the file system and automatically detecting, in real time, the modifications made to the digital image files by the monitored image application.

[0019] In some embodiments, monitoring the file system for digital image files modified by the monitored image application includes: automatically and periodically scanning the file system for digital image files modified by the monitored image application.

[0020] In some embodiments, monitoring the file system for digital image files modified by the monitored image application includes: scanning the file system for digital image files modified by the monitored image application in response to input received at an input interface.

[0021] In some embodiments, taking the action includes processing the digital image file to modify at least some of its contents.

[0022] In some embodiments, taking the action includes generating a notification based on the violation.

[0023] In some embodiments, the notification identifies the monitored image application.

[0024] In some embodiments, the notification prompts users to adjust settings associated with the monitored image application.

[0025] In some embodiments, the notification provides an option to modify the digital image file to comply with the policy.

[0026] In some embodiments, the method may further include: receiving at an input interface an input of selecting an option for modifying the digital image file to comply with the policy; and modifying the digital image file in response to receiving the input of selecting the option for modifying the digital image file from the input interface.

[0027] On the other hand, this application describes a computing device configured to implement these methods.

[0028] In another aspect, this application describes a computing device. The computing device includes a memory and a processor coupled to the memory. The processor is configured to: acquire a policy to be used by a main image application; monitor a file system for digital image files modified by a monitored image application; determine that the digital image files contain at least some content that violates defined settings for the main image application; and take action in response to determining that the digital image files contain at least some content that violates the defined settings for the main image application.

[0029] In some embodiments, the processor is configured to monitor the file system for digital image files modified by a monitored image application, including: the processor is configured to continuously monitor the file system and automatically detect, in real time, the modifications made to the digital image files by the monitored image application.

[0030] In some embodiments, the processor is configured to monitor the file system for digital image files modified by the monitored image application, including: the processor is configured to automatically and periodically scan the file system for digital image files modified by the monitored image application.

[0031] In some embodiments, the processor is further configured to perform the action, including: the processor is configured to process the digital image file to modify at least some content.

[0032] In some embodiments, the processor is further configured to take the action, including: the processor is configured to generate a notification based on the violation.

[0033] In some embodiments, the notification identifies the monitored image application.

[0034] In some embodiments, the notification prompts users to adjust settings associated with the monitored image application.

[0035] In some embodiments, the notification provides an option to modify the digital image file to comply with the policy.

[0036] In some embodiments, the processor is further configured to: receive at an input interface an input selecting an option for modifying the digital image file to comply with the policy; and modify the digital image file in response to receiving the input selecting the option for modifying the digital image file from the input interface.

[0037] In some embodiments, the processor is configured to monitor a file system for digital image files modified by a monitored image application, including: the processor is configured to scan the file system for digital images modified by the monitored image application in response to input received at an input interface.

[0038] On the other hand, this application describes a non-transitory computer-readable storage medium storing processor-executable instructions for detecting privacy violations in image files. When executed by a processor, the processor: acquires a policy to be used by a main image application; monitors a file system for digital image files modified by the monitored image application; determines that the digital image file contains at least some content that violates defined settings for the main image application; and takes action in response to determining that the digital image file contains at least some content that violates the defined settings for the main image application.

[0039] Other aspects and features of this application will be understood by those skilled in the art upon review of the following description in conjunction with the accompanying drawings.

[0040] In this application, the terms “about,” “approximately,” and “roughly” are intended to cover the amount of variation that can occur within the upper and lower limits of a range of values, such as the amount of variation in attributes, parameters, and dimensions. In a non-limiting example, the terms “about,” “approximately,” and “roughly” can mean plus or minus ten percent or less.

[0041] In this application, the term "and / or" is intended to cover all combinations and subcombinations of the listed elements, including any single element, any subcombination, or all of the listed elements, and does not necessarily exclude additional elements.

[0042] In this application, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any single element, any sub-combination, or all of the listed elements, and does not necessarily exclude additional elements, nor is it necessary to include all of the listed elements.

[0043] First refer to Figure 1This is a schematic diagram of the operating environment of an illustrated example embodiment. Network 120 is a computer network. Network 120 allows communicating computer systems to communicate with each other. For example, as shown, network 120 allows mobile computer system 100 to communicate with target computer system 110. Mobile computer system 110 can use network 120 to share images with target computer system 110. Target computer system 110 can be adapted to display digital images on a display interface.

[0044] Each of the mobile computer system 100 and the target computer system 110 can be geographically separated. In other words, the mobile computer system 100 can be located far from the target computer system 110.

[0045] Each of the mobile computer system 110 and the target computer system is or includes one or more computing devices. As shown, the mobile computer system 110 may be a smartphone. As shown, the target computer system 110 may be a laptop computer. Alternatively, the mobile computer system 100 and the target computer system 110 may be or include another type of device, such as a personal computer, laptop computer, tablet computer, notebook computer, handheld computer, personal digital assistant, wearable computing device (such as a smartwatch, wearable activity monitor, wearable smart jewelry, and glasses and other optical devices including head-mounted displays), or any other type of computing device that can be configured to store data and software instructions and execute software instructions to perform operations consistent with the disclosed embodiments.

[0046] In some embodiments, each of the mobile computer system 100 and the target computer system 110 may include multiple computing devices, such as email servers, web servers, database servers, social networking servers, file transfer protocol (FTP) servers, etc. The multiple computing devices may communicate using a computer network. For example, in some embodiments, the mobile computer system 100 is a laptop computer, and the target computer system 110 is a photo and video sharing social networking service. In some embodiments, the mobile computer system 110 is a photo and video sharing social networking service or a component of that service, and the target computer system 110 is a desktop computer acting as a client of that social networking service.

[0047] refer to Figure 2 Its illustration shows Figure 1A block diagram of an example embodiment of the mobile computing system 100. In one example embodiment, the computing device 200 may be a mobile communication device. This mobile communication device may be configured for bidirectional communication, having data and optional voice communication capabilities, as well as the ability to communicate with other computer systems (e.g., via the Internet). In some embodiments, the computing device 200 may take other forms, such as a smartwatch, computer, tablet, laptop, or any other electronic device configured for connection via a wireless network.

[0048] Figure 2 The computing device 200 may include a housing (not shown) that houses the components of the computing device 200. The internal components of the computing device 200 may be built on a printed circuit board (PCB). The computing device includes a controller that includes at least one processor 240 (such as a microprocessor) for controlling the overall operation of the computing device 200. The processor 240 interacts with device subsystems (such as a wireless communication subsystem 211) to exchange radio frequency signals with a wireless network to perform communication functions. The processor 240 interacts with additional device subsystems, including one or more input interfaces (which may include, but are not limited to, any of the following: one or more cameras 280, a keyboard, one or more control buttons, one or more microphones 258, a gesture sensor, and / or a touch-sensitive overlay associated with a touchscreen display), flash memory 224, random access memory (RAM) 246, read-only memory (ROM) 248, auxiliary input / output (I / O) subsystem 250, data port 252 (which may be a serial data port, such as a Universal Serial Bus (USB) data port), one or more output interfaces (such as a display 204), one or more speakers 256 or other output interfaces), short-range communication subsystem 262, and other device subsystems generally designated 264.

[0049] In some example embodiments, the auxiliary input / output (I / O) subsystem 250 may include an external communication link or interface, such as an Ethernet connection. The communication subsystem 211 may include additional wireless communication interfaces for communicating with other types of wireless networks, such as Wi-Fi networks.

[0050] In some example embodiments, computing device 200 also includes a removable storage module 230 (generally including flash memory) and a storage module interface 232. Network access can be associated with a subscriber or user of computing device 200 through storage module 230, which may be a subscriber identity module (SIM) card used in a GSM network or another type of storage module used in the relevant wireless network type. Storage module 230 can be inserted into or connected to storage module interface 232 of computing device 200.

[0051] The computing device 200 may store data 227 in an erasable persistent memory, which in one example embodiment is flash memory 244. In some example embodiments, data 227 may include service data containing information necessary for the computing device 200 to establish and maintain communication with the wireless network. Data 227 may also include user application data, such as messages (e.g., emails, text messages, multimedia messages, etc.), address book and contact information, camera data, calendar and schedule information, notepad documents, image files, and other user information typically stored on the computing device 200 by the user of the computing device 200, as well as other data.

[0052] Data 227 stored in persistent storage (such as flash memory 244) of computing device 200 can be at least partially organized into several databases or data warehouses, each containing data items of the same data type or associated with the same application. For example, image files, email messages, contact records, and task items can be stored in individual databases within the storage of computing device 200.

[0053] The short-range communication subsystem 262 provides communication between the computing device 200 and different systems or devices, which are not necessarily similar devices. For example, the short-range communication subsystem 262 may include an infrared device and associated circuitry and components, a wireless bus protocol-compliant communication device (such as a Bluetooth communication module) for communicating with systems and devices with similar functionality, and / or a near-field communication (NFC) interface.

[0054] The computing device 200 includes one or more cameras 280. The cameras 280 are configured to generate camera data, such as images in the form of still photographs and / or video data. This camera data can be captured in the form of electronic signals generated by an image sensor associated with the camera 280. More specifically, the image sensor is configured to generate electronic signals based on received light. The image sensor converts optical images into electronic signals, which can be output from the image sensor via one or more electrical connectors associated with it. These electronic signals represent electronic image data, which may be referred to as camera data.

[0055] A set of applications controlling basic device operation, including data and, possibly, voice communication applications, can be installed on computing device 200 during or after manufacturing. Additional applications and / or updates to the operating system 222 or software applications 224 can also be loaded onto computing device 200 via wireless network, auxiliary I / O subsystem 250, data port 252, short-range communication subsystem 262, or other suitable device subsystem 264. Downloaded programs or code modules can be permanently installed, for example, written to program memory (such as flash memory 224), or written to and executed from RAM 246 so that they can be executed by processor 240 at runtime.

[0056] Processor 240 operates under the control of a stored program and executes software modules 220 stored in memory (such as persistent memory, e.g., in flash memory 224). Figure 2 As shown, software module 220 may include operating system software 222 and one or more applications 224 (or modules). Specific examples of applications that may be stored on computing device 200 include: file-sharing applications, and media applications for capturing and / or editing one or more forms of digital media (including images, videos, and / or sound). Specific examples of file-sharing applications include: email communication applications, and other types of communication applications for instant messaging (IM), short message service (SMS), and social networking or communication applications. Media applications may include image applications. Specific examples of image applications include: image editors, digital photography applications for editing digital photos, and camera applications 290 for using camera 280 to capture and edit photos.

[0057] Operating system software 222 can provide a file system for storing, modifying, and accessing files stored in persistent memory (such as flash memory 244) of computing device 200. This file system can be accessible to other programs running on processor 240 through a programmable interface provided by operating system software 222.

[0058] refer to Figure 3 It illustrates an example method for removing sensitive information from digital images in the form of a flowchart. Method 300 is performed by a computer system (e.g., Figure 2 The method is implemented in a computing device 200. The computer system has a processor coupled to memory. The processor is configured to implement method 300. More specifically, the processor is configured to receive an instruction to share a digital image, and in response to receiving the instruction to share the digital image, determine that the digital image contains a depiction of a company display medium classified as sensitive based on a policy; and in response to determining that the digital image contains a depiction of a company display medium classified as sensitive based on the policy, process the digital image to modify the depiction; and share the digital image.

[0059] Method 300 begins with operation 302. In operation 302, an instruction to share a digital image is received. The instruction may correspond to or be based on input received at the input interface. For example, the instruction may correspond to or be generated by a user clicking or tapping... Figure 5 The share button 508 displayed on the electronic device 500 is triggered. Figure 5 This is a front view illustration of an example electronic device 500. Electronic device 500 can be an implementation of... Figure 2 The computing device 200 is a smartphone. The display interface 502 shows a graphical user interface for a photo gallery application. The gallery includes three digital images 506, where the first image is shown as selected via a checkbox user interface element 504. The user is presented with the option to share the selected photo via a share button 508. The share button 508 can be tapped or clicked to indicate the selection of the image sharing option.

[0060] In some embodiments, the instructions for sharing a digital image include a specification of the image that should be shared. In one example, the digital image is in JPEG file format, but other image formats may be used.

[0061] In response to the call to the share button 508, the display interface 502 can be as follows: Figure 6 As shown, a list of applications that can be invoked is presented. Email icon 606 and FTP icon 608 can each correspond to an email application and an FTP application, respectively, which can be used to share digital images. In some cases, operations 304 and 306 in method 300 occur before the application list is displayed to the user. Cleanup icon 604 can correspond to a cleanup application that provides options and features related to cleaning up images before sharing them with others. The instruction to share a digital image can correspond to or be triggered by the user clicking or tapping cleanup icon 604. In some embodiments, in response to an invocation of share button 508, the cleanup application is automatically launched without first displaying the application icon list.

[0062] Following operation 302, method 300 includes in operation 304: in response to receiving an instruction to share a digital image, determining that the digital image contains a depiction of a corporate display medium classified as sensitive based on a policy. The digital image may include depictions of one or more corporate display media. In some embodiments, corporate display media are corporate media configured to implement functions that visually present corporate information. For example, company display media may include: documents, specific boards (such as whiteboards, bulletin boards, corkboards, and blackboards), paper and stationery (such as one or more sheets of paper, a stack of multiple sheets of paper, loose-leaf charts, lined or grid paper, grid paper, sticky notes, notebooks, memo pads, sketchbooks, and memo pads), specific storage items (such as folders, labels, file tags, binders, binder labels, storage boxes, and storage box labels), desktop items and accessories (such as paper boxes, mail / outboxes for paperwork), mailing items (such as envelopes and address labels), wall or desk calendars, name tags, identification badges or security badges, or specific electronic equipment or devices (such as computer monitors, projector screens, laptops, tablets, Internet Protocol (IP) phones, and... Figure 7 (Smartphones in China).

[0063] Company display media may include display surfaces configured to implement the function of visually presenting company information. Examples of display surfaces include computer monitor screens, smartphone screens, and Internet Protocol (IP) phone screens. In some cases, display surfaces may provide additional functionality. In one example, the display surface includes a touchscreen input interface configured to receive input via touch. In another example, the display surface includes the writing surface of a whiteboard or a piece of paper configured to receive markings made by humans.

[0064] In some cases, company display media contains content. In other words, company display media presents information. For example, a piece of paper may contain markings in the form of handwritten, drawn, or printed information using electronic or mechanical methods. As another example, a computer monitor may display an electronic document containing text. In other cases, company display media may not contain content and may not present company information. Examples of company display media without content include, but are not limited to, a blank sheet of paper, a closed calculator monitor, and a blank lined notebook.

[0065] The depiction of a company's display media can be categorized as sensitive based on policies. Policies can be sanitization policies, including enterprise-defined policies and / or user-defined policies. In some embodiments, an enterprise policy server can be used to provide policy data that can be customized for each user. In some embodiments, enterprise policies cannot be customized for each user or can be rewritten by user-defined policies. Policies are generally data structures or other information, including a set of preferences or other criteria used to define the behavior of actions for sanitizing and sharing digital images and removing privacy violations from image files. Accordingly, enterprises can use the policy to avoid sharing or transmitting sensitive features in digital images.

[0066] Some criteria can be based on the characteristics of the company's display media being depicted. For example, criteria for classifying a depiction as sensitive could be based on the type of company's display media being depicted, whether the depiction contains content, and whether the company's display media has a display surface that is displayed or visible in the digital image. The strategy can include a list of sensitive company display media types. In some implementations, if a particular type of company display media is defined as sensitive in the strategy, and the digital image contains a depiction of that type of company display media, then the depiction is classified as, considered as, or determined to be sensitive. In some implementations, the strategy may also require the depicted company display media to contain content for the depiction to be classified as or considered sensitive. In some implementations, the strategy may also require the content of the depicted company display media to contain sensitive information for the depiction to be classified as or considered sensitive. For example, sensitive information could include sensitive words, faces, or company logos.

[0067] Various techniques and algorithms can be implemented to determine whether a digital image contains a depiction of a specific company's display medium. These algorithms may rely on one or more object or pattern recognition or detection models. The implemented algorithm may differ depending on the type of object being searched. For example, a method used to detect a face may differ from a method used to detect a computer monitor.

[0068] An object recognition method may involve image segmentation and blob analysis using object attributes such as color, texture, shape, and size. Using techniques such as contrast enhancement, digital images are segmented into fragments or sets of pixels that share some common visual features. These segmented regions or objects can be used for feature extraction. Typical features detected by feature extraction algorithms include edges, corners, connected components, and ridges. Other attributes, such as the object's color, texture, shape, and size, can also be analyzed. Various coarse and fine classification steps can be applied sequentially to an object to compare its feature set with a standard set of patterns stored in a database (e.g., patterns for smartphones, a piece of paper, a whiteboard, and other display media that can be classified as sensitive corporate display media based on a strategy) and determine the object's classification.

[0069] Another object recognition method may involve template matching, where a small image, called a template image, is used to locate matching regions within a larger source image. As the template image slides over the source image, it is compared to a region of the source image. This comparison involves determining the association between a region in the template image and the source image. The matched region is identified based on the degree of association between the template image and the source image. The template image could be an image of a company's display media stored in a database, while the source image could be a digital image that is about to be shared.

[0070] Various techniques and algorithms can be implemented to determine whether a particular depiction of a company's display medium is content-free. These algorithms may rely on one or more object or pattern recognition or detection models mentioned above. In some cases, a depiction is considered content-free if only standard patterns or content from a template image are detected in the depiction, without detecting other objects, text, or features (such as edges, corners, connected components, or ridges). For example, if a pattern of parallel, equidistant lines is identified in an image area occupied by a lined piece of paper, but no other features (such as connected components, circles, non-parallel lines, or other markings) are identified in that area, the lined paper is considered content-free. In some cases, a depiction of a company's display medium is considered content-free if the digital image area occupied by the display surface of the company's display medium is content-free, or if a specific portion of the company's display medium is content-free. Some algorithms may be based on examining the set of pixels in the digital image that contain the display surface, and confirming that the pixels have relatively identical colors. In some embodiments, a display surface is considered content-free if an area within the display surface is filled with a uniform color or color gradient.

[0071] In some embodiments, a display interface is used to display a digital image in a graphical user interface. Some or all of the objects detected in the digital image are highlighted. For example, the outline or boundary of a display medium or a portion thereof (such as a display surface), or objects identified as sensitive, may be highlighted.

[0072] Following operation 304, method 300 at operation 306 includes: in response to determining that a digital image contains a depiction of a corporate display medium classified as sensitive based on a strategy, processing the digital image to modify the depiction.

[0073] In some embodiments, processing a digital image to modify a depiction may involve modifying the depiction of the entire company's display medium. In some embodiments, processing a digital image to modify a depiction may involve modifying only a portion of the depiction (such as the display surface of the company's display medium).

[0074] A depiction can be modified in many ways. Modifying a depiction can include blurring the depiction or a portion thereof to reduce the detail in the depiction. In some cases, blurring can render the company's display medium unrecognizable, the type of the company's display medium unrecognizable, or any content of the company's display medium incomprehensible.

[0075] In some embodiments, processing a digital image to modify a depiction includes erasing the depicted corporate display medium or a portion thereof. Modifying a depiction can also involve erasing the depiction from the digital image. Erasing a depiction can involve replacing the image area occupied by the depiction of the corporate display medium with details surrounding the depiction. The result is that the depiction of the corporate display medium essentially disappears from the image. As an example, in an image depicting a piece of paper on a desk with nothing else on it, erasing the depiction of the paper could involve replacing the image area occupied by the paper with details of the writing surface. The modified image would show an empty writing desk without any paper on it. This example is as follows... Figure 7 and Figure 8 As shown. Figure 7 This is an illustration of an example digital image 700, which contains sensitive information present in the form of a stack of paper 706 and a smartphone 704. Figure 8 This is an illustration of the processed digital image 700. The modified digital image 800 has been cleaned to remove sensitive information. A stack of papers 706 has been erased and replaced with details of a writing desk surface. In this case, the grayscale of the writing desk is used to fill the area that was once occupied by the stack of papers 706. Note in this example, Figure 7 The depiction of computer monitor 702 has not been modified. The computer monitor is positioned such that its back is shown and the display screen is not visible. No modifications to the digital image 700 are necessary for the processing of computer monitor 702. Figure 8The computer monitor 702 is shown to remain unchanged.

[0076] In some embodiments, processing a digital image to modify its depiction includes erasing the content of the depicted corporate display medium. Erasing the depicted content may involve replacing the image area occupied by the content with details surrounding the content. In one example, the image may depict a lined piece of paper containing handwriting. Erasing the content of the lined paper (i.e., the handwriting) may involve determining that the lined paper includes markings (i.e., the handwriting), and removing the markings from the image by filling in the area previously occupied by the markings with details surrounding the markings on the lined paper. The modified image would show a depiction of an empty lined piece of paper. In another example, the image may depict a whiteboard containing drawings. Erasing the content of the whiteboard may involve confirming that the display surface of the whiteboard contains markings (i.e., drawings), and changing the color of the markings to match the color surrounding the markings on the whiteboard surface. Another example is... Figure 9 and Figure 10 As shown. Figure 9 This is an illustration of example image 900, which contains sensitive information present in the form of computer monitor 902, displaying the text 904 “Great Idea!”. Figure 10 This is an illustration of the processed image 1000. The modified digital image 1000 has been cleaned to remove sensitive information. Text 904 has been erased and... Figure 9 The grayscale surrounding the Chinese text 904 is replaced.

[0077] In some embodiments, processing a digital image to modify a depiction includes replacing or overlaying the depiction with a replacement object. Examples of replacement objects include icons, emojis, cartoons, shapes, advertisements, logos, depictions of corporate display media, depictions of portions of corporate display media, and depictions of the display surface of corporate display media. Replacement objects can be obtained from an object library.

[0078] When the replacement object is a shape, any suitable object or pattern can be used. In some embodiments, the replacement shape has the same boundaries or contours as the object being replaced, for example, the boundaries or contours of a sensitively depicted or sensitively displayed surface. An example is... Figure 7 and Figure 8 As shown above, Figure 7 This is an illustration of an example digital image 700, which contains sensitive information present in the form of a stack of paper 706 and a smartphone 704. Figure 8 This is an illustration of the processed digital image 700. The modified digital image 800 has been cleaned to remove sensitive information. The smartphone 704 has been replaced by a solid black connected region or shape 802 with the same shape and size as the smartphone 704. In other words, the depiction of the smartphone 704 has been filled with solid black.

[0079] A replacement object can be of the same type as the object being replaced. For example, a replacement display medium can be of the same type as the display medium being replaced. As a more concrete example, a drawing of a notebook can be replaced by a drawing of another notebook. A replacement object can also be of a different type than the object being replaced. For example, a replacement display medium can be of a different type than the display medium being replaced. As a more concrete example, a drawing of a notebook can be replaced by a drawing of a single sheet of paper.

[0080] In some cases, the replacement object may extend beyond the boundaries of the company's display media, and in others, the replacement object may occupy the same area as, or a smaller area than, the area occupied by the depiction on the company's display media or its display surface. For example, the replacement object may be a rectangle that covers the company's display surface and extends into other areas of the image. In another example, the replacement depiction of a computer monitor screen may substantially cover the same image area as the original depiction on the computer monitor screen.

[0081] In some embodiments, processing a digital image to modify a depiction includes replacing the depiction with a depiction of a display medium from the same type of company in a library, the library including another digital image containing a depiction of a display medium from another company. As an example, a depiction of lined paper may be replaced by a different depiction of lined paper obtained from a digital image library, or a depiction of a whiteboard may be replaced by a depiction of another whiteboard obtained from the library.

[0082] In some embodiments, processing a digital image to modify its depiction includes injecting false information into the digital image. For example, the content of a company's display media can be replaced with false information. As more specific examples, text on a document can be replaced with fake text, and the content of a whiteboard can be replaced with fake drawings. In some implementations, injecting false information into a digital image involves adding fake data but not replacing another object in the digital image.

[0083] For depictions of corporate display media without content, it may be necessary to avoid modifying the digital image. In some embodiments, the image is processed only if it contains corporate display media classified as sensitive and containing content. For example, if an image contains one or more corporate display media classified as sensitive and containing content, and one or more corporate display media classified as sensitive but without content, the image may be processed to modify the corporate display media classified as sensitive and containing content, but not the corporate display media classified as sensitive but without content.

[0084] In some embodiments, a digital image is processed to modify the depiction based on input received at an input interface. For example, the degree to which the depiction is blurred can be determined based on input received at the input interface.

[0085] In some embodiments, method 300 may further include: in response to processing a digital image to modify a description, generating a thumbnail based on the digital image and including the thumbnail in the metadata of the digital image.

[0086] In some cases, the digital image includes video frames, and method 300 may also include processing the video to modify the depiction in multiple frames of the video.

[0087] Method 300 also includes sharing a digital image at operation 308. Sharing a digital image may involve transferring a modified digital image to a third-party computing device or providing a copy thereof. The original digital image may be saved unmodified or deleted from the file system. For example, method 300 may be implemented by a company web server and may share the digital image by transferring the digital image to a client computing system that downloads the digital image.

[0088] Many of the embodiments described in this application focus on corporate display media. However, it is understood that this application is not limited to such embodiments, and the described embodiments can generally be readily extended to include images containing other sensitive content. Examples of other sensitive content may include faces and containers of alcoholic beverages (such as wine bottles or glasses, as well as beer bottles, cans, or mugs).

[0089] In some embodiments, method 300 may further include: in response to receiving a sharing digital image instruction, identifying a specific face in the digital image and modifying the specific face. Modifying the specific face may be implemented according to the technical and operational practices of enterprise display media described in this application, and may also include replacing the face with a cartoon. In some implementations, the specific face is modified unless the specific face is determined to correspond to the face shown on an identification badge (such as an employee ID badge), and the employee associated with that badge has provided permission to include their face in the digital image.

[0090] In some embodiments, method 300 may further include: displaying a digital image prior to processing, automatically pre-selecting depictions, providing instructions at a display interface of the pre-selected depictions, and receiving input at an input interface regarding the selection of objects depicted in the digital image. In some embodiments, input indicating the deselection of automatically pre-selected depictions may be received at the input interface. Input indicating the selection of depictions not automatically pre-selected, such as depictions of a display medium or other objects, may also be received. For example, a face in the digital image may be selected. The method may also involve processing the digital image to modify the selected depictions and objects. In cases where automatically pre-selected depictions are deselected, it may be necessary to avoid processing the digital image to modify the depictions.

[0091] The selected and deselected inputs received at the input interface can be used to update the strategy in method 300, and, for example, change the criteria used for automatically preselecting objects. More specifically, if a particular type of object or a particular face has already been selected, the strategy can be updated so that the particular type of object or the particular face will be automatically preselected in subsequent implementations of method 300. Accordingly, the strategy can be based on input regarding selections depicted in another digital image.

[0092] In some embodiments, method 300 may further include displaying metadata associated with the digital image on a display interface. A computer system implementing method 300 may include a processor coupled to the display interface, configured therein to display metadata associated with the digital image on the display interface. Examples of metadata associated with the digital image include descriptive information (including title, subject, rating, description, tags, and reviews), source information (including author, shooting date, shooting time, program name, acquisition date, copyright artist, and copyright details), image information (including image identifier, image dimensions, image width, image height, and thumbnail), camera information (including camera manufacturer, camera model, camera serial number, aperture value, exposure time, focal length, subject distance, and flash mode), and file information (including file name, file type, folder path, creation date, modification date, and size). The content of a data image file may include both metadata and image data. However, metadata associated with a digital image is not limited to metadata located within the digital image file. For example, while a title, image thumbnail, or copyright notice may be stored within the digital image file, the file may be stored outside the digital image file, in a file system directory. Accordingly, metadata associated with a digital image can be obtained from within the content of the digital image, from outside the content of the digital image, or from a combination of both.

[0093] In some embodiments, some, but not all, of the metadata associated with the digital image is displayed on the display interface. The metadata that should be displayed can be determined based on a strategy. This strategy may indicate that specific fields (such as location fields) are sensitive. In some embodiments, all sensitive metadata fields are displayed. Displaying metadata fields may involve displaying field names, field values, or both. In embodiments where the digital image is displayed on the display interface, displaying metadata may involve overlaying metadata onto the digital image.

[0094] In some embodiments, method 300 may further include automatically pre-selecting metadata fields associated with the digital image based on a strategy. In some embodiments, input indicating whether to deselect automatically pre-selected metadata fields may be received in an input interface. Input indicating selection of metadata fields that are not automatically pre-selected may also be received.

[0095] In some embodiments, method 300 may further include processing metadata to modify a pre-selected field. In some cases, the modification may involve removing the field (e.g., by deleting the set of values ​​for the field). In some cases, the modification may involve replacing the field value with other information. This other information may be true or false. As an example, an empty copyright field may be replaced with a true copyright notice. As another example, GPS coordinates in a location field may be replaced with false GPS coordinates. In some cases, the modification may involve adding a new field containing information that may include false information. False information may include randomly generated information (such as randomly generated GPS coordinates).

[0096] refer to Figure 4 The document illustrates an example method for detecting privacy violations in digital image files in the form of a flowchart. Method 400 refers to a "master" application and a "monitored" application. In this master / monitor model, the master application can represent an application with specific settings that need to be met, achieved, or complied with by the monitored application. The monitored application can represent an application whose file modification actions are monitored to determine whether they violate specific settings of the master application.

[0097] Method 400 is performed by a computing device (e.g., Figure 2 The computing device 200 implements this. The computing device has a processor configured to: acquire a policy used by the main image application; monitor the file system for digital image files modified by the monitored image application; determine that the digital image files contain at least some content that violates the defined settings for the main image application; and, in response to determining that the digital image files contain at least some content that violates the defined settings for the main image application, take action based on the determination of the violation.

[0098] The main image application and the monitored image application can be the same type or different types of image applications. For example, both the main image application and the monitored image application can be camera applications used to capture photos. As another example, the main image application can be a camera application used to capture photos, while the monitored image application can be an editing application used to modify photos captured by the camera application. As yet another example, the main image application could be... Figure 3 The example method described in the document is a cleanup application, and the monitored application can be a camera application.

[0099] Method 400 begins with operation 402. In operation 402, the main image application uses an acquisition strategy for use. In some embodiments, this strategy is substantially similar to... Figure 3 The example strategy described herein is the same. This strategy can be used by the main image application to set or adjust user preferences, or defined, default, or other settings. For example, this strategy can adjust the location settings of the main image application.

[0100] It is understandable that the settings of the main image application can be separated and differentiated from the settings of the monitored image application, but the settings of each application can provide the same or similar functionality.

[0101] Settings can correspond to one or more image metadata fields. For example, a setting can contain text used to populate a metadata field. In some embodiments, when a photograph is taken and an image file is created, the copyright artist's text and detail settings can each be copied to the copyright artist and detail metadata fields in the image data. It is understood that other metadata fields can be populated with text from other settings.

[0102] In some embodiments, a setting can enable the inclusion of metadata fields in a digital image. Specifically, the setting can indicate that a particular metadata field can be automatically populated by an image application when the digital image is modified. In some cases, the setting can be set to one of two states, such as "enabled" and "disabled." Other values, such as "on / off" or "yes / no," are also expected to be available. As an example, a location setting can correspond to a location metadata field. This location setting can control the addition of GPS location data to the metadata contained in a digital image or video file captured by the main image application. In this example, if the setting is set to "enabled," GPS location information can be added to the image metadata. If the setting is set to "disabled," no GPS location information is stored in the metadata. As another example, a date setting set to "yes" can indicate that the "date of capture" metadata field should be populated with the date the digital photo was taken, and a date setting set to "no" can indicate that the "date of capture" metadata field should not be populated. As yet another example, a thumbnail setting can indicate whether a thumbnail of the image should be generated and inserted into the image's metadata. As another example, there may be individual settings that enable the addition of various descriptive metadata such as title, topic, rating, description, tags, and comments.

[0103] Image application settings can also correspond to applications with one or more image processing capabilities. In some embodiments, settings may indicate that specific image processing capabilities should be applied to an image. For example, a stamp setting may be set to "Enabled" or "Stamp Photos" to trigger the image application to modify the image to display specific information. For example, a stamp may be overlaid on the image using the date and time the photo was captured, GPS altitude or location coordinates of the location where the photo was captured, another image (such as a copyright logo), and / or text (such as a copyright notice). In some embodiments, for example, a stamp may be formatted in yellow to visually distinguish it from the image. In some embodiments, a stamp may be hidden within the image, blended into the colors or objects surrounding the stamp in the image, and / or substantially transparent. In some cases, a stamp may be a watermark. An application may include one or more stamp settings.

[0104] Method 400 may also include, in operation 404, monitoring the file system for digital image files modified by the monitored image application. It is understood that in some cases, monitoring the entire file system of a computer system may be inefficient or unnecessary. In some embodiments, only specific directories (such as image library directories) may be monitored. In some embodiments, the file system of the computer system is monitored for digital image files modified by any application or other types of applications besides camera applications (such as photo editing applications). In some embodiments, only digital images modified by one or more specific applications are monitored.

[0105] In some embodiments, monitoring a file system for digital image files modified by a monitored image application includes continuously monitoring the file system and automatically detecting modifications made to the digital image files by the monitored image application in real time. This monitoring can be performed by a background process that continuously runs and monitors file system modification events, such as, for example, file creation events (indicating a new file is created in the monitored directory), file update events (indicating data is written to a file), file metadata update events (indicating a timestamp or other file metadata is changed), and file move events (indicating a file or subdirectory is moved to the monitored directory). In some embodiments, if one or more file system modification events associated with a digital image file occur, the file can be considered to have been modified. Once a modification event is detected, the corresponding modified file can be analyzed to determine whether the file is an image file. Various techniques can be used to determine the file type. In some cases, this determination can be based on the file extension and involves comparing the file extension of the modified file with entries in a defined list of image file extensions. If the list contains the file extension, the modified file can be considered an image file. In some embodiments, processes running on a computer system can be monitored to identify the processes and image applications that have modified the image files. In other embodiments, there may be another triggering event that causes the file system to be monitored.

[0106] In some embodiments, monitoring a file system for digital image files modified by a monitored image application includes scanning the system for the modified digital image files. The scanning operation can be performed automatically and periodically. In some cases, the scanning operation can be performed in response to input received in an input interface. In some embodiments, metadata associated with the image file (e.g., a program name metadata field) can be examined to determine the application that modified the digital image file. In some embodiments, the system can determine whether a file has been modified since the file system was last monitored. This determination may involve, for example, comparing a timestamp from a "modification date" metadata field with the timestamp of the last scan.

[0107] Method 400 further includes in operation 406: determining that the digital image file contains at least some content that violates the defined settings for the main image application. This determination may involve comparing the values ​​of the defined settings with one or more metadata fields of the digital image file.

[0108] In some embodiments, a violation may occur if a defined setting is disabled and the corresponding image metadata field exists or is set. For example, the location, capture date, and thumbnail settings applied to the main image may all be disabled. If the content of a digital image file includes GPS location information, capture date, or a thumbnail, this could be considered a violation of the respective defined settings applied to the main image. As another example, if the image metadata includes any of the fields title, subject, rating, description, tags, and comments, the disabled description setting may be violated. As yet another example, if the image metadata includes any of the fields author, capture date, capture time, program name, acquisition date, copyright artist, and copyright details, the disabled source information setting may be violated.

[0109] In some embodiments, for example, a violation may occur if a defined setting corresponding to the application of one or more image processing functions is disabled, but those image processing functions have already been applied to a digital image file. As an example, if a stamping setting applied to a main image is set to disabled, and the modified digital image includes a stamp, this could be considered a violation of that setting.

[0110] In some embodiments, for example, if a particular definition setting is enabled and includes text, but the digital image file includes a corresponding metadata field that does not match the text in the definition setting, a violation may occur. As an example, if the digital image file includes copyright metadata text that does not match the copyright text in the definition setting, a violation may occur.

[0111] In some embodiments, for example, a violation may occur if the thumbnail setting is enabled and the thumbnail metadata does not match or correspond to the digital image. In some cases, this determination may involve downscaling the dimensions of the digital image to the dimensions of the thumbnail and comparing the pixels of the thumbnail with the pixels of the downscaled digital image. In some cases, this determination may involve detecting objects depicted in both the thumbnail and the digital image and comparing the detected objects in the thumbnail with the objects in the digital image.

[0112] In some embodiments, the main image application and the monitored image application are the same image application, and method 400 verifies that the image application does not violate its own settings. In some embodiments, the main image application and the monitored application are different applications.

[0113] Any number of suitable techniques may be considered in this application. In some embodiments, method 400 may involve confirming that a digital image file includes at least some content depicting a company's display media that violates its policy, such as... Figure 3 The example method 300 described herein is generally as follows.

[0114] The method may also include taking an action in operation 408. This action may be in response to determining that the digital image file contains at least some content that violates a policy or defined settings applied to the main image. Other operations may also trigger the action. This action may be based on and involve, for example, generating a notification and / or automatically modifying the digital image file.

[0115] In some cases, notifications can trigger visual or audio alerts to provide feedback to the user. This feedback can identify the monitored image application exhibiting violations and allow the user to take corrective action to prevent further violations from the violating application. The notification can identify the monitored image application by, for example, providing the application name. The notification can also identify the violated defined settings.

[0116] In some cases, a notification can trigger the transmission of a message to a third-party system. This notification can be delivered to the user, the company's privacy policy department, or more than one entity.

[0117] Other actions can also be triggered by the notification. This notification may prompt adjustments to settings associated with the monitored image application. Specifically, the notification may prompt, for example, to toggle between defined values ​​in the settings, add text, revise existing text, add a date, update a date, remove text, or remove a date. In some embodiments, the notification may prompt adjustments to settings associated with the monitored image application to match, reflect, or comply with policies or corresponding defined settings associated with the main image application. The notification may also prompt synchronization of settings for one or more monitored image applications with policies or one or more settings of the main image application.

[0118] In cases of violations based on main image application settings that include customizable text, the notification may prompt the monitored image application to edit the settings. The notification may include text that should be used to edit the settings. For example, if the main image application includes a copyright author setting, and the setting is violated by a digital image file modified by the monitored image application, the notification may provide the name of the copyright author listed in the main image application and prompt the monitored image application to edit the copyright author setting to match the provided name. In some embodiments, the notification may prompt the removal of all text from a setting of the monitored image application. For example, the notification may prompt the removal of all copyright information in all settings of the monitored image application. As another example, the notification may prompt the editing or removal of content from the description, title, theme, rating, description, tag, and review settings of one or more monitored image applications.

[0119] In cases of violations based on defined settings set to "disabled" in the main image application, a notification can prompt the disabling of the corresponding settings in the monitored image application. For example, if GPS location settings for the main image application are used to determine that a violation has occurred, the notification can prompt the disabling of GPS location settings for the monitored image application. As another example, notifications prompting the disabling of individual settings for the monitored image application can be triggered based on one or more descriptions, titles, themes, ratings, descriptions, tags, reviews, authors, shooting dates, shooting periods, application names, acquisition times, copyright artists, copyright details, thumbnails, and stamp settings of the main image application.

[0120] The notification may also provide options to modify the digital image to comply with the policies or defined settings of the main image application. Input can be received at the input interface, indicating the selection of options for modifying the digital image to comply with the policies. In response to such input, the digital image file can be modified.

[0121] Other operations can also trigger modification of digital image files. For example, a digital image file can be modified in response to a violation of a policy by determining the content of the digital image file or the metadata associated with the digital image file.

[0122] Method 400 may involve at least modifying the content of a digital image that violates defined settings. Modifications may include, for example, adjusting metadata fields, removing metadata fields corresponding to defined settings marked as disabled for the main image application, updating metadata fields to match text for defined settings applied to the main image application, generating thumbnails and inserting thumbnails into the metadata, and removing stamps from the image.

[0123] In method 400, the modification of the digital image file and associated metadata can generally be based on the description in this application. Figure 3 The example method 300 described herein is implemented by modifying digital image-related techniques and operations.

[0124] It is also desirable that the various methods described above be presented in flowchart form to show the order of operations for illustration and discussion. However, in some implementations, different order of operations may be used without changing the substance of the process, additional operations may be included, and / or some sequentially shown operations may occur simultaneously or in parallel.

[0125] It will be understood that applications, modules, routines, processes, threads, or other software components implementing the described methods / processes can be implemented using standard computer programming techniques and languages. This application is not limited to specific processors, computer languages, computer programming conventions, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated circuit (ASIC), etc.

[0126] Certain modifications and alterations can be made to the described embodiments. Therefore, the embodiments discussed above are considered illustrative and not restrictive.

Claims

1. A computer-implemented method for detecting privacy violations in image files, the method comprising: Get the strategy that will be used by the main image application; Monitor the file system for digital image files that have been modified by the monitored image application; The digital image file was determined to contain at least some content that violated the defined settings for the application of the main image. as well as In response to determining that the digital image file contains at least some content that violates the defined settings applied to the main image, an action is taken. The action includes generating a notification based on the violation, which prompts adjustments to settings associated with the monitored image application to comply with the policy used by the main image application.

2. The method according to claim 1, wherein, The monitoring file system for digital image files modified by the monitored image application includes: continuously monitoring the file system and automatically detecting, in real time, the modifications made to the digital image files by the monitored image application.

3. The method according to claim 1, wherein, Monitoring the file system for digital image files modified by the monitored image application includes: automatically and periodically scanning the file system for digital image files modified by the monitored image application.

4. The method according to claim 1, wherein, Monitoring the file system for digital image files modified by the monitored image application includes: scanning the file system for digital image files modified by the monitored image application in response to input received at an input interface.

5. The method according to claim 1, wherein, Taking the action includes processing the digital image file to modify at least some of its contents.

6. The method according to claim 1, wherein, The notification identifies the monitored image application.

7. The method of claim 1, wherein the notification provides an option to modify the digital image file to comply with the policy.

8. The method according to claim 7, further comprising: Receive input at the input interface, indicating the option to modify the digital image file to comply with the policy; as well as In response to receiving input from the input interface to select the option to modify the digital image file, the digital image file is modified.

9. A computing device, comprising: Memory, and A processor, coupled to the memory, is configured to: Get the strategy that will be used by the main image application; Monitor the file system for digital image files that have been modified by the monitored image application; The digital image file was determined to contain at least some content that violated the defined settings for the application of the main image. as well as In response to determining that the digital image file contains at least some content that violates the defined settings applied to the main image, an action is taken. The action includes generating a notification based on the violation, which prompts adjustments to settings associated with the monitored image application to comply with the policy used by the main image application.

10. The device according to claim 9, wherein, The processor is configured to monitor the file system for digital image files modified by the monitored images, including: The processor is configured to continuously monitor the file system and automatically detect, in real time, any modifications made to the digital image files by the monitored image applications.

11. The device according to claim 9, wherein, The processor is configured to monitor the file system for digital image files modified by the monitored images, including: The processor is configured to automatically and periodically scan the file system for digital image files modified by the monitored images.

12. The device of claim 9, wherein the processor is further configured to perform the action, comprising: The processor is configured to process the digital image file to modify at least some of its contents.

13. The device according to claim 9, wherein, The notification identifies the monitored image application.

14. The device of claim 9, wherein the notification provides an option to modify the digital image file to comply with the policy.

15. The device of claim 14, wherein the processor is further configured to: Receive input at the input interface, selecting the option to modify the digital image file to comply with the policy; and In response to receiving input from the input interface to select the option to modify the digital image file, the digital image file is modified.

16. A non-transitory computer-readable storage medium storing processor-executable instructions for detecting privacy violations in an image file, wherein the processor-executable instructions, when executed by a processor, cause the processor to: Get the strategy that will be used by the main image application; Monitor the file system for digital image files that have been modified by the monitored image application; The digital image file is determined to contain at least some content that violates the defined settings for the main image application; and In response to determining that the digital image file contains at least some content that violates the defined settings applied to the main image, an action is taken. in, Taking the action includes generating a notification based on the violation, which prompts adjustments to settings associated with the monitored image application to comply with the policy used by the main image application.

Images