Third party access to end user device assets

Abstract

The present disclosure relates to third-party access to end-user device assets. The present disclosure relates to systems, methods, and computer-readable media for identifying asset privacy management triggers related to third-party applications on an end-user device. In response to identifying an asset privacy management trigger, a privacy selection interface is displayed for enabling a user to select a restricted asset access option. In response to selecting the restricted asset access option, an asset selection interface is displayed, where the asset selection interface is configured to define a subset of assets of the end-user device as authorized for the third-party application based on user selection. In response to a subsequent request for access to assets of the end-user device by the third-party application, the third-party application is only able to access the defined subset of assets. The asset privacy management triggers and asset subset definitions can vary for different third-party applications or scenarios.

Description

[0001] Cross-references to related applications

[0002] This patent application claims priority to U.S. Patent Application No. 16 / 906593, filed June 19, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates generally to the field of asset management for user devices in a networked environment. More specifically, this disclosure relates to systems and processes for enabling users of end-user devices to restrict access to the assets of end-user devices by third-party applications. Background Technology

[0004] Currently, many end-user devices, including portable devices (e.g., smartphones and tablets), offer audio and image capture capabilities, resulting in audio recordings, photos, and video recordings. For example, a typical end-user device (e.g., a home computer, laptop, camera, tablet, or mobile phone) user can capture both still images and videos via the device's camera or camera interface. The captured images or videos can then be accessed in the photo library on the end-user device (or other networked devices for which the end-user has granted access to the assets). Many third-party applications offer features involving access to user assets in the end-user device's photo library. Currently, users typically only have a binary option (i.e., allow full access to assets or deny full access to assets) for managing access to end-user device assets such as the photo library by third-party applications. This binary option is typically a one-time selection for each third-party application (e.g., specified when the end-user first uses the third-party application) and can be perceived as overly restrictive or over-granted for many end-user device users. Therefore, technologies to improve the security and availability of end-user device assets in networked environments with third-party applications are desirable. Summary of the Invention

[0005] Typical user interaction with third-party applications involves installing the third-party application on an end-user device and executing it. Another option is to use the end-user device to access websites, social networks, or other platforms, as well as related features associated with the third-party application. In either case, the end-user device has assets, and the third-party application includes features for accessing those assets. Exemplary assets of the end-user device include pictures or videos in a photo library, a microphone, or a camera. The disclosed systems and processes do not employ binary access schemes (e.g., full access to assets or no access to assets), but rather allow restricted and / or application-specific access to the assets of the end-user device. Exemplary features of third-party applications that can access the assets of the end-user device include teleconferencing features, photo editing features, social media features, and online storage features.

[0006] The embodiments of this disclosure attempt to provide users of end-user devices with improved asset privacy management options by adding restricted asset access options. In some examples, the restricted asset access option is a feature of the operating system and can be accessed via one or more settings within the end-user device's operating system. Once the operating system is updated, the restricted asset access option can be selected in response to an asset privacy management trigger (e.g., after the end-user device is rebooted following an operating system update to include the restricted asset access option, receiving an asset privacy management request from a third-party application, first installation or use of a third-party application, using a third-party application for a predetermined amount of time, receiving an asset access request from a third-party application, or selecting an operating system asset management setting associated with a third-party application). Once an asset privacy management trigger is recognized, a privacy selection interface is displayed, allowing the user to select a restricted asset access option for a given third-party application. In some examples, the privacy selection interface allows the user to choose between a full asset access option (e.g., access to all photos in a photo library), a no-asset access option (e.g., access denied, making photos in the photo library inaccessible), and a restricted asset access option (e.g., access to some photos in the photo library, but not access to others).

[0007] When the restricted asset access option is selected, an asset selection interface is displayed, allowing the user to define a subset of assets that are authorized for use by third-party applications. In some examples, the subset of assets is defined by the user selecting individual assets or asset groups (e.g., folder-based, date-based, or content-based definitions). The third-party application is then able to access the subset of assets defined as authorized for its use. Asset privacy management may be performed multiple times for the same third-party application as needed (e.g., based on asset privacy management scheduling or in response to new asset availability). For example, if a third-party application requests access to the end-user device's photo library, and new photos have been added to the photo library since the last time the third-party application accessed the photo library, the request to access the photo library can be identified as an asset privacy management trigger.

[0008] One consideration for the proposed asset privacy management technology involves addressing different third-party application scenarios, including backward compatibility with traditional privacy management interfaces. In some cases, third-party applications use traditional request interfaces to communicate with end-user devices, where the third-party application may or may not have an asset privacy policy adapted to the end-user device. When using a traditional request interface, a request for access to assets by the third-party application can be identified as an asset privacy management trigger. In response to an asset privacy management trigger, a privacy selection interface for selecting asset privacy management options and an asset selection interface for defining a subset of restricted assets (if the restricted asset option is selected) can be used with the traditional request interface in a transparent manner to the third-party application. As used herein, "transparent" means that the third-party application is unaware of the given interface or related actions.

[0009] In one example, even if a third-party application has previously been granted full access to the end-user device's photo library, access to the photo library can be restricted using privacy selection and photo selection interface operations as described herein. In this example, the restricted photo library option selected by the user and related operations results in a virtual access policy for the third-party application's access to the photo library. From the third-party application's perspective, full access to the photo library is provided, where the photo library is restricted to a user-defined subset of photos via a virtual access policy.

[0010] In other scenarios, third-party applications using updated request interfaces have an opt-out option, allowing them to use traditional binary indicators (e.g., full access or no access) with the updated request interface. In such scenarios, a third-party application can request access to assets on an end-user device, where a privacy selection interface for selecting asset privacy management options in response to the request and an asset selection interface for defining a restricted subset of assets (if the restricted asset option is selected) are implemented transparently to the third-party application. If the user selects the restricted asset access option for the third-party application, a virtual access interface can be used to restrict the assets authorized for use by the third-party application, as needed. From the third-party application's perspective, access to assets on the end-user device is provided based on traditional binary options (full access or no access). However, if the restricted asset access option is selected by the user, the assets visible to the third-party application are restricted via the virtual access interface to a subset of assets defined by the asset selection interface (as if the entire subset of assets existed).

[0011] In other scenarios, third-party applications using the updated request interface have an opt-in option that allows the third-party application to request asset privacy management from the user, where the end-user device returns a non-binary value indicating whether full access, restricted access, or no access is authorized. In such cases, the third-party application may or may not have an asset privacy policy adapted to the end-user device. In either case, the third-party application can request asset privacy management from the user. As another option, the third-party application can request access to the assets of the end-user device. In response to an asset privacy management trigger, a privacy selection interface for selecting asset privacy management options and an asset selection interface for defining a subset of restricted assets (if the restricted asset option is selected) are used, where the third-party application receives a non-binary indication of the asset privacy option selected by the user. Utilizing the updated request interface and opt-in option, once the user has made an asset privacy decision, the third-party application cannot submit new privacy management requests. However, the end-user can change the asset access level in the settings / preferences user interface (UI).

[0012] In one example, an asset privacy management request made by a third-party application allows the user and related actions to select a restricted photo library option. In this case, a virtual access policy for the photo library is established for the third-party application. From the third-party application's perspective, restricted access to the photo library is provided, where the photo library is limited to a subset of photos defined by the user through the virtual access policy. Attached Figure Description

[0013] Figure 1This is a schematic diagram illustrating a representative hardware environment.

[0014] Figure 2 This is a schematic diagram illustrating a representative network environment.

[0015] Figure 3 This is a schematic diagram illustrating a representative software architecture.

[0016] Figure 4 This is a schematic diagram illustrating a photo library privacy management scenario.

[0017] Figures 5 to 8 This is a flowchart illustrating an asset privacy management method according to some embodiments of this disclosure. Detailed Implementation

[0018] This disclosure relates to systems, methods, and computer-readable media for improving asset privacy management on end-user devices by supporting restricted asset management options. This disclosure also relates to supporting various asset privacy management triggers (e.g., rebooting the end-user device after updating the operating system to include restricted asset management options, receiving an asset privacy management request from a third-party application, receiving an asset access request from a third-party application, first installation or use of a third-party application, using a third-party application for a predetermined amount of time, or selecting operating system asset management settings associated with a third-party application), which trigger a user to select an asset privacy management option. If the restricted asset management option is selected by the user for a given third-party application, an asset selection interface is displayed so that the user can define a subset of assets as authorized for use by the third-party application. In some examples, the user selects between full asset access options, no asset access options, and restricted asset access options for each of multiple third-party applications. For the restricted asset access option, the subset of assets is defined by the user selecting individual assets or groups of assets (e.g., a folder-based definition, a date-based definition, or a content-based definition). The third-party application is then able to access the subset of assets defined as authorized for use by the third-party application. Perform asset privacy management multiple times for the same third-party application as needed (e.g., based on asset privacy management scheduling, new asset availability, or other triggers).

[0019] In one example, in response to an asset privacy management trigger, asset privacy management actions are performed on privacy-sensitive resources such as the photo library, contacts, or calendar. After the user selects asset privacy options and defines restricted assets (if necessary), the relevant policies are enforced and used for future requests from third-party applications. Subsequent asset privacy management triggers allow the user to confirm or change their asset privacy management selections as needed. For example, if a third-party application requests access to the end-user device's photo library, and new photos have been added to the photo library since the last time the third-party application accessed it, the request to access the photo library can be identified as an asset privacy management trigger, even if the third-party application previously received full or restricted access authorization. Similarly, if a third-party application requests access to the end-user device's contacts, and new contact items have been added to the contacts since the last time the third-party application accessed them, the request to access the contacts can be identified as an asset privacy management trigger, even if the third-party application previously received full or restricted access authorization. Likewise, if a third-party application requests access to the end-user device's calendar, and new calendar items have been added to the calendar since the last time the third-party application accessed it, the request to access the calendar can be identified as an asset privacy management trigger, even if the third-party application previously received full or restricted access authorization. In addition, this article discusses many other asset privacy management technologies and options.

[0020] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the disclosed concepts. As part of this description, some of the accompanying drawings of this disclosure are block diagrams illustrating structures and devices to avoid obscuring the novel aspects of the disclosed concepts. For clarity, not all features of actual specific embodiments are described. Furthermore, the language used in this disclosure has been chosen primarily for readability and instructional purposes and may not have been selected to describe or limit the subject matter of the invention, thus requiring the claims to determine such subject matter. References to “an embodiment” or “an embodiment” in this disclosure mean that a particular feature, structure, or characteristic described in this disclosure is included in at least one embodiment of the disclosed subject matter, and the repeated references to “an embodiment” or “an embodiment” should not be construed as necessarily referring to all of the same embodiments.

[0021] It should be understood that in any actual implementation of development (as in any software and / or hardware development project), numerous decisions must be made to achieve the developer's specific objectives (e.g., compliance with system and business-related constraints), and these objectives may differ between different implementations. It should also be understood that such development work can be complex and time-consuming, but nevertheless, it will be a routine task for those who benefit from this disclosure and for those skilled in the art in designing and implementing computing and / or graphics systems.

[0022] Exemplary hardware and software

[0023] The inventive embodiments described herein relate to asset privacy management operations on end-user devices. Specifically, the proposed asset privacy management operations include restricted asset privacy options. In an exemplary embodiment, the end-user device includes a camera and microphone to capture photos and videos, wherein the captured photos, captured videos, the end-user device microphone, and / or the end-user device camera correspond to assets of the end-user device. The proposed asset privacy management operations do not support binary-only access options (e.g., full access or no access), but instead allow restricted asset access options, wherein the user of the end-user device can define a subset of assets as authorized for use by third-party applications. Because many embodiments rely on computing operations and systems, this disclosure is applicable and usable in and relative to all types of smart devices, including single-processor and multi-processor computing systems and vertical devices (e.g., cameras, gaming systems, appliances, etc.) containing single-processor and multi-processor computing systems. The discussion herein is made with reference to a common computing configuration that can be discussed as a server system or an end-user system. This common computing configuration may have CPU resources including one or more microprocessors. This discussion is for illustrative purposes only regarding sample implementations and is not intended to limit the application of this disclosure to the disclosed hardware. Other systems with other (now or in the future) known or common hardware configurations are also fully envisioned and anticipated. Taking into account the foregoing statements, a typical hardware and software operating environment is discussed below. Hardware configurations may, for example, reside in cameras, security systems, servers, workstations, laptops, tablets, desktop computers, gaming platforms (whether portable or not), televisions, entertainment systems, smartphones, telephones, or any other computing device (whether mobile or stationary).

[0024] See Figure 1The disclosed implementation scheme can be executed by a representative computer system 100. For example, the representative computer system 100 can act as a server or end-user device. System 100 can be implemented in any type of device, such as a camera, general-purpose computer system, television, set-top box, media player, multimedia entertainment system, image processing workstation, handheld device (such as a telephone), or any device that can be coupled to or combined with image capture, audio capture, and / or processing capabilities. Computer system 100 may include one or more processors 105, memory 110 (110A and 110B), one or more storage devices 115, and graphics hardware 120. The computing system 100 may also include device sensors 125, which may include one or more of the following: depth sensors (such as depth cameras), 3D depth sensors, imaging devices (such as fixed image capture units and / or video-enabled image capture units), RGB sensors, proximity sensors, ambient light sensors, accelerometers, gyroscopes, any type of still or video camera, LiDAR devices, SONAR devices, microphones, CCDs (or other image sensors), infrared sensors, thermometers, etc. These and other sensors may be combined with one or more GPUs, DSPs, or conventional microprocessors, with appropriate programming, so that the sensor outputs can be correctly interpreted and / or combined and interpreted.

[0025] return Figure 1 System 100 may also include a communication interface 130, a user interface adapter 135, and a display adapter 140, all of which may be coupled via a system bus or backplane 145. Memory 110 may include one or more different types of media (e.g., solid-state, DRAM, optical, magnetic, etc.) used by processor 105 or graphics hardware 120. For example, memory 110 may include memory cache, read-only memory (ROM), and / or random access memory (RAM). Storage device 115 may include one or more non-transitory storage media, including, for example, magnetic disks (fixed disks, floppy disks, and removable disks) and magnetic tape, optical media (such as CD-ROMs and digital video optical discs (DVDs)), and semiconductor memory devices (such as electrically programmable read-only memory (EPROM) and electrically erasable programmable read-only memory (EEPROM)). Memory 110 and storage device 115 may be used to store media (e.g., audio, image, and video files), preference information, device profile information, computer program instructions organized into one or more modules and written in any desired computer programming language, and any other suitable data. When executed by processor 105 and / or graphics hardware 120 (which may also be a processor), such computer program code may implement one or more of the methods or processes described herein.

[0026] Communication interface 130 may include semiconductor-based circuitry and may be used to connect computer system 100 to one or more networks. Exemplary networks include, but are not limited to: local area networks, such as USB networks; business local area networks; and wide area networks, such as the Internet; and any suitable technology (e.g., wired or wireless technology) may be used. Possible communication technologies include cellular-based communications (e.g., NR, LTE, CDMA, GSM, HSDPA, etc.) or other communications (Ethernet, WiFi, Bluetooth, USB, Thunderbolt, Firewire, etc.). User interface adapter 135 may be used to connect keyboard 150, microphone 155, pointing device 160, speaker 165, and other user interface devices such as touchpads and / or touchscreens (not shown). Display adapter 140 may be used to connect one or more display units 170 via a frame buffer (not shown).

[0027] Processor 105 can execute instructions necessary for implementing or controlling various functions performed by system 100 (e.g., image evaluation or processing). Processor 105 can, for example, drive display 170 and receive user input from user interface adapter 135 or any other user interface implemented by the system. User interface adapter 135 can, for example, present various forms such as buttons, keypad, dial pad, click wheel, keyboard, display screen and / or touchscreen or any combination thereof. User interface items or desktop applets can be generated in real time by graphics hardware 120 as the user interacts with the interface. Processor 105 can be any type of computing device such as one or more microprocessors working alone or in combination with a GPU, DSP, and / or system-on-a-chip devices such as those found in mobile devices. Processor 105 may include one or more dedicated GPUs or graphics subsystems that accept program instructions to generate or modify display information such as pixels. Furthermore, processor 105 can be based on a Reduced Instruction Set Computer (RISC) or Complex Instruction Set Computer (CISC) architecture or any other suitable architecture and may include one or more processing cores. The graphics hardware 120 may be dedicated computing hardware for processing graphics and / or for coprocessor 105 to perform computational tasks. In some embodiments, the graphics hardware 120 may include CPU-integrated graphics and / or one or more programmable GPUs.

[0028] Various embodiments of this disclosure may use sensors such as cameras. Cameras and similar sensor systems may include autofocus systems for accurately capturing video or image data ultimately used to interpret user intentions or commands. Since user movement can be based on subtle movements in small regions (e.g., hands, fingers, face, mouth, forehead, etc.) of the captured image, the autofocus system can be used to focus separately on multiple regions of the image to obtain better information.

[0029] return Figure 1 Sensor 125 can capture scene and / or environmental phenomena, such as time; location information; the state of the device relative to light, gravity, and magnetic north pole; and even still and video images. Furthermore, network-accessible information such as weather information can also be used as part of the scene. All captured scene and environmental phenomena can be used to provide context for user or subject activities or information about user or subject activities. For example, when acquiring a series of captured images, scene information can be used as part of analysis. System 100 can react to environmental and scene events and reflect the reaction in real time, for example, by attaching scene information to images / frames, performing one or more functions such as image capture using specific modes, or by inducing activity on a display system using graphics hardware 120.

[0030] The output from sensor 125 may be processed, at least in part, by processor 105 and / or graphics hardware 120, and / or by a dedicated image processing unit integrated within or without system 100. The information thus captured may be stored in memory 110 and / or storage device 115 and / or in any storage device accessible on an attached network (such as the Internet). Memory 110 may include one or more different types of media used by processor 105, graphics hardware 120, and sensor 125 to perform device functions. Storage device 115 may store data such as media (e.g., audio, image, and video files); metadata of the media; computer program instructions; and other software, including database applications (e.g., a database storing character frames), preference information, device profile information, and any other suitable data. Memory 110 and storage device 115 may be used to hold computer program instructions or code, organized into one or more modules in compiled form or written in any desired computer programming language. When executed by, for example, processor 105, such computer program code may perform one or more of the behaviors or functions described herein (e.g., performing image analysis and trimming).

[0031] exist Figure 1In the example, memory 110 and / or storage device 115 are represented as including asset privacy management interface instructions 116 executed by processor 105. When executed by processor 105, asset privacy management interface instructions 116 perform operations such as: identifying asset privacy management triggers, defining a subset of assets for each third-party application to authorize restricted asset access as needed based on user selection, storing asset privacy policy information for future use, and providing asset privacy management transparency options for different third-party applications (to support backward compatibility with legacy request interfaces or legacy binary indicators). Additional details and options related to asset privacy management interface instructions 116 are provided below.

[0032] See now Figure 2 The proposed asset privacy management technology can be implemented in an exemplary network architecture 200 comprising multiple networks 205 (i.e., 205A, 205B, and 205C), each of which can take any form, including but not limited to a local area network (LAN) or wide area network (WAN) such as the Internet. Additionally, networks 205 can use any desired technology (wired, wireless, or a combination thereof) and protocol (e.g., Transmission Control Protocol TCP). Coupled to network 205 is a data server computer 210 (i.e., 210A and 210B), which is capable of operating server applications such as databases and also capable of communicating over network 205. One implementation using the server computer may involve operating one or more central systems to collect, process, and evaluate image information (e.g., frames) and / or contextual information, or other information acting as an agent for mobile computing devices such as smartphones or network-connected tablets.

[0033] Also coupled to network 205 and / or data server computer 210 are client computers or end-user devices 215 (i.e., 215A, 215B, and 215C), which can take the form of any computer, set-top box, entertainment device, communication device, or smart machine (including embedded systems). In some implementations, the user may use a client computer in the form of a smartphone or tablet. Figure 2 In the example, each client computer in client computer 215 includes hardware (see example...) Figure 1The network architecture 200 includes corresponding asset privacy management interface instructions 116A to 116C to support the proposed asset privacy management techniques described herein. Additionally, in some embodiments, the network architecture 200 may also include network printers such as printer 220 and storage systems such as 225, which can be used to store multimedia items (e.g., images) mentioned herein. To facilitate communication between different network devices (e.g., data server 210, end-user computer 215, network printer 220, and storage system 225), at least one gateway or router 230 may optionally be coupled between them. Furthermore, to facilitate such communication, each device utilizing the network may include network adapter circuitry. For example, if an Ethernet network is expected to be used for communication, each participating device must have an Ethernet adapter or an embedded IC that supports Ethernet. Furthermore, devices may carry network adapters for any network in which they may participate.

[0034] As described above, embodiments of the invention disclosed herein include software or asset privacy management interface instructions 116. In this regard, a description of common computing software architectures is provided, such as... Figure 3 The layer diagram is illustrated herein. Similar to the hardware example, the software architecture discussed here is not intended to be exclusive in any way, but rather exemplary. This is especially true for layer type diagrams, which software developers often represent in a slightly different manner. In this case, the description begins with the layer starting from the O / S kernel 310; therefore, low-level software and firmware have been omitted from the illustration rather than from the intended implementation. The notation used here is generally intended to indicate that the software elements shown in the layers use resources from the layers below and provide services to the layers above. However, in practice, not all components of a particular software element may function exactly in this manner.

[0035] See also those statements about the software. Figure 3Layer 310 is the O / S kernel, which provides core O / S functionality in a protected environment. Above the O / S kernel is Layer 320, the O / S core services, which extends functional services to the layers above, such as disk and communication access. Layer 330 is inserted to illustrate the general relative positioning of the Open Graphics (Open GL) library 332 and similar applications, as well as privacy-sensitive resources 334. As used herein, privacy-sensitive resources 334 encompass the scope of system resources accessed by the framework access control system. Utilizing the described technique, access to privacy-sensitive resources 334 (e.g., photo library, contacts, or calendar) that have assets (e.g., photos or videos, contact items, or calendar items) is managed at the asset layer or a subset of the asset layer, rather than the entire resource layer. Layer 340 is a merging of functionality, typically represented as multiple layers: application framework and application services. For the purposes of our discussion, these layers can provide a high level and generally functional support for applications residing in the highest layer (shown here as layer 350). Exemplary applications in the higher layer 350 include third-party application 351, photo application 352, financial application 353, movie application 354, and another third-party application 355. Figure 3 In the examples, photo application 352, financial application 353, and movie application 354 are examples of main applications (not third-party applications) that do not require requesting asset access to access privacy-sensitive resource 334. In other exemplary embodiments, applications in the higher layer 350 include additional applications and / or omit one or more applications among those represented.

[0036] In some examples, the access control system used to provide asset privacy management operations attributes access requests to an entity that the user knows (i.e., interacts with). Typically, third-party applications 351 and 355 are "applications" from an "app store." In other cases, each of third-party applications 351 and 355 can be a Swift Playground, which is part of the Swift programming language development environment. In this case, the third-party application is a third-party entity with a name or identity, and the end user can install, interact with, and delete that third-party entity.

[0037] Project 360 aims to demonstrate asset privacy management interface instructions or software (e.g., Figure 1 and Figure 2The asset privacy management interface (116) is located within the system architecture of a general relative position within the system, which can execute the asset privacy management techniques described herein. An asset control system abstraction is implemented between the application framework and the boundary of privacy-sensitive resources (databases and media). In other words, there is a process boundary between the application framework and the implementation of restricted library policies to ensure that any malicious third party cannot circumvent the policy. Specifically, in some implementations, the asset privacy management techniques discussed herein can be executed by framework software using an application programming interface (API). In some implementations, the framework software (or other software) accessible via an API can identify asset privacy management triggers and provide options related to restricted asset access as described herein. Of course, the application can also perform the same functionality without the aid of a framework. Furthermore, on the server side, some implementations described herein can be implemented using a combination of server application-level software and database software, either of which may include a framework and / or multiple resource modules. In some implementations, the server can be accessed over a network to perform asset privacy management operations by a portable device.

[0038] In some examples, asset privacy management instructions or software 116 are as follows: Figure 4 This is part of the photo library privacy management scenario 400 shown. Figure 4 In the example, photo library privacy management scenario 400 includes a photo library 402, which corresponds to cloud-based photos (e.g., photos uploaded from an end-user device to a user's account) and / or a local photo library, wherein photo library 402 is accessible via photo library interface 404. Photo library interface 404 (e.g., the PhotoKit interface from Apple Inc.) provides the following categories: support for photo applications on end-user devices (e.g., ... Figure 3 The photo editing extension is built for the photo application (352). For different operating systems (e.g., ...), and Operating systems (where iOS is a registered trademark of Cisco Technology Inc., and macOS and tvOS are registered trademarks of Apple Inc.) The Photo Gallery interface 404 also provides direct access to photo and video assets managed by the Photos application. Exemplary operations 410 processed by the Photo Gallery interface 404 include retrieving and caching assets of the Photo Gallery 402 for display and playback, editing image and video content, or managing collections of assets such as albums or shared albums. Exemplary items 408 supported by the Photo Gallery interface 404 include prints, slideshows, books, calendars, cards, and wallpapers.

[0039] exist Figure 4In the photo library privacy management scenario 400, a third-party application 406 is shown, which is capable of submitting asset access requests 412 and / or related communications to the photo library interface 404. It should be understood that the third-party application 406 shown herein may alternatively include third-party websites, social networks, or other online platforms. The photo library interface 404 is capable of providing a response 414 to such requests based on asset privacy management instructions or software 116. In the proposed implementation, the asset privacy management instructions or software 116 supports restricted asset access options, wherein a subset of photos or videos in the photo library 402 is defined as authorized for use by the third-party application 406 based on user selection. In some examples, the user can, for example, choose between a full asset access option, a no-asset access option, and a restricted asset access option in response to different asset privacy management triggers. Figure 4 In the photo library privacy management scenario 400, a restricted library policy is applied to photo library 402. For further clarity, photo library interface 404 is instantiated in a third-party process, and photo library 402 runs in an out-of-process service that manages access to assets containing privacy-sensitive resources. The restricted library policy is implemented in this out-of-process photo service.

[0040] Figures 5 to 8 This is a flowchart illustrating an asset privacy management method according to some embodiments of this disclosure. Figure 5 In the asset privacy management method 500, at box 505, an asset privacy management trigger related to a third-party application is identified. Exemplary asset privacy management triggers include rebooting the end-user device after updating the operating system to include a restricted asset access option, receiving an asset privacy management request from a third-party application, receiving an asset access request from a third-party application, first-time installation or use of a third-party application, using a third-party application for a predetermined amount of time, or selecting operating system asset management settings related to a third-party application.

[0041] In response to the detection of an asset privacy management trigger, a privacy selection interface is displayed, allowing the user to choose a restricted asset access option at box 510. In some examples, the privacy selection interface allows the user to choose between a full asset access option, a no asset access option, and a restricted asset access option. In response to the selection of a restricted asset access option, an asset selection interface is displayed at box 515, where the asset selection interface is configured to define a subset of assets as authorized for use by third-party applications based on the user's selection. Exemplary assets of an end-user device include photos or videos in a photo library, microphones, cameras, and audio recording libraries. In different examples, a subset of assets is defined using a folder-based definition, a date-based definition, or a content-based definition. Other subset definitions are possible and can even rely on image analysis (e.g., facial recognition), negative definitions (e.g., excluding photos of a specific type, date, or folder), or dynamic subsets (e.g., including all photos in a given folder, even if new photos are added to the folder). In some examples, the actions of boxes 510 and 515 are provided by a UI that allows the user to select the scope of access and choose a subset of assets in one action. For example, an empty set of selected assets can be interpreted as denied access, a partial set of selected assets can be interpreted as restricted asset access, and all selected assets can be interpreted as full access. In response to a request for access to assets on an end-user device by a third-party application, at box 520, access to a defined subset of assets is provided to the third-party application. In some examples, the operation of box 520 involves adding the definition of the asset subset to an asset privacy management database that stores index information associating each subset of a plurality of defined asset subsets with a corresponding third-party application identifier. In this case, in response to the recognition of a request for access to assets on an end-user device by a third-party application, the operation of box 520 involves using the index information in the database to provide the third-party application with access only to the defined subset of assets.

[0042] Other options for the asset privacy management method 500 include using transparent operations regarding third-party applications. For example, the operations of the privacy selection interface at box 510 and the photo selection interface at box 515 can be transparent to third-party applications. As used herein, "transparent" means that the third-party application is unaware of a given interface or related operation. In other examples, the third-party application is at least aware that the user has selected the restricted asset access option. Another option involves resetting the asset privacy management trigger or otherwise identifying new asset privacy management triggers after an asset privacy policy has been established with the third-party application. In one example, a new request for access to assets on the end-user's device by a third-party application is interpreted as an asset privacy management trigger in response to the recognition that a new asset has been added since the last update of the asset privacy management database. As needed, the asset privacy management operation is repeated for each third-party application as new assets become available.

[0043] Typical user interaction with a third-party application involves installing the application on an end-user device and executing it. Another option is to use the end-user device to access websites and related features associated with the third-party application. In either case, the end-user device has assets (e.g., a photo library, microphone, camera, audio recording library), and the third-party application includes features for accessing those assets. Exemplary features of a third-party application that accesses assets on the end-user device include teleconferencing features, photo editing features, social media features, and online storage features.

[0044] The embodiments of this disclosure attempt to provide users of end-user devices with improved asset privacy management options by adding a restricted asset access option. In some examples, the restricted asset access option is added by updating the operating system of the end-user device. Once the operating system is updated, the restricted asset access option can be selected in response to an asset privacy management trigger, as described herein. Once an asset privacy management trigger is detected, a privacy selection interface is displayed, which allows the user to select a restricted asset access option for a given third-party application. In some examples, the privacy selection interface allows the user to choose between a full asset access (e.g., access to all photos in a photo library), no asset access (e.g., access to zero photos in a photo library), and restricted asset access (e.g., access to a limited number of photos in a photo library).

[0045] When the restricted asset access option is selected, an asset selection interface is displayed, allowing users to define a subset of assets as authorized for use by third-party applications. The third-party application can then access the subset of assets defined as authorized for its use. Asset privacy management can be performed multiple times for the same third-party application as needed.

[0046] One consideration of the proposed asset privacy management technology involves handling different third-party application scenarios. In some cases, third-party applications use traditional request interfaces to communicate with end-user devices, where the third-party application may or may not have an asset privacy policy adapted to the end-user device. Such scenarios... Figure 6 As stated in Asset Privacy Management Method 600. In Asset Privacy Management Method 600, third-party applications use a traditional request interface to request access to photos in the end-user device's photo library (see, for example...). Figure 4 In this scenario, the installation of an updated operating system with restricted asset access options can be considered an asset privacy management trigger, overriding previous photo library privacy management policies.

[0047] As shown in the figure, the asset privacy management method 600 includes receiving a request for access to a photo library from a third-party application at box 605. At box 610, photo library privacy management is triggered. At box 615, a privacy selection interface is displayed, and the user's selection is received. At box 620, if the user selects the "deny access" option at box 615, access to the photo library by the third-party application is denied. At box 625, if the user selects the "full access" option at box 615, full access to the photo library is allowed. If the user selects the "restricted access" option, at box 630, a photo selection interface is displayed, and this photo selection interface is configured to define a subset of photos in the photo library as authorized for use by the third-party application based on the user's selection. At box 635, access to the defined subset of photos is provided. In the asset privacy management method 600, the use of the restricted access options and related operations in boxes 615, 630, and 635 is transparent to the third-party application. From the perspective of third-party applications, access to the photo library is either allowed or not allowed, and the definition of the subset of photos used to restrict the photos available to third-party applications is unknown to them.

[0048] In some examples, asset privacy management method 600 involves identifying a request for access to assets by a third-party application as an asset privacy management trigger, wherein the end-user device is configured to return a value indicating whether full access is authorized or access is not authorized based on user selection. In response to the asset privacy management trigger, a privacy selection interface for selecting asset privacy management options and an asset selection interface for defining a restricted subset of assets (i.e., if the restricted asset option is selected) are used in a manner transparent to the third-party application, using a conventional request interface. In one example, even if a third-party application has previously been granted full access to the end-user device's photo library, access to the photo library can be restricted using the privacy selection interface and photo selection interface operations as described herein. In this example, the restricted photo library option selected by the user and related operations results in a virtual access policy for the photo library against the third-party application. From the third-party application's perspective, full access to the photo library is provided, where the photo library is restricted to a user-defined subset of photos via the virtual access policy.

[0049] In other scenarios, third-party applications use an update request interface and an opt-out option, enabling them to request access to photos in the end-user device's photo library without being aware of the restricted asset access option. In other words, traditional binary indicators (full access or no access) are used for communication between the end-user device and the third-party application. Therefore, the third-party application will receive a response from the end-user device that does not indicate when the restricted access option is used (full access or no access). In the scenario utilizing the updated request interface and opt-out option, the installation of an updated operating system with restricted asset access options can be considered an asset privacy management trigger, which uses the third-party application to override any previous photo library privacy management policies. In some examples, the installation of an updated operating system results in the introduction of new features, such as restricted asset access options and related operations, without a request from the relevant third-party application. In this case, the user can use the available operating system settings or the new feature introduction process to select the restricted asset access option for use with one or more third-party applications. Subsequently, the photo library privacy management settings selected by the user (including the restricted access option and related subset definitions for the third-party application) will be used in response to subsequent photo library access requests from the third-party application.

[0050] exist Figure 7The asset privacy management method 700 illustrates a scenario based on an update request interface and an exit option selection (where a conventional binary indicator is used for communication between the end-user device and the third-party application). As shown in the figure, at box 705, the asset privacy management method 700 includes receiving a request from a third-party application for access to the photo library. If photo library privacy management (decision box 710) is not triggered upon receiving the request, existing photo library privacy management options or policies are used at box 740. This might be the case, for example, if the user selects restricted access options and related policies via available operating system settings. If photo library privacy management (decision box 710) is triggered upon receiving the request, a privacy selection interface is displayed at box 715, and the user's selection is received. At box 720, if the user selects the deny access option at box 715, access to the photo library by the third-party application is denied. At box 725, if the user selects the full access option at box 715, full access to the photo library is allowed. If the user selects the restricted access option, a photo selection interface is displayed at box 730. This interface is configured to define a subset of photos in the photo library as authorized for use by third-party applications based on the user's selection. Access to the defined subset of photos is provided at box 735. From the third-party application's perspective, access to the photo library is either allowed or not allowed, and the definition of the subset of photos used to restrict access to the third-party application is unknown to the third-party application.

[0051] In other scenarios, third-party applications use updated request interfaces and opt-in options that enable them to submit photo library privacy management requests to end-user devices and receive non-binary instructions from the end-user devices indicating whether to grant full access, restricted access, or no access. In such cases, the third-party application will know when the user selects the restricted access option and has the ability to submit photo library privacy management requests so that the user can update the subset of assets associated with the restricted access option. Even if the third-party application receives an indication that the restricted access option is being used, and even if the third-party application can submit photo library privacy management requests, the privacy opt-in interface for selecting asset privacy management options and the asset opt-in interface for defining the restricted asset subset (if the restricted asset option is selected) are still used in a transparent manner to the third-party application. In one example, even if the third-party application was previously granted full access to the end-user device's photo library, access to the photo library can be restricted using the privacy opt-in interface and photo opt-in interface operations as described herein. As needed, the restricted photo library option, selected by the user and related operations, produces a virtual access policy for the photo library for the third-party application. From the perspective of third-party applications, privacy management of the photo library is based on updated non-binary options (e.g., full access, restricted access, or no access). If a user selects the restricted access option, the photos visible to third-party applications will be restricted by the virtual access interface to a subset of photos defined by the photo selection interface.

[0052] exist Figure 8The asset privacy management method 800 illustrates an updated request interface and selection scenarios. As shown, the asset privacy management method 800 includes receiving a photo library privacy management request or photo library access request from a third-party application at box 805. If photo library privacy management (decision box 810) is not triggered upon receiving the request from box 805, an existing photo library privacy management selection or policy is used at box 840. This might be the case, for example, if the user selects restricted access options and related policies via available operating system settings. At box 815, a privacy selection interface is displayed, and the user's selection is received. At box 820, if the user selects the deny access option at box 815, access to the photo library by the third-party application is denied. At box 825, in response to the user selecting the full access option at box 815, full access to the photo library is allowed. At box 830, a photo selection interface is displayed, and this photo selection interface is configured to define a subset of photos in the photo library as authorized for use by the third-party application based on the user's selection. At box 835, access to the defined subset of photos is provided. From the perspective of third-party applications, access to the photo library is permitted, restricted, or prohibited (at least an indication is provided). Additionally, third-party applications can be allowed to provide messages to users when making photo library privacy management requests. However, the definition of the subset of photos used to restrict access to third-party applications will be unknown to them. When the restricted access option is selected, a virtual access policy for the photo library is established for third-party applications. From the perspective of third-party applications, restricted access to the photo library is provided, where the details of the virtual access policy are unknown to them.

[0053] In some examples, asset privacy management with restricted access options as described herein can be provided by an access control system that determines the authorized rights granted by a third-party application to a specified privacy-sensitive resource. Authorized rights can represent an unknown state, a denied state, a permitted state, or a restricted state. An unknown state is a state in which the access control system has no existing record for the tuple {third-party application, privacy-sensitive resource}. A denied state is a state in which the user does not consent to allowing the third-party application access to the privacy-sensitive resource. A permitted state is a state in which the user has consented to allowing the third-party application access to the privacy-sensitive resource and all its assets at any given point in the future. A restricted state is a state in which the user has consented to allowing the third-party application access to a subset of the assets of the privacy-sensitive resource.

[0054] In some examples, the access control system can maintain additional information associated with the tuple {third-party application, privacy-sensitive resource}, such as version information, date and timestamps, to enforce access policies that require that data. Additionally, the access control system grants authorization rights to the asset management system for the tuple {third-party application, privacy-sensitive resource}, which the asset management system can then use under its control to enforce access policies for the asset.

[0055] These hardware and software descriptions are not intended to be limiting, and different embodiments of the invention described herein may include any type of computing device, such as a Mac, PC, PDA, telephone, server, or even an embedded system.

[0056] It should be understood that the above description is intended to be exemplary and not restrictive. The material presented is intended to enable any person skilled in the art to make and use the invention protected by the claims, and provides that material in the context of a particular embodiment, variations of which will be apparent to a person skilled in the art (e.g., multiple embodiments of the disclosed embodiments may be used in combination with each other). Furthermore, it should be understood that some of the operations identified herein may be performed in a different order. Therefore, the scope of this disclosure should be determined by reference to the appended claims and the full scope of equivalents to such claims. In the appended claims, the terms “including” and “in which” are used as common English equivalents to the corresponding terms “comprising” and “wherein”.

Claims

1. A method for managing third-party applications' access to assets on end-user devices, the method comprising: Identify asset privacy management triggers related to third-party applications on the end-user device; In response to the detection of the asset privacy management trigger, a privacy selection interface is displayed, which allows the user to select between the full asset access option, the no asset access option, and the restricted asset access option; In response to selecting the restricted asset access option, an asset selection interface is displayed, which is configured as follows: Displays a single asset in the asset pool; as well as A subset of assets on the end-user device is defined as authorized for use by the third-party application based on the user's selection of a subset of individual assets displayed in the asset library; In response to a request for access to assets of the end-user device by the third-party application, the third-party application is provided with access to only a defined subset of assets; The definition of the asset subset is added to a database storing index information, which associates each of the multiple defined asset subsets with a corresponding third-party application identifier. as well as In response to the recognition that a new asset has been added since the last update of the database, a new request for access to the asset on the end-user device by the third-party application is interpreted as an asset privacy management trigger, even if the third-party application previously received full access authorization by selecting the full asset access option.

2. The method of claim 1, wherein the asset privacy management trigger involves an operating system settings selection made by the user.

3. The method of claim 1, wherein the asset privacy management trigger involves a third-party application request to access the assets of the end-user device.

4. The method according to claim 3, wherein the operation of the privacy selection interface and the asset selection interface is transparent to the third-party application.

5. The method of claim 1, wherein the asset privacy management trigger involves a third-party application request for the user to initiate asset privacy management.

6. The method according to claim 1, further comprising: Identify requests for access to the assets of the end-user device by the third-party application; as well as The index information in the database is used to provide the third-party application with access to only a defined subset of assets.

7. A terminal user equipment, the terminal user equipment including a user interface and instructions, the instructions configuring the terminal user equipment as follows when executed: Identify asset privacy management triggers related to third-party applications; In response to the detection of the asset privacy management trigger, a privacy selection interface is displayed, which allows the user to select between the full asset access option, the no asset access option, and the restricted asset access option; In response to selecting the restricted asset access option, an asset selection interface is displayed, which is configured as follows: Displays a single asset in the asset pool; as well as A subset of assets on an end-user device is defined as authorized for use by the third-party application based on the user's selection of a subset of individual assets displayed in the asset library. In response to a subsequent request for access to assets of the end-user device by the third-party application, the third-party application is provided with access to only a defined subset of assets; The definition of the asset subset is added to a database storing index information, which associates each of the multiple defined asset subsets with a corresponding third-party application identifier. as well as In response to the recognition that a new asset has been added since the last update of the database, a new request for access to the asset on the end-user device by the third-party application is interpreted as an asset privacy management trigger, even if the third-party application previously received full access authorization by selecting the full asset access option.

8. The terminal user equipment according to claim 7, wherein the instructions configure the terminal user equipment to: The user's operating system settings selection is identified as a trigger for the asset privacy management; and A third-party application request for access to the assets of the end-user device is identified as an asset privacy management trigger.

9. The end-user device of claim 7, wherein the instructions configure the end-user device to display the privacy selection interface and the asset selection interface in a manner transparent to the third-party application.

10. The end-user device of claim 7, wherein the instructions configure the end-user device to display the privacy selection interface in response to a request from a third-party application for initiating asset privacy management by the user.

11. A system comprising: One or more processors; One or more cameras, the one or more cameras being configured to capture photographs; A memory for storing program instructions for the one or more processors, wherein the instructions, when executed, cause the one or more processors to: Maintaining a photo library based on captured photos Identifying photo library privacy management triggers related to third-party applications; In response to the detection of the photo library privacy management trigger, a privacy selection interface is displayed, which allows the user to choose between the full photo library access option, the no photo library access option, and the restricted photo library access option; In response to selecting the restricted photo library access option, a photo selection interface is displayed, which is configured as follows: Displays a single photo from the photo library; as well as A subset of photos in the photo library is defined as authorized for use by the third-party application based on the user's selection of a subset of the individual photos displayed in the photo library; In response to a subsequent request for access to the photo library by the third-party application, the third-party application is provided with access to only a defined subset of photos; The definition of the subset of the photos is added to a database storing index information, which associates each subset of photos in the multiple defined subsets of photos with a corresponding third-party application identifier; as well as In response to the recognition that new photos have been added since the last update of the database, a new request for access to photos on the end-user device by the third-party application is interpreted as a photo privacy management trigger, even if the third-party application previously received full access authorization by selecting the full photo library access option.

12. The system of claim 11, wherein the instructions, when executed, further cause the one or more processors to: The user's operating system settings selection is identified as a trigger for the photo library privacy management; and A third-party application request to access the photo library is identified as a trigger for the photo library's privacy management.

13. The system of claim 11, wherein the instructions, when executed, further cause the one or more processors to display the privacy selection interface and the photo selection interface in a manner transparent to the third-party application.

14. The system of claim 11, wherein the instructions, when executed, further cause the one or more processors to display the privacy selection interface in response to a request from a third-party application for initiating photo library privacy management by the user.

Images