Camera device capability API framework and shared OEM repository system
By providing hardware request processing methods in computing devices, using application service modules and hardware abstraction layers, the problem of difficulty in synchronizing the latest hardware features by application developers is solved, and flexible adaptation and competitive advantages between applications and hardware are achieved.
Patent Information
- Application Number
- CN202080107768.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-10
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-12-10
AI Technical Summary
As camera devices on portable computing devices become more complex, application developers have difficulty keeping synchronizing with the latest hardware features and improvements, and OEMs are reluctant to incorporate access to hardware features into the mobile operating system API, making application development more difficult , and it is difficult to provide a competitive advantage.
By providing a hardware request processing method in the computing device, using the application service module to receive hardware-related function requests, determine whether it is provided by the OEM specific SDK, if provided, commands and parameters are provided to the OEM SDK, and if not provided, the operating system hardware abstraction layer is provided to the operating system hardware. Provides to realize the download and use of customized hardware integration codes, ensuring the synchronization of applications with specific devices.
The synchronization of applications and the latest hardware features is achieved, the competitive advantage is provided, and the application development is simplified, ensuring the capability of image/video processing under various conditions.
Smart Images

Figure CN116635837B_ABST
Abstract
Description
Background Art
[0001] With the increase in the use of digital images and videos, the affordability of portable computing devices, the availability of increasing capacity of digital storage media, and the increasing bandwidth and accessibility of network connections, digital images and videos have become a part of daily life for more and more people. In addition, the expectation of device users is that the experience of using apps on portable computing devices will continue to become more sophisticated and media-rich. BRIEF DESCRIPTION OF THE DRAWINGS
[0002] In the drawings, which are not necessarily drawn to scale, similar reference numerals may describe similar components in different views. To easily identify the discussion of any particular element or action, the highest digit or digits in the reference numeral refer to the figure number in which the element is first introduced. Some embodiments are shown by way of example and not limitation in the figures of the accompanying drawings, in which:
[0003] Figure 1 is a block diagram illustrating an example system 100 for providing a camera capabilities API framework and a shared data repository.
[0004] Figure 2 Shows Figure 1 An example of the architecture of an app.
[0005] Figure 3 Shows Figure 1 An alternative example of the architecture of an app.
[0006] Figure 4 Shows Figure 1 An alternative example of the architecture of an app.
[0007] Figure 5 is a flow chart illustrating a method for implementing a camera or other hardware API framework in an app running on a mobile device.
[0008] Figure 6 is a flow chart illustrating a method of providing a shared repository for developing a hardware API framework and for providing customized versions of apps running on mobile devices.
[0009] Figure 7 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein.
[0010] Figure 8 is a block diagram illustrating a software architecture in which examples may be implemented. DETAILED DESCRIPTION
[0011] Users with a wide range of interests from a variety of locations can capture digital images or videos of a variety of subjects and make the captured images or videos available to others via a network, such as the Internet. Enabling a computing device to perform image / video processing or image / video enhancement operations on a variety of objects and / or features captured under a variety of changing conditions (e.g., changes in image scale, noise, lighting, movement, or geometric distortion) can be challenging and computationally intensive. In addition, as camera devices in mobile devices become more sophisticated and diverse, it is more difficult for application developers to keep their apps up to date with the latest features and improvements provided by the OEM's latest hardware. In addition, OEMs may not want to incorporate access to these features into the APIs of mobile operating systems (such as Android) because this may allow competitors to quickly adopt the latest features and improvements. The ability of OEMs to provide enhancements to versions of applications running on their hardware can provide a competitive advantage.
[0012] In one example, a method for processing a hardware request in a computing device including one or more processors is provided. The method includes: receiving a request for a hardware-related function from an application service module; determining whether the hardware-related function can be provided by an OEM-specific SDK; providing commands and parameters related to the hardware-related function to the OEM-specific SDK based on determining that the hardware-related function can be provided by the OEM-specific SDK; and providing commands and parameters related to the hardware-related function to an operating system hardware abstraction layer based on determining that the hardware-related function cannot be provided by the OEM-specific SDK.
[0013] The determining step may be performed by a custom library downloaded to the OEM-specific hardware device.In addition, the application service module may be included as part of a non-OEM-specific application.
[0014] The hardware-related function may be supported by default in the operating system of the computing device, and the request for the hardware-related function may be a default request, in which case the method may further include: providing commands and parameters related to the default request to the OEM-specific SDK.
[0015] In one example, a method for providing an application customized for a specific computing device including one or more processors is provided. The method includes: downloading a first version of the application by the specific computing device for execution on the specific computing device; querying the specific computing device by the application to obtain information identifying the specific computing device; and downloading, by the application executed on the specific computing device, customized hardware integration code for use with the application when executed on the specific computing device. The customized hardware integration code may be specific to a specific OEM or a device provided by the specific OEM.
[0016] Figure 11 is a block diagram illustrating an example system 100 for providing a camera (or other hardware) capabilities API framework and a shared data repository. The system 100 includes multiple instances of mobile devices 104, each of which hosts several applications including an app 116 that has been developed by an app developer. Each mobile device 104 can be communicatively coupled to other mobile devices 104 that are also running instances of the app 116 and to an application server system 114 via a network 102 (e.g., the Internet). The mobile devices 104 can also be coupled to an app store 112 via the network, from which the app 116 can be downloaded and installed on the mobile device 104. The app 116 can be any kind of app that can be run on the mobile device 104 that uses the camera capabilities of the mobile device 104. In one example, the app 116 is a messaging application or a social networking application.
[0017] The mobile device 104 includes an OEM SDK 128 that is preloaded on the mobile device 104 by the OEM and that provides an interface for the app 116 to access, for example, a camera or other hardware, operating system, firmware, or software capabilities that have been included by the OEM of the mobile device 104. The app 116 also includes an integration code 126 that provides a bridge between the OEM SDK 128 and the app 116. The integration code 126 and the OEM SDK 128 are unique to the model or line of products to which the mobile device belongs and are unique to a particular OEM.
[0018] The mobile device also includes an OS hardware abstraction layer 130. The hardware abstraction layer 130 defines a standard interface for OEMs to implement. The hardware abstraction layer 130 allows the OS of the mobile device 104 to interact with hardware devices at a general or abstract level rather than a detailed hardware level. The hardware abstraction layer 130 may, for example, include one or more modules corresponding to different hardware types (e.g., cameras, audio components, Bluetooth components, sensors, storage devices, peripherals, etc.).
[0019] In use, when the app 116 calls a hardware-related function or feature, the app 116 will provide a call to the hardware abstraction layer 130 or the OEM SDK 128 (via the integration code 126) to complete the hardware-related function or call the hardware-related feature. The OEM SDK 128 may in turn call or coordinate with the hardware abstraction layer 130, or may interact more directly with the relevant hardware to complete the hardware-related function or call the feature.
[0020] The application server system 114 supports various services and operations that are provided to the app 116 and allow the app 116 to work as expected. Such operations include sending data to the app 116, receiving data from the app 116, and processing data generated by the app 116. As an example, the data may include message content, mobile device information, geographic location information, user-generated media, media enhancements and overlays, message content persistence conditions, social network information, live event information, login and profile information, friend groups and friend status. Data exchanges with the application server system 114 and other mobile devices 104 are invoked and controlled through functions available through the user interface (UI) of the app 116.
[0021] The system 100 may also include an OEM system 106 that hosts an OEM SDK 128 and OEM integration code 118. As is known in the art, the OEM SDK 128 is developed by the OEM for use on devices provided by the OEM. The OEM system 106 may also include candidate OEM integration code 118 that is being developed by the OEM for submission to app developers to be included in the app 116 when running on the mobile device 104 provided by the OEM.
[0022] The system 100 also includes an integrated code repository 108, which is managed by the app provider and can be hosted on a code hosting platform such as GitHub, for example. The integrated code repository 108 includes an API repository 120 (provided by the app developer) containing the API, from which the OEM can create its own unique version of the OEM integrated code 118. The purpose of the API repository 120 is to share the visibility of the API with the OEM developer. The API repository 120 is maintained by the application developer, and the OEM only has permission to read but not write. Any updates to the API repository 120 are provided by the app developer. Any changes made by the app developer to the API repository 120 are published to the integrated code repository 108 and updated in the reference version of the API repository 120 maintained in the app developer code repository 110 owned and maintained by the app developer.
[0023] Each participating OEM will have a corresponding OEM implementation repository, such as OEM 1 implementation repository 122 for OEM 1, OEM 2 implementation repository 124 for OEM 2, etc. Each OEM implementation repository is jointly maintained by the app developer and the corresponding OEM, and access to each OEM implementation repository is limited to the app developer and the specific OEM. Each OEM implementation repository contains an OEM integration code library, which is the OEM's implementation of the API provided by the app developer and is maintained in the API repository 120. After reviewing and approving the OEM implementation of the OEM integration code 118, the app developer can transfer the approved integration code to the app developer code repository 110, where the approved integration code can be incorporated into the customized version of the app 116.
[0024] Also included in the system is an app store 112, such as Google Play or Apple's App Store, which hosts apps for download and use on mobile devices 104 running on Android or iOS, respectively. After the app developer integrates the approved OEM integration code 118 into the version of the app 116 (which is now customized for use with the mobile device 104 provided by the OEM), the app developer can upload the version of the app 116 to the app store 112, where the version of the app 116 is hosted and available for download and installation by the mobile device 104. Alternatively, the approved OEM integration code 118 can be provided as an executable library that can be downloaded to the mobile device 104 and called by the generic version of the app 116.
[0025] In the use of the system 100, the OEM, after registering with the integration code repository 108 (approved by the app developer), downloads the API from the API repository 120 for developing the OEM integration code 118. The OEM then prepares a customized version, i.e., the OEM integration code 118, from the API downloaded from the API repository 120. Upon completion, the OEM integration code 118 is uploaded from the OEM system 106 to the integration code repository 108 into an OEM implementation repository, such as the OEM 1 implementation repository 122 or the OEM 2 implementation repository 124, etc., where the OEM integration code 118 can be downloaded by the app developer for review. Alternatively, the OEM integration code 118 is uploaded directly from the OEM system 106 to the app developer code repository 110 for review.
[0026] After the OEM integration code 118 is approved by the app developer, the approved OEM integration code 118 is incorporated by the app developer into a customized version of the app 116 or into a customized executable library for use on the mobile device 104 provided by the OEM. The customized version of the app 116 or the customized executable library will include information (e.g., OEM name, model, series name, operating system, etc.) that identifies the mobile device 104 for which the OEM integration code 126 has been created. The customized executable library or the customized version of the app 116 is then uploaded to the app store 112 where it is made available for download to users of the mobile device 104.
[0027] When a user of the mobile device 104 requests to download the app 116, the mobile device 104 queries the app store 112 and downloads the universal version of the app 116. When the app 116 is executed by the mobile device 104, the app 116 queries the mobile device 104 to obtain information identifying the mobile device 104 (e.g., OEM name, model, operating system, etc.).
[0028] The app 116 then queries the app store 112 to determine if there is an integration code 126 for the app 116 that is customized for the specific mobile device 104. If such an integration code 126 is present, an addition to the app 116 is provided to the mobile device 104 by the app store 112 to supplement or update the generic version of the app. In one embodiment, the addition is a custom executable library that is dynamically transmitted from the app store 112 to the mobile device 104. The custom executable library will be called by the generic version of the app 116 in use, thereby providing a customized version of the app 116.
[0029] Alternatively, the app 116 queries the app store 112 to determine if there is a version of the app 116 with the integration code 126 customized for the specific mobile device 104 , which version can then replace the generic version of the app 116 .
[0030] Figure 2 Shows Figure 1 1 and the interaction of the app 116 with the hardware abstraction layer 130 and the OEM SDK 128. For clarity, only the integration code 126 and the camera service module 202 of the app 116 are shown. It will be apparent that the app 116 includes many functional and service modules.
[0031] The camera service module 202 receives a request (e.g., invoking Super Night Mode, Face Detection, etc.) to activate a camera function or feature from one of the other modules of the app 116 (e.g., a user interface module (not shown)). The request and any related parameters (e.g., zoom in or out, flash on or off) are passed from the camera service module 202 to the integration code 216 for processing.
[0032] The integration code 216, as an example of the integration code 126, includes several camera control interfaces. As shown, the integration code 216 includes a first group 204 of OS camera control interfaces 206a to 206f, which are standard for the operating system. For example, the group 204 may include camera control interfaces from Google's Android Camera1, Camera2, or CameraX libraries, which provide hardware-independent or general camera control. For example, auto focus control, setting flash, camera preview, taking pictures, recording video, optical or electronic image stabilization, face detection, zoom control, etc.
[0033] The integration code 216 also includes a group 210 of custom OEM camera control interfaces 214a to 214c. The custom OEM control interfaces have been designed by the application developer and implemented by the OEM to enable access to OEM-specific implementations of features or functions or camera functions or features specific to the hardware of the mobile device 104, and include unique or implementation-specific portions of the integration code 216. For example, the group 210 might include camera control interfaces related to High Dynamic Range capture, an enhanced night mode, a mode in which the camera is controlled by gestures, a bokeh mode, one or more post-capture (or "beautification") modes, a mode that gives autofocus priority to faces, and the like.
[0034] Upon receiving a request from the camera service module 202, a relevant one of the camera control interfaces in the group 204 or the group 210 transmits appropriate commands and parameters to the hardware abstraction layer 130 (eg, Figure 2 The OS hardware abstraction module 208 or the OS hardware abstraction module 212 in the OEM SDK 128, which in turn implements the appropriate commands at the hardware level.
[0035] In the case of the OS camera control interfaces 206a to 206f, the relevant commands and parameters are appropriately directed to and received by the OS hardware abstraction module 208 or the OS hardware abstraction module 212. For example, for flexibility, the OS hardware abstraction module 208 may be Android Camera1, and 212 may be AndroidCamera2. In the case of the standard operating system camera control interfaces in the group 204 having corresponding OEM-specific implementations in the OEM SDK 128, the relevant commands and parameters are directed to and received by the OEM SDK 128 instead of one of the hardware abstraction modules.
[0036] In the case of a custom OEM camera control interface 214a to 214c, the relevant commands and parameters are directed by the integration code 216 to and received by the OEM SDK 128, enabling the app 116 to access custom camera functionality that is unique or specific to the OEM. In this way, app developers can benefit from custom OEM functionality or features that are specific to the mobile device 104 and that may be different from or improved upon the OS standard or generic camera controls. Similarly, the OEM can benefit by providing users of the app 116 with custom OEM functionality or features that are specific to the OEM's mobile device 104 and that are different from competitors' products, thereby enhancing the attractiveness of the mobile device 104.
[0037] Figure 3 Shows Figure 1 1 and the interaction of app 116 with the OS hardware abstraction module 212 and OEM SDK 128. In this case, app 116 includes a service manager 302 and integration code 304. When the camera service module 202 receives a request from one of the other modules of app 116, the request and any related parameters (e.g., zoom in / out, flash on / off) are passed from the camera service module 202 to the service manager 302. The service manager 302 is common to the OS. For example, the service manager 302 can be the Android Camera2 manager, which is a system service manager for detecting, characterizing, and connecting to the Android Camera2 API. The service manager 302 in turn passes the request from the camera service module 202 to the integration code 304, rather than to the OS hardware abstraction layer 130 as would normally occur.
[0038] Integration code 304 is a simplified version of integration code 216 that may enable partial override of any default camera behavior of camera functions or features specific to the mobile device's 104 hardware or the OEM's implementation.
[0039] That is, when a default request without an OEM-specific implementation is received from the service manager 302, the relevant commands and parameters are directed by the integration code 304 to the OS hardware abstraction module 212 and received by the OS hardware abstraction module 212 for default processing by the OS hardware abstraction module 212. However, when a default request that does have an OEM-specific implementation is received from the service manager 302, the relevant commands and parameters are directed by the integration code 304 to the OEM SDK 128 and received by the OEM SDK 128, so that the app 116 can access the customized camera device functions that are unique or specific to the OEM, rather than the default camera device functions envisioned in the OS hardware abstraction module 212.
[0040] like Figure 3 As shown by the arrow between the OEM SDK 128 and the OS hardware abstract module 212 in FIG. 1 , the OEM SDK 128 can also call the OS hardware abstract module 212 to perform camera operations, such as opening the camera preview, closing the camera, taking pictures, etc.
[0041] Figure 3 An advantage of the architecture is that integration code 304 may be more concise than integration code 216. Additionally, OEMs may be more familiar with service manager 302 and OS hardware abstraction module 212, potentially providing an easier implementation. Figure 3 A disadvantage of the architecture is that it is limited to existing service manager 302 commands and functions.
[0042] Figure 4 Shows Figure 1 Another alternative example of the architecture of the app 116 and the interaction of the app 116 with the OS hardware abstraction module 212 and the OEM SDK 128. Figure 2 and Figure 3 A hybrid implementation of the architecture.
[0043] like Figure 4 As shown, in this example, app 116 includes both integration code 402 and integration code 404, which generally corresponds to Figure 2 Integration code 216, integration code 404 generally corresponds to Figure 3 In this case, the integrated code 404 is used as a submodule of the integrated code 402.
[0044] Figure 4The architecture may provide advantages in terms of flexibility and extensibility. For example, as described above with respect to the integration code 304, the integration code 402 that provides a partial override of any default camera behavior for camera functions or features that are specific to the OEM's implementation or the hardware of the mobile device 104 may be reusable with little or no modification between different mobile devices 104 with different capabilities provided by a particular OEM. In addition, the ability to override an existing camera control interface (e.g., the OS camera control interface 206d in the group 204) with a more concise integration code 404 / integration code 304 may provide advantages in terms of flexibility and extensibility. Figure 2 In this case, the OEM does not need to modify the integration code 402 provided by the app developer in order to provide a certain degree of customization.
[0045] However, Figure 4 The architecture also provides scalability advantages. An OEM may develop a customized version of the integration code 402 that incorporates one or more customized OEM camera control interfaces, such as Figure 4 4. As described above, such a custom OEM control interface has been generated by the OEM to enable access to camera functions or features that are specific to the OEM's implementation or the hardware of the mobile device 104 and may not be discovered as default camera behaviors in the service manager 302 that may be overridden by the integration code 404.
[0046] For example, when a default request is received from the camera service module 202, the relevant commands and parameters are directed by the integration code 402 to and received by the integration code 404. If there is no OEM-specific implementation of the default command, the integration code 404 passes the command and parameters to the OS hardware abstraction module 212 for processing using default functions and behaviors.
[0047] However, when a default request that does have an OEM-specific implementation is received from the camera service module 202, the relevant commands and parameters are directed to and received by the OEM SDK 128 by the integration code 404, enabling the app 116 to access customized camera functionality that is unique or specific to the OEM, rather than the default camera functionality envisioned in the OS hardware abstraction module 212. For example, if the OS hardware abstraction module 212 supports a night mode capture functionality, but the OEM SDK 128 has an OEM-specific implementation of night mode capture, the integration code 404 will direct the relevant commands and parameters to the OEM SDK 128 rather than the OS hardware abstraction module 212.
[0048] like Figure 4 As shown by the arrow between the OEM SDK 128 and the OS hardware abstract module 212 in FIG. 1 , the OEM SDK 128 can also call the OS hardware abstract module 212 to perform camera operations, such as opening the camera preview, closing the camera, taking pictures, etc.
[0049] On the other hand, if the camera service module 202 provides the integration code 402 with a request that is not supported by the OS hardware abstraction module 212, the integration code 402 will provide the relevant command and parameters to the integration code 404, which in turn transmits the relevant command and parameters to the OEM SDK 128. For example, if the OEM SDK 128 has a gesture control function that is not envisioned by the default command (e.g., requested via the OEM camera control interface 214b), the integration code 402 will transmit the relevant command and parameters to the OEM SDK 128 via the integration code 404, such as Figure 4 shown.
[0050] Figure 5 1 is a flow chart illustrating a method of implementing a camera or other hardware API framework in an app 116 running on a mobile device 104. The method begins at operation 502, where the app receives user input requesting that a hardware-related function be performed by the app 116 and the mobile device 104 in operation 602. The user input may be, for example, a selection of a hardware-related function via a user interface of the app displayed on a display of the mobile device, or an actuation of one or more buttons of the mobile device 104, which will be received by a UI service component of the app 116. The user input may be, for example, a selection of a camera function as discussed above.
[0051] The commands and parameters associated with the particular user input are provided by the UI service component to the relevant hardware service module contained in the app 116. In the case of selecting a camera function such as "enable night mode" or "record video", the appropriate commands and parameters are provided to the app hardware service module, such as the camera service module 202, as shown in operation 504.
[0052] When the app hardware service module, such as the camera service module 202, receives the appropriate commands and parameters, at operation 506, the app hardware service module provides the relevant commands and parameters to a hardware integration code module, such as the integration code 126, which is specific to the model or series of products to which the mobile device belongs and is unique to a specific OEM.
[0053] Upon receiving the appropriate commands and parameters by the hardware integration code module, at operation 508, the hardware integration code module determines at 508 whether the hardware-related functions can be processed by an SDK provided by the OEM that provides the mobile device 104. In the case where the hardware-related functions can be processed by the SDK provided by the OEM (e.g., OEM SDK 128), the relevant commands and parameters are provided to the SDK at operation 510. In the case where the hardware-related functions cannot be processed by the SDK provided by the OEM (e.g., OEM SDK 128), the relevant commands and parameters are provided to the operating system hardware abstraction layer, such as hardware abstraction layer 130, at operation 514.
[0054] Upon receiving the relevant command and parameters by the OEM SDK at operation 510 , at operation 512 , the OEM SDK performs the requested hardware-related function using the OEM-specific customized implementation of the hardware-related function implemented in the OEM SDK.
[0055] Upon receiving the relevant command and parameters by the hardware abstraction layer at operation 516 , the hardware abstraction layer performs the requested hardware-dependent function at operation 512 using a conventional or standard implementation of the hardware-dependent function implemented in the hardware abstraction layer.
[0056] Figure 6 is a flow chart illustrating a method of providing a shared repository for developing a hardware API framework in the shared repository and for providing a customized version of an app running on a mobile device.
[0057] The method begins at operation 602, where an app developer creates and defines a shared code repository, such as the integration code repository 108. Access to the shared code repository is subject to login requirements and security settings of authorized OEMs and app developers.
[0058] Then, at operation 604, the app developer creates OEM-specific implementation code repositories within the shared code repository, such as OEM 1 implementation repository 122, OEM 2 implementation repository 124, etc. within the integrated code repository 108. Read / write access to a particular OEM-specific implementation code repository is restricted by the login requirements and security settings of the app developer and the particular OEM.
[0059] Also in operation 604, the app developer creates a shared API library within a shared code repository, such as API repository 120 within integration code repository 108 at operation 604. The OEM is limited by security settings and login requirements to read-only access to API repository 120, while the app developer has read / write access.
[0060] Then, at operation 606, the app developer uploads the hardware capability APIs used in the universal version of the app to a shared API repository for read-only access by all authorized OEMs.
[0061] Then, at operation 608 , each OEM downloads and customizes the API to generate customized OEM integration code 118 , which is then uploaded to an implementation code repository corresponding to the specific OEM or to an app developer code repository, such as app developer code repository 110 .
[0062] The custom OEM integration code is then reviewed by the app developer and, if appropriate, approved at operation 610. The approved OEM integration code is then uploaded to the app store 112 at operation 612. The OEM integration code includes various identifiers, such as identification of the app 116, the mobile device 104, the operating system, etc., that allow the app 116 to identify the relevant version of the OEM integration code to the app store 112. It will be appreciated that a single app 116 may run on different mobile devices provided by several different OEMs, and therefore, there may be several different approved OEM integration code versions that may be used with the app 116.
[0063] In response to the user request, the mobile device 104 then downloads and installs the app 116 from the app store 112 at operation 614. The app 116 then queries the mobile device 104 to obtain identification information at operation 616. The app 116 then queries the app store 112 at operation 618 using the identification information obtained at operation 616 to determine whether a custom integration code is available for the particular combination of the app 116 and the mobile device 104. If a custom integration code is available for the particular combination of the app 116 and the mobile device 104, the app 116 downloads and incorporates the custom integration code 126 at operation 620, which is then used by the app 116 described above, as described with reference to FIG. Figures 1 to 5 Just as described.
[0064] Machine Architecture
[0065] Figure 7700 within which instructions 710 (e.g., software, programs, applications, applet, app, or other executable code) may be executed for causing the machine 700 to perform any one or more of the methods discussed herein. For example, the instructions 710 may cause the machine 700 to perform any one or more of the methods described herein. The instructions 710 transform a general-purpose, non-programmed machine 700 into a specific machine 700 that is programmed to perform the functions described and illustrated in the manner described. The machine 700 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 700 may operate in the capacity of a server machine or a client machine in a server-client network environment or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 700 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web device, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing instructions 710 specifying actions to be taken by the machine 700. In addition, although only a single machine 700 is shown, the term "machine" should also be considered to include a collection of machines that execute instructions 710 individually or jointly to perform any one or more of the methods discussed herein. For example, the machine 700 may include any of the mobile device 104 or a number of server devices that form part of the SDK server system. In some examples, the machine 700 may also include both a client and a server system, wherein certain operations of a particular method or algorithm are performed on the server side and certain operations of a particular method or algorithm are performed on the client side.
[0066] The machine 700 may include a processor 704, a memory 706, and an input / output I / O component 638 that may be configured to communicate with each other via a bus 740. In an example, the processor 704 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 708 and a processor 712 that execute instructions 710. The term "processor" is intended to include a multi-core processor that may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although Figure 7Multiple processors 704 are shown, but machine 700 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.
[0067] The memory 706 includes a main memory 714, a static memory 716, and a storage unit 718 that are accessible to the processor 704 via the bus 740. The main memory 706, the static memory 716, and the storage unit 718 store instructions 710 that implement any one or more of the methods or functions described herein. The instructions 710 may also reside, completely or partially, within the main memory 714, within the static memory 716, within the machine-readable medium 720 within the storage unit 718, within at least one of the processors 704 (e.g., within a cache memory of a processor), or any suitable combination thereof during execution thereof by the machine 700.
[0068] The I / O components 702 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O components 702 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine will be less likely to include such a touch input device. It will be appreciated that the I / O components 702 may include Figure 7 , and the like. In various examples, the I / O components 702 may include a user output component 726 and a user input component 728. The user output component 726 may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The user input component 728 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a point-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or other pointing instrument), a tactile input component (e.g., a physical button, a touch screen that provides the location and force of a touch or touch gesture, or other tactile input component), an audio input component (e.g., a microphone), etc.
[0069] In another example, the I / O component 702 may include a biometric component 730, a motion component 732, an environmental component 734, or a position component 736, as well as various other components. For example, the biometric component 730 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 732 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope).
[0070] Environmental components 734 include, for example, one or more cameras (with still image / photo and video capabilities), an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects concentrations of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.
[0071] With respect to cameras, the mobile device 104 can have a camera system including, for example, a front camera on the front surface of the mobile device 104 and a rear camera on the rear surface of the mobile device 104. The front camera can, for example, be used to capture still images and videos of a user of the mobile device 104 (e.g., a "selfie"), which can then be enhanced with the enhancement data (e.g., filters) described above. For example, the rear camera can be used to capture still images and videos in a more conventional camera mode, which are similarly enhanced with the enhancement data. In addition to the front camera and the rear camera, the mobile device 104 can also include a 360° camera for capturing 360° photos and videos.
[0072] Additionally, the camera system of the mobile device 104 may include dual rear cameras (e.g., a main camera and a depth sensing camera), or even triple, quad, or quintuple rear camera configurations on the front and back sides of the mobile device 104. For example, these multiple camera systems may include a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, a macro camera, and a depth sensor.
[0073] The position component 736 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or a barometer that detects air pressure, from which the altitude can be obtained), an orientation sensor component (e.g., a magnetometer), and the like.
[0074] Various technologies may be used to achieve communication. The I / O component 702 also includes a communication component 738 that is operable to couple the machine 700 to the network 722 or device 724 via a corresponding coupling or connection. For example, the communication component 738 may include a network interface component or another suitable device to interface with the network 722. In other examples, the communication component 738 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, Components (e.g. Low energy consumption), Components, and other communication components for providing communication via other modalities. Device 724 can be any peripheral device among other machines or various peripheral devices (e.g., a peripheral device coupled via USB).
[0075] In addition, the communication component 738 can detect the identifier or include components operable to detect the identifier. For example, the communication component 738 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional bar codes such as universal product code (UPC) bar codes, multi-dimensional bar codes such as quick response (QR) codes, Aztec codes, data matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar codes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be derived via the communication component 738, such as a location via Internet Protocol (IP) geolocation, a location via Internet Protocol (IP), ... The location of signal triangulation, the location of an NFC beacon signal via detection that can indicate a specific location, etc.
[0076] Various memories (e.g., main memory 714, static memory 716, and memory of processor 704) and storage unit 718 may store one or more sets of instructions and data structures (e.g., software) used by or implementing any one or more of the methods or functions described herein. These instructions (e.g., instructions 710) when executed by processor 704 cause various operations to implement the disclosed examples.
[0077] The instructions 710 may be sent or received via a network interface device (e.g., a network interface component included in the communication component 738) using a transmission medium and using any of several well-known transmission protocols (e.g., the Hypertext Transfer Protocol (HTTP)) over the network 722. Similarly, the instructions 710 may be sent or received via a coupling (e.g., a peer-to-peer coupling) to the device 724 using a transmission medium.
[0078] Software Architecture
[0079] Figure 8 800 is a block diagram illustrating a software architecture 804 that can be installed on any one or more of the devices described herein. The software architecture 804 is supported by hardware, such as a machine 802 including a processor 820, a memory 826, and an I / O component 838. In this example, the software architecture 804 can be conceptualized as a stack of layers in which each layer provides specific functionality. The software architecture 804 includes layers such as an operating system 812, a library 810, a framework 808, and an application 806. In operation, the application 806 invokes an API call 850 through the software stack and receives a message 852 in response to the API call 850.
[0080] The operating system 812 manages hardware resources and provides public services. The operating system 812 includes, for example, a kernel 814, services 816, and drivers 822. The kernel 814 acts as an abstraction layer between the hardware layer and other software layers. For example, the kernel 814 provides memory management, processor management (e.g., scheduling), component management, networking and security settings, and other functions. Services 816 can provide other public services to other software layers. Drivers 822 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 822 may include display drivers, camera drivers, or Low-power drivers, Flash drivers, Serial communication drivers (e.g., USB drivers), Drivers, audio drivers, power management drivers, etc.
[0081] The library 810 provides a common low-level infrastructure used by the application 806. The library 810 may include a system library 818 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 810 may include an API library 824, such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for rendering in two dimensions (2D) and three dimensions (3D) in graphics content on a display), a database library (e.g., SQLite providing various relational database functions), a web library (e.g., WebKit providing web browsing functions), etc. The library 810 may also include various other libraries 828 to provide many other APIs to the application 806.
[0082] The framework 808 provides a common high-level infrastructure used by the applications 806. For example, the framework 808 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. The framework 808 can provide a wide range of other APIs that can be used by the applications 806, some of which may be specific to a particular operating system or platform.
[0083] In an example, applications 806 may include a home application 836, a contacts application 830, a browser application 832, a book reader application 834, a location application 842, a media application 844, a messaging application 846, a game application 848, and a variety of other applications such as third-party applications 840. Applications 806 are programs that execute functions defined in the program. Various programming languages can be used to create one or more applications 806 constructed in various ways, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C language or assembly language). In a specific example, third-party applications 840 (e.g., those used by entities other than the vendor of a particular platform using ANDROID TM or IOS TM Software Development Kit (SDK) can be used to develop applications on platforms such as IOS TM ANDROID TM , Mobile software running on the mobile operating system of the phone or other mobile operating system. In this example, the third-party application 840 can activate the API call 850 provided by the operating system 812 to facilitate the functions described in this article.
[0084] Glossary
[0085] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media to facilitate communication of such instructions. Instructions may be sent or received over a network via a network interface device using a transmission medium.
[0086] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communications device that a user may use to access a network.
[0087] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, other type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate GSM evolution (EDGE) technology, including the third generation partnership project (3GPP) of 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standard, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.
[0088] "Component" refers to a device, physical entity or logic with the following boundaries, which is defined by function or subroutine calls, branch points, APIs or other technologies that provide partitioning or modularization for specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be packaged functional hardware units designed for use with other components, and can be part of a program that generally performs specific functions in related functions. Components can constitute software components (e.g., codes implemented on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., independent computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., applications or application parts) to operate to perform certain operations described herein. Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic that is permanently configured to perform certain operations. The hardware component can be a dedicated processor such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component can also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component can include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It will be recognized that it can be determined to implement the hardware component mechanically, in a dedicated and permanently configured circuit system, or in a temporarily configured (e.g., configured by software) circuit system for cost and time considerations. Therefore, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an embodiment in which the hardware component is temporarily configured (e.g., programmed), it is not necessary to configure or instantiate each of the hardware components at any time. For example, in the case where a hardware component includes a general-purpose processor that is configured by software to become a special-purpose processor, the general-purpose processor can be configured as different special-purpose processors (e.g., including different hardware components) at different times. The software configures one or more specific processors accordingly to constitute a specific hardware component, for example, at one time, and to constitute different hardware components at different times. Hardware components can provide information to other hardware components and receive information from other hardware components. Therefore, the described hardware components can be considered to be communicatively coupled.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission between or among two or more hardware components (e.g., by appropriate circuits and buses). In an embodiment in which multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessed by multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device to which it is communicatively coupled. Then, other hardware components can access the memory device at a subsequent time to retrieve and process the stored output. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., a collection of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Whether temporarily configured or permanently configured, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some operations in the operation of the method can be performed by one or more processors 820 or the parts implemented by the processor. In addition, one or more processors can also be operated to support the execution of related operations in a "cloud computing" environment or operate as "software as a service" (SaaS). For example, at least some operations in the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (for example, the Internet) and via one or more appropriate interfaces (for example, API). The execution of certain operations can be distributed between processors, not only resident in a single machine, but deployed across multiple machines. In some example embodiments, a processor or a part implemented by a processor can be located in a single geographical location (for example, in a home environment, an office environment, or a server cluster). In other example embodiments, a processor or a part implemented by a processor can be distributed across multiple geographical locations.
[0089] "Computer-readable storage media" refers to both machine storage media and transmission media. Therefore, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium" and "device-readable medium" mean the same thing and can be used interchangeably in this disclosure.
[0090] “Machine storage media” refers to a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage media,” “device storage media,” and “computer storage media” mean the same thing and are used interchangeably in this disclosure. The terms “machine storage media,” “computer storage media,” and “device storage media” expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are encompassed by the term “signal media.”
[0091] “Non-transitory computer-readable storage medium” refers to tangible media capable of storing, encoding, or carrying instructions to be executed by a machine.
[0092] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be deemed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.
Claims
1. A method of processing a hardware request in a computing device including one or more processors, comprising: receiving a request for a hardware-related function from an application service module; Determine whether the hardware-dependent functionality can be provided by an OEM-specific SDK; Based on determining that the hardware-related function can be provided by the OEM-specific SDK, providing commands and parameters related to the hardware-related function to the OEM-specific SDK; as well as Based on determining that the hardware-related function cannot be provided by the OEM-specific SDK, providing commands and parameters related to the hardware-related function to the operating system hardware abstraction layer, The determining whether the hardware-related functions can be provided by the OEM-specific SDK is performed by an integrated code, wherein the integrated code includes a group of operating system hardware control interfaces that are standard for the operating system and a group of customized OEM hardware control interfaces corresponding to the OEM-specific SDK.
2. The method according to claim 1, wherein: The determining is performed by the integrated code downloaded to the computing device as a custom library.
3. The method according to claim 1, wherein: The hardware-related function is supported by default in the operating system of the computing device, and the request for the hardware-related function is a default request, and the method further includes: Commands and parameters associated with the default request are provided to the OEM-specific SDK.
4. The method according to claim 2, wherein: The application service module is included in a portion of the non-OEM specific application.
5. The method according to claim 1, wherein: After receiving the command and parameters related to the hardware-related functions, the OEM-specific SDK calls the hardware abstraction layer.
6. The method according to claim 1, wherein: After receiving the commands and parameters related to the hardware-related functions, the OEM-specific SDK interacts more directly with the corresponding hardware to complete the hardware-related functions.
7. The method according to claim 1, wherein: The request for the hardware-dependent function is received indirectly from the application service module via a service manager that is common to an operating system running on the computing device.
8. A computer system comprising: one or more processors; as well as a memory storing instructions that, when executed by the one or more processors, configure the computer system to perform operations for processing a hardware request, the operations comprising: receiving a request for a hardware-related function from an application service module; Determine whether the hardware-dependent functionality can be provided by an OEM-specific SDK; Based on determining that the hardware-related function can be provided by the OEM-specific SDK, providing commands and parameters related to the hardware-related function to the OEM-specific SDK; and Based on determining that the hardware-related function cannot be provided by the OEM-specific SDK, providing commands and parameters related to the hardware-related function to the operating system hardware abstraction layer, The determining whether the hardware-related functions can be provided by the OEM-specific SDK is performed by an integrated code, wherein the integrated code includes a group of operating system hardware control interfaces that are standard for the operating system and a group of customized OEM hardware control interfaces corresponding to the OEM-specific SDK.
9. The computer system according to claim 8, wherein: The determination is performed by the integrated code downloaded to the computer system as a custom library.
10. The computer system according to claim 8, wherein: The hardware-related function is supported by default in the operating system of the computer system, and the request for the hardware-related function is a default request, and the operation further includes: Commands and parameters associated with the default request are provided to the OEM-specific SDK.
11. The computer system according to claim 9, wherein: The application service module is included in a portion of the non-OEM specific application.
12. The computer system according to claim 8, wherein: After receiving the command and parameters related to the hardware-related functions, the OEM-specific SDK calls the hardware abstraction layer.
13. The computer system according to claim 8, wherein: After receiving the commands and parameters related to the hardware-related functions, the OEM-specific SDK interacts more directly with the corresponding hardware to complete the hardware-related functions.
14. The computer system according to claim 8, wherein: The request for the hardware-dependent function is received indirectly from the application service module via a service manager that is common to an operating system running on the computer system.
