Function execution method, function execution device, electronic device, and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2024-11-27
- Publication Date
- 2026-05-29
Smart Images
Figure CN122097950A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of edge identification technology, and in particular to a function execution method, function execution device, electronic device and storage medium. Background Technology
[0002] As users' demands for electronic devices gradually increase and the functions of electronic devices become more abundant and complex, operating some functions of electronic devices has become too cumbersome. In this context, the ability of electronic devices to intelligently identify the current interface scene and provide auxiliary operation becomes increasingly important, which can greatly reduce the frequency of user operation and improve the user experience.
[0003] In related technologies, electronic devices often need to collect pixel color values at the system level to intelligently identify the current scene. However, when related technologies perform color sampling through the graphics display compositing service (SurfaceFlinger), they can only periodically collect the comprehensive color value of a certain area, which makes it impossible for electronic devices to accurately identify the current scene based on the comprehensive color value corresponding to the area. Summary of the Invention
[0004] To overcome the problems existing in the related technologies, this disclosure provides a function execution method, a function execution device, an electronic device, and a storage medium.
[0005] According to a first aspect of the present disclosure, a function execution method is provided, comprising: in response to an electronic device activating a preset function, determining M pixels corresponding to the preset function, the preset function including a function executed by a system service layer based on color values of the M pixels collected by the system's underlying layer within a set threshold range, where M is a positive integer; registering the pixel coordinates corresponding to the M pixels and a listener corresponding to the M pixels to the system's underlying layer, the listener being used to monitor changes in the color values of the M pixels within the set threshold range; collecting the color values of the M pixels within the set threshold range based on the listener and the M pixel coordinates; and executing the preset function at the system service layer in response to the color values of the M pixels within the set threshold range satisfying a preset color value condition.
[0006] In some possible implementations, the preset function corresponds to a time period for collecting pixel color values; after registering the M pixel coordinates and the listeners corresponding to the M pixels to the system layer, the method further includes: setting the time period corresponding to the preset function as a sampling period for collecting the color values of the M pixels within a set threshold range; the step of collecting the color values of the M pixels within the set threshold range according to the listeners and the M pixel coordinates includes: periodically collecting the color values of the M pixels within the set threshold range according to the sampling period.
[0007] In some possible implementations, the electronic device refreshes and synthesizes a new image frame based on a preset refresh rate, and the acquisition of the color values of the M pixels within a set threshold range includes: in response to the synthesis of a new image frame, acquiring the color values of the M pixels within the set threshold range.
[0008] In some possible implementations, the preset function includes a function performed at the system service layer based on the color values of the M pixels within a set threshold range when displaying the display interface corresponding to the preset application; the step of collecting the color values of the M pixels within the set threshold range includes: in response to a partial display interface corresponding to the preset application being covered by a first type of layer, and the first type of layer and the M pixels not overlapping within the set threshold range, continuing to collect the color values of the M pixels within the set threshold range, wherein the first type of layer is not configured with a system sampling function.
[0009] In some possible implementations, the method further includes: in response to a portion of the display interface corresponding to the preset application being covered by a first type of layer, and the first type of layer and the M pixels overlapping in at least a portion of a set threshold range, stopping the acquisition of color values of the M pixels within the set threshold range.
[0010] In some possible implementations, the method further includes: in response to a partial display interface corresponding to the preset application being covered by a second type of layer, stopping the collection of color values of the M pixels within a set threshold range, wherein the second type of layer is provided with a system sampling function.
[0011] In some possible implementations, the method further includes: in response to disabling the preset function, destroying the registered M pixels and the listener at the system layer, and setting the sampling period to the default sampling period.
[0012] In some possible implementations, the preset function corresponds to a preset touch area, and executing the preset function at the system service layer includes: simulating a touch operation at the system service layer for the preset touch area, wherein the preset touch area includes or does not include the display area where the M pixels are located.
[0013] According to a second aspect of the present disclosure, a function execution apparatus is provided, comprising: a determining unit, configured to determine M pixels corresponding to the preset function in response to an electronic device activating a preset function, the preset function including a function executed at a system service layer based on color values of the M pixels collected at a system layer within a set threshold range, where M is a positive integer; a registration unit, configured to register pixel coordinates corresponding to the M pixels and listeners corresponding to the M pixels to the system layer, the listeners being configured to monitor changes in color values of the M pixels within the set threshold range; a collection unit, configured to collect color values of the M pixels within the set threshold range based on the listeners and the coordinates of the M pixels; and an execution unit, configured to execute the preset function at a system service layer in response to the color values of the M pixels within the set threshold range satisfying a preset color value condition.
[0014] In some possible implementations, the preset function corresponds to a time period for collecting pixel color values, and the device further includes a sampling period setting unit; after registering the M pixel coordinates and the listeners corresponding to the M pixels to the system layer, the sampling period setting unit is used to: set the time period corresponding to the preset function as the sampling period for collecting the color values of the M pixels within the set threshold range; the collection unit collects the color values of the M pixels within the set threshold range according to the listeners and the M pixel coordinates in the following manner: periodically collects the color values of the M pixels within the set threshold range according to the sampling period.
[0015] In some possible implementations, the electronic device refreshes and synthesizes a new image frame based on a preset refresh rate, and the acquisition unit acquires the color values of the M pixels within a set threshold range in the following manner: in response to the synthesis of a new image frame, the color values of the M pixels within the set threshold range are acquired.
[0016] In some possible implementations, the preset function includes a function performed at the system service layer based on the color values of the M pixels within a set threshold range when displaying the display interface corresponding to the preset application; the acquisition unit acquires the color values of the M pixels within the set threshold range in the following manner: in response to the partial display interface corresponding to the preset application being covered by a first type of layer, and the first type of layer and the M pixels not overlapping within the set threshold range, the color values of the M pixels within the set threshold range are continued to be acquired, wherein the first type of layer is not configured with a system sampling function.
[0017] In some possible implementations, the acquisition unit is further configured to: stop acquiring the color values of the M pixels within the set threshold range in response to a partial display interface corresponding to the preset application being covered by a first type of layer, and the first type of layer and the M pixels overlapping in at least a portion of a set threshold range.
[0018] In some possible implementations, the acquisition unit is further configured to: stop acquiring the color values of the M pixels within a set threshold range in response to the partial display interface corresponding to the preset application being covered by a second type of layer, wherein the second type of layer is configured with a system sampling function.
[0019] In some possible implementations, the sampling period setting unit is further configured to: in response to disabling the preset function, destroy the registered M pixels and the listener at the system layer, and set the sampling period to the default sampling period.
[0020] In some possible implementations, the preset function corresponds to a preset touch area, and the execution unit executes the preset function in the system service layer in the following manner: simulating a touch operation for the preset touch area in the system service layer, wherein the preset touch area includes or does not include the display area where the M pixels are located.
[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: perform the function execution method described in the first aspect or any of some possible embodiments of the first aspect.
[0022] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the processor to perform the function execution method described in the first aspect or any of the possible embodiments of the first aspect.
[0023] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: After a preset function is enabled on an electronic device, M pixels corresponding to the preset function are determined. The preset function includes a function executed by the system service layer based on the color values of the M pixels within a set threshold range collected by the system's underlying layer. The pixel coordinates and listeners corresponding to the M pixels are registered to the system's underlying layer. The listeners are used to monitor changes in the color values of the M pixels within the set threshold range. Based on the listeners and the coordinates of the M pixels, the color values of the M pixels within the set threshold range are collected. When the color values of the M pixels within the set threshold range meet the preset color value conditions, the preset function is executed at the system service layer. Through this disclosure, a color sampling function for single or multiple pixels within a set threshold range is provided, achieving fine-grained edge recognition and ensuring the stability of preset function triggering.
[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0025] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0026] Figure 1 This is a schematic diagram illustrating a pixel color sampling method in a related art according to an exemplary embodiment of the present disclosure.
[0027] Figure 2 This is a block diagram illustrating a pixel color sampling process in a related art according to an exemplary embodiment of the present disclosure.
[0028] Figure 3 This is a flowchart illustrating a function execution method according to an exemplary embodiment.
[0029] Figure 4 This is a schematic diagram illustrating a pixel color sampling method according to an exemplary embodiment.
[0030] Figure 5 This is a flowchart illustrating a method for setting a sampling period and acquiring color values of M pixels within a set threshold range according to an exemplary embodiment.
[0031] Figure 6 This is a flowchart illustrating a method for acquiring color values of M pixels within a set threshold range based on the refresh rate of an electronic device, according to an exemplary embodiment.
[0032] Figure 7This is a flowchart illustrating a method for collecting color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by a first type of layer, according to an exemplary embodiment.
[0033] Figure 8 This is a flowchart illustrating a method for collecting color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by a first type of layer, according to yet another exemplary embodiment.
[0034] Figure 9 This is a flowchart illustrating a method for collecting color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by a second type of layer, according to an exemplary embodiment.
[0035] Figure 10 This is a flowchart illustrating a method for collecting color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by other layers, according to an exemplary embodiment.
[0036] Figure 11 This is a flowchart illustrating a method for setting a sampling period according to an exemplary embodiment.
[0037] Figure 12 This is a flowchart illustrating a method for performing a preset function by simulating touch operation according to an exemplary embodiment.
[0038] Figure 13 This is a schematic diagram illustrating an application scenario of a function execution method according to an exemplary embodiment of the present disclosure.
[0039] Figure 14 This is a schematic diagram illustrating an application scenario of a function execution method according to yet another exemplary embodiment of this disclosure.
[0040] Figure 15 This is a block diagram illustrating a pixel color sampling process according to an exemplary embodiment of the present disclosure.
[0041] Figure 16 This is a schematic diagram illustrating a code flow for sampling pixel color according to an exemplary embodiment of the present disclosure.
[0042] Figure 17 This is a block diagram illustrating a function execution device according to an exemplary embodiment.
[0043] Figure 18 This is a block diagram illustrating an apparatus for performing a function according to an exemplary embodiment.
[0044] Figure 19This is a block diagram illustrating an apparatus for performing a function according to yet another exemplary embodiment. Detailed Implementation
[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.
[0046] The function execution method provided in this disclosure is applied to scenarios where a corresponding preset function is triggered and executed at the application layer based on the color values of independent pixels sampled from the system's underlying layer.
[0047] The functional execution method proposed in this disclosure belongs to the field of edge recognition technology, and provides a pixel color sampling scheme to provide underlying support for various edge recognition technologies.
[0048] As users' demands for electronic devices increase and their functions become more abundant and complex, operating some functions has become overly cumbersome. For example, in gaming scenarios, interfaces with too many controls or requiring users to frequently deviate from normal operations to trigger additional functions often negatively impact the gaming experience. In such situations, the ability of electronic devices to intelligently recognize the current interface context and provide assisted operation becomes increasingly important, significantly reducing user interaction frequency and improving the user experience. For instance, in gaming scenarios, automatic triggering functions can be set for certain controls. Once the game context is identified as meeting the automatic triggering conditions for a particular control, the corresponding control is automatically triggered to assist user operation, thereby enhancing the gaming experience.
[0049] In related technologies, electronic devices often need to collect pixel color values at the system level to intelligently identify the current scene. However, when related technologies perform color sampling through the graphics display compositing service (SurfaceFlinger), they can only periodically collect the comprehensive color value of a certain area. However, the comprehensive color value of the entire area cannot accurately reflect the display content of the corresponding area, causing the electronic device to be unable to accurately identify the current scene based on the comprehensive color value corresponding to the area. Therefore, the pixel color sampling technology in related technologies cannot support the implementation of end-to-end recognition functions.
[0050] In an exemplary embodiment of this disclosure, Figure 1 This is a schematic diagram illustrating a pixel color sampling method in a related art according to an exemplary embodiment of the present disclosure, such as... Figure 1As shown, the related technologies use the following method for pixel color sampling: a sampling area containing multiple pixels is selected in the display interface, and the brightness value (luma value) of the sampling area is obtained by calculating all pixels in the sampling area. The luma value is a value obtained by dividing the total brightness value of the pixels in the sampling area by the number of pixels. It can only reflect the average parameter value of multiple pixels in a sampling area, and cannot reflect the parameter value of a single independent pixel. Therefore, the luma value obtained by the color sampling technology in the related technologies cannot accurately identify the current scene, and the luma value collected in the related technologies cannot support the implementation of end-to-end recognition function.
[0051] In an exemplary embodiment of this disclosure, the native sampling process for pixel color sampling in the related technology can realize automatic picking function in game AI function. In the game scene, the color channel (RBG) value of the target coordinate pixel to be sampled needs to be customized. According to the game scene, specific RBG value conditions are set. For the sampling of the target pixel, the related technology needs to extract the minimum common buffer of the coordinates of multiple pixels. Finally, the target color value is obtained from the extracted buffer and stored in an array for callback to the upper layer. It does not need to traverse every pixel in the entire buffer. As mentioned above, the related technology can only sample the overall color value corresponding to multiple pixels and cannot sample the color value of independent pixels. Therefore, the RBG value conditions set by the related technology are also for the overall color value corresponding to multiple pixels. Therefore, the related technology can only be used for scene recognition for the overall color value corresponding to multiple pixels. The applicable edge recognition scenarios are limited and cannot be applied to more granular edge recognition scenarios.
[0052] In an exemplary embodiment of this disclosure, an electronic device in the related art periodically samples a region through a sub-thread (RegionSampling) in the SurfaceFlinger process to periodically acquire the pixel color value of a certain region. Figure 2 This is a block diagram illustrating a pixel color sampling process in a related art according to an exemplary embodiment of this disclosure. For example... Figure 2As shown, the relevant technologies use the following method to obtain the pixel color value of a certain region at the system level through SurfaceFlinger: When the pixel color sampling function needs to be enabled on the business side, the listener is registered through the static method register of the CompositionSamplingPixelColorListener, mainly passing the sampling area and the listener: The listener is registered by registering the CompositionSamplingPixelColorListener, i.e., passing in the region to be sampled. The listener is registered across processes to the SurfaceFlinger process: The CompositionSamplingListener is registered (CompositionSamplingListener::register), and then the CompositionSamplingListener is registered at the system level, i.e., local registration is performed for the CompositionSamplingListener (CompositionSamplingListener::nativeRegister). A region sampling listener is added through the graphics compositing service (SurfaceFlinger::addRegionSamplingListener). Finally, a listener is added through the region sampling thread (RegionnSamplingThread::addListener). After registering the listener in the graphics compositing service, sampling of the target area begins: sampling starts when the default sampling period is met and when a new frame is being composed. Samples are captured via the region sampling thread (RegionSamplingThread::captureSample). The luma values corresponding to the color descriptor set (mDescriptor) are collected via the region sampling thread's sampling buffer (RegionSamplingThread::sampleBuffer). The luma values of the target area are collected via a floating-point sampling area (float sampleArea). After the view composition sampling listener completes sample collection (view CompositionSamplingListeneronSampleCollected), the sampled color array is called back. If the business side confirms that samples have been collected (onSampleCollected) and uses the luma values of the collected area, the callback method is overridden. After the business side confirms that samples have been collected and the luma values of the collected area have been used, the listener is destroyed, i.e., the composition sampling listener is unregistered: the CompositionSamplingListener is unregistered (CompositionSamplingListener::unregisterListener).The system performs deregistration of the composition sampling listener (CompositionSamplingListener::nativeUnregisterListener) at the system level. It also removes the region sampling listener (SurfaceFlinger::removeRegionSamplingListener) via the graphics compositing service. Finally, it removes the listener from the corresponding color descriptor set on the region sampling thread (RegionSamplingThread::removeListener). In summary, when color values are sampled using a sub-thread for region sampling at the system level, only the luma value of the sampled region can be obtained. This reflects the average parameter value of multiple pixels within a sampled region, but not the parameter value of a single independent pixel. Furthermore, the luma value obtained through color sampling in these technologies does not support the acquisition of pixel colors and cannot accurately identify the current scene. Therefore, the luma value collected in these technologies cannot support end-to-end recognition functionality.
[0053] In view of this, this disclosure proposes a method for function execution. After an electronic device activates a preset function, M pixels corresponding to the preset function are determined. The preset function includes a function executed at the system service layer based on the color values of the M pixels within a set threshold range collected by the system's underlying layer. The pixel coordinates and listeners corresponding to the M pixels are registered at the system's underlying layer. The listeners are used to monitor changes in the color values of the M pixels within the set threshold range. Based on the listeners and the M pixel coordinates, the color values of the M pixels within the set threshold range are collected. When the color values of the M pixels within the set threshold range meet the preset color value conditions, the preset function is executed at the system service layer. This disclosure provides a color sampling function for single or multiple pixels within a set threshold range, achieving fine-grained edge recognition and ensuring the stability of preset function triggering.
[0054] Figure 3 This is a flowchart illustrating a method for performing a function according to an exemplary embodiment. For example... Figure 3 As shown, the method includes the following steps.
[0055] In step S101, in response to the electronic device activating a preset function, M pixels corresponding to the preset function are determined. The preset function includes the function executed by the system service layer based on the color values of the M pixels collected from the system's underlying layer within a set threshold range, where M is a positive integer.
[0056] In step S102, the pixel coordinates corresponding to M pixels and the listeners corresponding to M pixels are registered to the system layer. The listeners are used to monitor the changes in the color values of the M pixels within a set threshold range.
[0057] In step S103, based on the listener and the coordinates of M pixels, the color values of M pixels within a set threshold range are collected.
[0058] In step S104, in response to the fact that the color values of M pixels within a set threshold range meet the preset color value conditions, a preset function is executed in the system service layer.
[0059] In this embodiment, the preset function includes a function executed by the system service layer corresponding to the color values of M pixels collected from the system's underlying layer within a set threshold range. The set threshold range is the area range corresponding to each independent pixel among the M pixels. The set range can be adapted based on user needs and can be a circular area centered on the independent pixel with a radius equal to the pixel length, or a rectangular area centered on the independent pixel with a variable pixel length. In one example, when the set range is 1x1, the color values of the M pixels within the set threshold range are the color values corresponding to the M pixels respectively. In another example, the preset function in this disclosure can be an automatic item pickup function in a game scene, automatically picking up items when the presence of dropped items is determined based on the color values of the M pixels within the set threshold range. The preset function in this disclosure can also be an automatic de-control function, automatically triggering the de-control function when the control of an operation object in the current scene is determined based on the color values of the M pixels within the set threshold range. It is understood that this disclosure provides different preset functions, and considering that different preset functions address different system scenarios, different pixel combinations are provided for different preset functions.
[0060] In this implementation, after the electronic device enables a preset function, the pixel coordinates of M pixels and the corresponding listeners for M pixels are registered to the system's underlying layer. The system's underlying layer then monitors the color value changes of the M pixels within a set threshold range using these listeners. In this disclosure, the pixel coordinates of the M pixels can each correspond to a separate listener, or multiple pixel coordinates can correspond to a single listener. Given that detecting pixel color value changes through listeners places certain demands on the device's system computing resources, and the more listeners there are, the higher the computational resource requirements become, this disclosure sets different registration methods for pixel coordinates and corresponding listeners for different devices. For example, for electronic devices with high computing performance, given their sufficient system computing resources, multiple pixel coordinates can each correspond to a separate listener; conversely, for electronic devices with low computing performance, given their insufficient system computing resources, multiple pixel coordinates can each correspond to a single listener.
[0061] In this embodiment, after registering the pixel coordinates and listeners corresponding to M pixels to the system layer, the system layer continuously collects the color values of the M pixels within a set threshold range based on the listeners and coordinates. If the color values of the M pixels within the set threshold range satisfy a preset color value condition, a preset function is executed at the system service layer. The preset color value condition includes color value conditions specific to each pixel, and these conditions differ for each pixel. The preset color value condition can correspond to a fixed color value; that is, if the color values of the M pixels within the set threshold range are all the color values specified by the color value condition, then the color values of the M pixels within the set threshold range are determined to satisfy the preset color value condition. Alternatively, the preset color value condition can correspond to a preset color value orientation; that is, if the color values of the M pixels within the set threshold range are respectively within the color value range specified by the color value condition, then the color values of the M pixels within the set threshold range are determined to satisfy the preset color value condition.
[0062] In this embodiment, after the electronic device activates a preset function, M pixels corresponding to the preset function are determined. The pixel coordinates of the M pixels and the corresponding listeners are registered to the system's underlying layer. Color values of the M pixels within a set threshold range are collected based on the listeners and the M pixel coordinates. If the color values of the M pixels within the set threshold range meet the preset color value conditions, the preset function is executed at the system service layer. This disclosure provides a color sampling function for single or multiple pixels within a set threshold range, achieving fine-grained edge recognition and ensuring the stability of preset function triggering.
[0063] In an exemplary embodiment of this disclosure, Figure 4 This is a schematic diagram illustrating a pixel color sampling method according to an exemplary embodiment. For example... Figure 4 As shown, this disclosure allows for pixel color sampling in the following manner: In response to enabling the preset function, a sampling area is determined on the display interface, and multiple target pixels to be sampled are identified within the sampling area. The pixel coordinates of the multiple target pixels to be sampled and their corresponding listeners are registered to the system's underlying layer. After the pixel coordinates of the multiple target pixels to be sampled and their corresponding listeners are registered to the system's underlying layer, the color values of the target pixels are continuously acquired and saved for the multiple target images to be sampled. The pixel coordinates and color values corresponding to the multiple target pixels are stored as a pixel point cloud, and the pixel point cloud is transmitted to the application side.
[0064] In this embodiment, after registering the pixel coordinates of M pixels and the listeners corresponding to the M pixels to the system layer, it is necessary to continuously collect the color values of the M pixels within a set threshold range based on the listeners and the M pixel coordinates. When the color values of the M pixels within the set threshold range meet a preset color value condition, a preset function is executed at the system service layer. Based on this, this solution sets a specific sampling period for the preset function, thereby continuously collecting the color values of the M pixels within the set threshold range based on the sampling period. The following embodiments of this disclosure describe the method for collecting the color values of M pixels within the set threshold range.
[0065] Figure 5 This is a flowchart illustrating a method for setting a sampling period and acquiring color values of M pixels within a set threshold range according to an exemplary embodiment. Figure 5 As shown, the method includes the following steps.
[0066] In step S201, the M pixel coordinates and the M corresponding listeners are registered to the system's underlying layer.
[0067] In step S202, the time period corresponding to the preset function is set as the sampling period for collecting the color values of M pixels within a set threshold range.
[0068] In step S203, the color values of M pixels within a set threshold range are periodically collected according to the sampling period.
[0069] In this embodiment, the electronic device's system is configured with a default pixel color sampling function, which has a fixed default sampling period. This default pixel color sampling function is a sampling function that samples the overall color values of a specified display area. Given that the sensitivity requirements of different preset functions in this disclosure vary, the sampling period requirement for collecting color values of M pixels within a set threshold range changes with the preset function requirements. A fixed default sampling period cannot meet the dynamic requirements of pixel color value sampling in this disclosure. Therefore, this disclosure sets different time periods for different preset functions. After registering the M pixel coordinates and their corresponding listeners to the system layer, the pixel color sampling period is dynamically modified. The fixed default sampling period is discontinued, and the time period corresponding to the preset function is set as the sampling period for collecting color values of M pixels within a set threshold range. Based on the sampling period, the color values of M pixels within the set threshold range are periodically collected.
[0070] In one example, the default function in this disclosure is an automatic pickup function in a game scene, which can set 100ms as the sampling period for color sampling, thereby greatly improving the success rate of picking up items.
[0071] This disclosure provides different sampling periods for different preset functions. After enabling a preset function, and with M pixel coordinates and their corresponding listeners registered at the system's underlying layer, the sampling period for the preset function is set to the sampling period for collecting color values of the M pixels within a set threshold range. This ensures that the color values of the M pixels within the set threshold range are periodically sampled at a sampling period consistent with the sensitivity range of the preset function, guaranteeing that the color values of the M pixels within the set threshold range meet the preset color value conditions. The preset function then executes stably at the system service layer with low latency.
[0072] It is understood that electronic device displays have a default refresh rate or a specific refresh rate set by the user, such as 144Hz or 60Hz. The electronic device refreshes its display interface according to the set refresh rate, and the display interface may change while the device is refreshing. Therefore, this disclosure requires pixel color acquisition while the electronic device is refreshing its display to ensure the rapid execution of preset functions. The following embodiments of this disclosure further illustrate the method for acquiring color values of M pixels within a set threshold range.
[0073] Figure 6 This is a flowchart illustrating a method for acquiring color values of M pixels within a set threshold range based on the refresh rate of an electronic device, according to an exemplary embodiment. Figure 6 As shown, the method includes the following steps.
[0074] In step S301, in response to the electronic device activating a preset function, the M pixels corresponding to the preset function are determined, and the pixel coordinates and listeners corresponding to the M pixels are registered to the system layer. The electronic device refreshes and synthesizes a new image frame based on a preset refresh rate and collects the color values of the M pixels within a set threshold range.
[0075] In step S302, in response to the synthesis of a new image frame, the color values of M pixels within a set threshold range are acquired.
[0076] In this embodiment, after the electronic device enables a preset function and registers the pixel coordinates and listeners corresponding to M pixels to the system layer, it collects the color values of the M pixels within a set threshold range. While periodically collecting data according to the sampling period corresponding to the preset function, it also collects the color values of the M pixels within the set threshold range each time the electronic device refreshes its display interface and synthesizes a new image frame, based on the refresh rate of the electronic device. Therefore, when the color values of the M pixels within the set threshold range meet the preset color value conditions, the preset function is quickly executed at the system service layer.
[0077] This disclosure provides two sampling conditions for color sampling in the preset sampling function. The first is a sampling period set for the preset function, during which M pixels with color values within a set threshold range are collected. The second is based on the device's refresh rate, where M pixels with color values within a set threshold range are collected each time the electronic device refreshes the display interface and synthesizes a new image frame. These two sampling conditions ensure that the preset function is executed quickly at the system service layer when the color values of the M pixels within the set threshold range meet the preset color value conditions.
[0078] In this embodiment, the preset function is a preset function set for a preset application of the electronic device. The preset function includes a function executed at the system service layer based on the color values of M pixels within a set threshold range when the display interface corresponding to the preset application is displayed. If other layers are overlaid on the display interface of the preset application, and these other layers overlap with the M pixels, then the color values of the pixels overlapping with other layers are not the color values of the display interface of the preset application and cannot be used as the triggering condition for the preset function. Based on this, when other layers are overlaid on the display interface of the preset application, this disclosure needs to determine whether to collect the color values corresponding to the M pixels based on the overlap relationship between the M pixels and other layers. The following embodiments of this disclosure further illustrate the method for collecting the color values of M pixels within a set threshold range.
[0079] Figure 7 This is a flowchart illustrating a method for acquiring color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by a first type of layer, according to an exemplary embodiment. Figure 7 As shown, the method includes the following steps.
[0080] In step S401, in response to the electronic device enabling a preset function, M pixels corresponding to the preset function are determined, and the pixel coordinates and listeners corresponding to the M pixels are registered to the system layer. The preset function includes a function executed at the system service layer based on the color values of the M pixels within a set threshold range when displaying the display interface corresponding to the preset application.
[0081] In step S402, in response to the fact that the part of the display interface corresponding to the preset application is covered by the first type of layer, and the first type of layer and M pixels do not overlap within the set threshold range, the color values of M pixels within the set threshold range are collected again, and the first type of layer is not set with system sampling function.
[0082] In this embodiment, the first type of layer is a layer category without system sampling functionality. The system sampling function in this disclosure is a system function that consumes public buffer resources and graphics processing unit (GPU) resources. The system sampling function can be touch sampling or slider sampling. The pixel color sampling function used to trigger the execution of preset functions in this disclosure also requires public buffer resources and graphics processing unit (GPU) resources. Therefore, when the first type of layer is a layer category without system sampling functionality, the pixel color sampling function in this disclosure remains enabled, and the decision to continue sampling the color values of M pixels within a set threshold range is made based on the overlap relationship between the first type of layer and M pixels. In one example, the first type of layer is a pop-up window from another application or a system pop-up window without system sampling functionality.
[0083] In this embodiment of the present disclosure, when the partial display interface corresponding to the preset application is covered by the first type of layer and the first type of layer and M pixels do not overlap within the set threshold range, the color values of the M pixels within the set threshold range are still the color values of the display interface corresponding to the preset application, which can be used as the triggering condition for the preset function. Therefore, the color values of the M pixels within the set threshold range are continued to be collected.
[0084] The following embodiments further illustrate a method for acquiring color values of M pixels within a set threshold range.
[0085] Figure 8 This is a flowchart illustrating a method for acquiring color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by a first-type layer, according to yet another exemplary embodiment. Figure 8 As shown, the method includes the following steps.
[0086] In step S501, in response to the electronic device enabling a preset function, M pixels corresponding to the preset function are determined, and the pixel coordinates and listeners corresponding to the M pixels are registered to the system layer. The preset function includes a function executed at the system service layer based on the color values of the M pixels within a set threshold range when displaying the display interface corresponding to the preset application.
[0087] In step S502, in response to the fact that a portion of the display interface corresponding to the preset application is covered by the first type of layer, and the first type of layer and M pixels overlap in at least a part of the set threshold range, the acquisition of color values of M pixels within the set threshold range is stopped.
[0088] In this embodiment of the disclosure, the overlap between the first type layer and the M pixels within a set threshold range includes: the first type layer and each pixel in the M pixels overlap within the set threshold range; or at least a portion of the first type layer and the M pixels overlap within the set threshold range. It is understood that when a pixel in the first type layer intersects with the M pixels, the content displayed by the pixels in the M pixels within the set threshold range that intersect with the corresponding pixel in the first type layer is the display content of the first type layer, not the display content of the portion of the interface corresponding to the preset application. Since the display content of the first type layer cannot be used as a trigger for executing a preset function, the portion of the M pixels overlapping with the first type layer within the set threshold range cannot be used as a trigger for executing a preset function. Therefore, when the first layer and M pixels partially or completely overlap within the set threshold range, the collection of color values of the M pixels within the set threshold range will be stopped to prevent the color of the display interface corresponding to the first layer from being mistakenly collected, which would cause the preset function to be falsely triggered. Furthermore, by stopping pixel color sampling during the period when the first layer and M pixels overlap within the set threshold range, the power consumption of the device can be reduced.
[0089] In this embodiment of the present disclosure, when the part of the display interface corresponding to the preset application is covered by the first type of layer and the first type of layer and M pixels overlap at least part of the set threshold range, it is characterized that part of the color values of the M pixels in the set threshold range are the color values corresponding to the first type of layer, which cannot be used as the triggering condition for the preset function. Therefore, the collection of the color values of the M pixels in the set threshold range is stopped.
[0090] The following embodiments further illustrate a method for acquiring color values of M pixels within a set threshold range.
[0091] Figure 9 This is a flowchart illustrating a method for acquiring color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by a second type of layer, according to an exemplary embodiment. Figure 9 As shown, the method includes the following steps.
[0092] In step S601, in response to the electronic device enabling a preset function, M pixels corresponding to the preset function are determined, and the pixel coordinates and listeners corresponding to the M pixels are registered to the system layer. The preset function includes a function executed at the system service layer based on the color values of the M pixels within a set threshold range when displaying the display interface corresponding to the preset application.
[0093] In step S602, in response to the fact that the part of the display interface corresponding to the preset application is covered by the second type of layer, the collection of color values of M pixels within the set threshold range is stopped. The second type of layer is equipped with a system sampling function.
[0094] In this embodiment, the second type of layer is a layer category with system sampling functionality. As mentioned above, the system sampling functionality in this disclosure consumes public buffer resources and graphics processing unit (GPU) resources. The pixel color sampling function used to trigger the execution of the preset function also requires public buffer resources and GPU resources. Therefore, when the second type of layer is a layer category with system sampling functionality, public buffer resources and GPU resources are occupied by the second type of layer. If the pixel color sampling function in this disclosure remains enabled, excessive system load and device power consumption will occur. Therefore, when the portion of the display interface corresponding to the preset application is covered by the second type of layer and the second type of layer is a layer category with system sampling functionality, this disclosure stops collecting color values of M pixels within a set threshold range. This maintains a normal system load, avoids excessive device power consumption, and extends the device's battery life.
[0095] In an exemplary embodiment of this disclosure, Figure 10 This is a flowchart illustrating a method for acquiring color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by other layers, according to an exemplary embodiment. Figure 10 As shown, this disclosure employs the following method to control the acquisition of color values of M pixels within a set threshold range when a portion of the display interface corresponding to a preset application is covered by other layers: In response to enabling the preset function, the acquisition of color values of M pixels within the set threshold range begins, i.e., sample capture begins. If it is determined that a multi-coordinate color sampling scenario has been entered and an overlay layer exists above the preset application layer, the category of the overlay layer is determined. If the overlay layer is classified as a second-type layer overlay (with system sampling function enabled), multi-coordinate color sampling is stopped, and the sample capture process waits to be executed. If the overlay layer is classified as a first-type layer overlay (without system sampling function enabled), it is determined whether there is an intersection between the overlay layer, the sampling point, and the simulated click point. If there is an intersection between the overlay layer, the sampling point, and the simulated click point, multi-coordinate color sampling is stopped, and the sample capture process waits to be executed. If there is no intersection between the overlay layer, the sampling point, and the simulated click point, multi-coordinate color sampling is performed, and sample capture continues.
[0096] In this embodiment, the preset functions are functions that are enabled or disabled based on user needs. The sampling period set for the preset functions is also dynamically adjusted depending on whether the preset functions are enabled or disabled.
[0097] Figure 11 This is a flowchart illustrating a method for setting a sampling period according to an exemplary embodiment. Figure 11 As shown, the method includes the following steps.
[0098] In step S701, in response to the electronic device enabling a preset function, M pixels corresponding to the preset function are determined, and the pixel coordinates and listeners corresponding to the M pixels are registered to the system layer.
[0099] In step S702, in response to disabling the preset function, the registered M pixel coordinates and listeners are destroyed at the system level, and the sampling period is set to the default sampling period.
[0100] In this embodiment of the disclosure, when the electronic device has a preset function enabled, the pixel coordinates corresponding to M pixels and the listeners corresponding to M pixels are registered at the system layer, and the time period corresponding to the preset function is set to the sampling period for collecting the color values of M pixels within a set threshold range. When the preset function is disabled, the registered M pixel coordinates and listeners are destroyed at the system layer, and the sampling period is set to the default sampling period for the default pixel color sampling function.
[0101] In this embodiment of the disclosure, if the color values of M pixels within a set threshold range meet a preset color value condition, a preset function is executed at the system service layer. The following embodiments of this disclosure describe the method for executing the preset function.
[0102] Figure 12 This is a flowchart illustrating a method for performing a preset function by simulating touch operation, according to an exemplary embodiment. Figure 12 As shown, the method includes the following steps.
[0103] In step S801, in response to the M pixels having color values within a set threshold range satisfying a preset color value condition.
[0104] In step S802, a touch operation is simulated at the system service layer for a preset touch area, which may include or exclude the display area containing M pixels.
[0105] In this embodiment, a preset touch area is set for a preset function. Depending on the preset function, the preset touch area may or may not include the display area containing M pixels. When the color values of the M pixels within a set threshold range meet a preset color value condition, the preset function is triggered by simulating a touch operation on the preset touch area at the system service layer. The simulated touch operation on the preset touch area may vary depending on the preset function and includes, but is not limited to, single click, continuous click, long press, and drag.
[0106] Understandably, the overlay of other layers can affect the normal execution of simulated touch operations on the preset touch area. Therefore, this disclosure controls the acquisition of color values for M pixels within a set threshold range based on the layer categories of other layers and the overlap relationship between other layers and the preset touch area: If the preset touch area corresponding to the preset application is not covered by a first-type layer and the first-type layer and the M pixels do not overlap within the set threshold range, the acquisition of color values for the M pixels within the set threshold range continues. If the preset touch area corresponding to the preset application is covered by a first-type layer, or if the first-type layer and the M pixels overlap at least part of the set threshold range, the acquisition of color values for the M pixels within the set threshold range is stopped.
[0107] In an exemplary embodiment of this disclosure, Figure 13 This is a schematic diagram illustrating an application scenario of a function execution method according to an exemplary embodiment of this disclosure, such as... Figure 13 As shown, the sampling area corresponding to the M pixels in this disclosure can be a sampling area including multiple text boxes such as A, B, and C. The preset touch area set for this sampling area does not include the display area where the M pixels are located. The above exemplary embodiment can be applied to automatic collection in a game scene. By continuously collecting the color values of the M pixels, it is determined whether equipment drops. When the color values of the M pixels meet the preset conditions, it is determined that equipment drops, and a simulated click operation is performed in the preset touch area to achieve automatic collection.
[0108] In an exemplary embodiment of this disclosure, Figure 14 This is a schematic diagram illustrating an application scenario of a function execution method according to yet another exemplary embodiment of this disclosure, such as... Figure 14As shown, the sampling area corresponding to the M pixels in this disclosure can be the sampling area of the corresponding virtual control. The preset touch area set for this sampling area includes the display area where the M pixels are located. The above exemplary embodiment can be applied to automatic de-control in a game scene. By continuously collecting the color values of the M pixels, it is determined whether a virtual control for automatic de-control appears. When the color values of the M pixels meet the preset conditions, it is determined that a virtual control for automatic de-control has appeared, and a simulated click operation is performed in the preset touch area to achieve automatic de-control.
[0109] In an exemplary embodiment of this disclosure, the function execution method of this disclosure is implemented in a system based on an electronic device in the following manner:
[0110] 1. When the system business layer has a preset function and requires pixel color sampling, it registers the listening function through the system's encapsulated HyperCompositionSamplingListener. The listener and the coordinate array of the color to be sampled are required. The coordinate array of the color to be sampled includes the pixel coordinates of M pixels.
[0111] 2. Call the newly added registerPixelColorListener method in the native class CompositionSamplingListener, passing in the listener and the color sampling coordinate array.
[0112] 3. Call the newly added native unregister pixel color listener (nativeRegisterPixelColorListener) method in the native class composite sampling listener to prepare to pass the listener and the array parameters of the color to be sampled to the system bottom layer through the system bottom layer interface (Java Native Interface, JNI).
[0113] 4. Using a cross-process approach (Binder), the newly added addRegionSamplingPixelColorListener method in the graphics compositing service (SurfaceFlinger) is invoked, passing the listener and the array of colors to be sampled as parameters.
[0114] 5. Call the newly added addPixelColorListener method in the RegionSamplingThread to perform the final listening registration, save the parameter listener and the array of colors to be sampled in the ColorDescriptor structure, and further save the ColorDescriptors structure in the color descriptor collection (mColorDescriptors) of type map.
[0115] 6. When a color descriptor exists in the color descriptor set, dynamically modify the sampling period to the expected time, such as 100ms.
[0116] 7. When the sampling period is met and a new frame is being synthesized, sampling begins. The `RegionSamplingThread::captureSample` method of the registered sampling thread is called. When a layer is overlaid on the layer to be color sampled, different strategies are adopted for different layer types. For system small window scenarios: color sampling is disabled. For regular layer overlays: if the overlaid layer intersects with the sampling point and the simulated click point, color sampling is disabled; otherwise, the color sampling function operates normally.
[0117] 8. Call the newly added combined pixel color sampling (sampleMultiPixelColors) method in the region sampling thread to traverse the color descriptors stored in the color descriptor collection.
[0118] 9. Call the newly added "sampleBufferColor" method in the region sampling thread to traverse the coordinates of each color to be sampled in the color descriptor's coordinate array.
[0119] 10. Call the newly added sampling area color (sampleAreaColor) method in the area sampling thread to collect the color value of the target coordinate pixel.
[0120] 11. The `onPixelColorCollected` method registered in the callback listener is defined in the system view (android_view_) composite sampling listener class. The system business layer overrides the `onPixelColorCollected` method and passes the array of color coordinates of successfully sampled pixels to the system business layer through a callback.
[0121] 12. When the system business layer does not require real-time color sampling, the registered listeners need to be destroyed. The system business layer destroys the listeners through the system-encapsulated hypercomposite sampling listener, which requires passing in the listener.
[0122] 13. Call the newly added unregisterPixelColorListener method in the native class composite sampling listener, and pass in the listener to be destroyed.
[0123] 14. Call the newly added system-level unregisterPixelColorListener method in the native class composite sampling listener to prepare to pass the listener to be destroyed to the system level through the system-level interface.
[0124] 15. By calling the newly added removeRegionSamplingPixelColorListener method in the graphics compositing service across processes.
[0125] 16. Call the newly added removePixelColorListener method in the region sampling thread to remove the corresponding listener stored in the color descriptor set.
[0126] 17. When the color descriptor set data structure is empty, dynamically adjust the sampling period to the native period.
[0127] In an exemplary embodiment of this disclosure, Figure 15 This is a block diagram illustrating a pixel color sampling process according to an exemplary embodiment of the present disclosure, such as... Figure 15As shown, this disclosure can execute the pixel color sampling process in the following manner: A system service registers a listener, i.e., it inputs the area to be sampled, which includes M pixels and supports distributed multi-coordinate sampling, and registers a combined sampling pixel color listener. During system service registration, a super-combined sampling listener encapsulation method, a combined sampling listener::register pixel color listener method, and a combined sampling listener::system-level pixel color listener registration method are used. The pixel coordinates of the M pixels and the corresponding listeners are registered to the system-level layer through the graphics display compositing service::add region sampling pixel color listener method. The graphics display compositing service adds pixel color listeners through the region sampling thread, storing parameters such as the coordinate array in the color descriptor set. The sampling period is dynamically modified; sampling begins when the sampling period is met and when a new frame is being composited. The region sampling thread captures samples. For layer overlay scenarios, a preset layer overlay scenario sampling strategy is adopted: 1. First layer overlay: Color sampling is disabled. The first type of layer overlay can be used in system small window scenarios; 2. Second type of layer overlay: If there is an intersection with the sampling point and the simulated click point, color sampling is disabled; otherwise, the color sampling function runs normally. The second type of layer overlay can be used in regular layer overlay scenarios. Through the region sampling thread: Sample multiple pixel colors by traversing the color descriptors in the color descriptor set. Collect the target coordinate color sampling region color. Through the combined sampling listener: pixel color collection method, the sampled color array view is called back. If the pixel color has been collected, the collected coordinate color data is used, and the preset function is exited, the callback method is overridden. Furthermore, the registered listeners are destroyed by unregistering the combined sampling listener method: through the super combined sampling listener encapsulation method, combined sampling listener: unregister pixel color listener method, combined sampling listener: system-level unregister pixel color listener method, graphics display compositing service: remove region sampling pixel color listener method, region sampling thread: remove pixel color listener method, remove the corresponding data from the color descriptor set, thereby destroying the registered listeners.
[0128] In an exemplary embodiment of this disclosure, Figure 16 This is a schematic diagram illustrating a code flow for sampling pixel color according to an exemplary embodiment of the present disclosure, such as... Figure 16As shown, this disclosure employs the following method for pixel color sampling: First, after enabling the preset function and determining the M pixels corresponding to the preset function, a coordinate color sampling listener registration process is executed, registering pixel color listeners (registerPixelColorListener), registering the pixel coordinates and listeners corresponding to the M pixels to the system layer. The CompositionSamplingListener executes the native RegisterPixelColorListener process (nativeRegisterPixelColorListener). After the CompositionSamplingListener transmits the natively registered pixel color listeners to the graphics CompositionSamplingListener (graphics_CompositionSamplingListener), the graphics CompositionSamplingListener adds a region sampling pixel color listener (addRegionSamplingPixelColorListener) to the SurfaceComposerClient. The SurfaceComposerClient adds a region sampling pixel color listener (addPixelColorListener) to the SurfaceFlinger service. The SurfaceFlinger service adds a pixel color listener to the RegionSamplingThread, thus completing the coordinate color sampling listener registration process. Then, sampling begins when the sampling conditions are met. A sampling request is sent to the graphics compositing service, and M pixel color values within a set threshold range are periodically collected according to the sampling period corresponding to the preset function. This is repeated when the electronic device refreshes and composites new image frames based on a preset refresh rate. The graphics compositing service sends a "composition input complete" (onCompositionComplete) instruction to the region sampling thread. The region sampling thread performs sampling (doSample) and capture (captureSample), sampling multiple independent pixel colors (captureSample). After sampling multiple independent pixel colors (sampleMultiPixelColor), the region sampling thread determines that the pixel colors have been collected (onPixelColorCollected) and passes the collected pixel colors to the graphics compositing sampling listener. The graphics compositing sampling listener dispatches a pixel color collection event (dispatchOnPixelColorCollected) to the compositing sampling listener. The compositing sampling listener confirms that the pixel colors have been collected.Upon confirming that pixel colors have been collected, the coordinate color sampling listener is destroyed, and an instruction to unregister the pixel color listener is sent to the combined sampling listener. The combined sampling listener unregisters the pixel color listener locally (nativeUnregisterPixelColorListener) and then passes this instruction to the graphics combined sampling listener. The graphics combined sampling listener sends an instruction to the surface compositing client to remove the region sampling pixel color listener (removeRegionSamplingPixelColorListener). The surface compositing client sends an instruction to the graphics display service to remove the region sampling pixel color listener. The graphics display compositing service sends an instruction to the region sampling thread to remove the pixel color listener (removePixelColorListener). After the region sampling thread removes the pixel color listener, the coordinate color sampling listener is confirmed to have been destroyed, and the sampling period is set to the default sampling period.
[0129] In this embodiment, when a preset function is enabled on the electronic device, M pixels corresponding to the preset function are determined. The preset function includes a function executed by the system service layer corresponding to the color values of the M pixels within a set threshold range, collected from the system's underlying layer. The pixel coordinates and listeners corresponding to the M pixels are registered to the system's underlying layer. The time period corresponding to the preset function is set as the sampling period for collecting the color values of the M pixels within the set threshold range. According to the sampling period, the color values of the M pixels within the set threshold range are periodically collected, and when the electronic device synthesizes a new image frame based on a preset refresh rate, the color values of the M pixels within the set threshold range are collected again. When a portion of the display interface corresponding to the preset application is covered by a first type of layer and the first type of layer and the M pixels do not overlap within the set threshold range, the collection of the color values of the M pixels within the set threshold range continues, wherein the first type of layer does not have a system sampling function enabled. When a portion of the display interface corresponding to the preset application is covered by the first type of layer and the first type of layer and at least a portion of the M pixels within the set threshold range overlap, the collection of the color values of the M pixels within the set threshold range is stopped. When the portion of the display interface corresponding to the preset application is covered by the second type of layer, the sampling of color values of M pixels within a set threshold range is stopped. The second type of layer has a system sampling function. If the color values of the M pixels within the set threshold range meet the preset color value conditions, a touch operation is simulated at the system service layer for a preset touch area. The preset touch area may or may not include the display area containing the M pixels. When the preset function is disabled, the registered coordinates and listeners of the M pixels are destroyed at the system level, and the sampling period is set to the default sampling period. This disclosure provides color sampling functionality for single or multiple pixels by modifying system-level code. The pixel color sampling function is modified based on the native sampling logic, while isolating pixel color sampling from native sampling, sharing only one set of listener management mechanisms to avoid affecting the native sampling process and ensuring system stability and process rationality. The pixel color sampling function can be used for edge recognition in various scenarios, such as automatic pickup and automatic de-control functions in game scenes. Furthermore, by sampling color values of single or multiple pixels within a set threshold range, fine-grained edge recognition is achieved, ensuring the stability of preset function triggering.
[0130] Based on the same concept, this disclosure also provides a function execution device 100.
[0131] It is understood that the function execution device 100 provided in this disclosure embodiment includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure embodiment, this disclosure embodiment can be implemented in hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of this disclosure embodiment.
[0132] Figure 17 This is a block diagram illustrating a function execution device 100 according to an exemplary embodiment. (Refer to...) Figure 17 The device includes a determination unit 101, a registration unit 102, a data acquisition unit 103, and an execution unit 104.
[0133] The determining unit 101 is used to determine M pixels corresponding to the preset function in response to the electronic device activating the preset function. The preset function includes the function executed by the system service layer based on the color values of the M pixels collected from the system's underlying layer within a set threshold range, where M is a positive integer.
[0134] Registration unit 102 is used to register the pixel coordinates corresponding to M pixels and the listeners corresponding to M pixels to the system bottom layer. The listeners are used to monitor the changes in the color values of the M pixels within a set threshold range.
[0135] The acquisition unit 103 is used to acquire the color values of M pixels within a set threshold range based on the listener and the coordinates of M pixels.
[0136] The execution unit 104 is used to execute a preset function in the system service layer in response to the fact that the color values of M pixels within a set threshold range meet the preset color value conditions.
[0137] In some possible implementations, the preset function corresponds to a time period for acquiring pixel color values, and the device further includes a sampling period setting unit 105. After registering the M pixel coordinates and the corresponding listeners to the system layer, the sampling period setting unit 105 is used to: set the time period corresponding to the preset function as the sampling period for acquiring the color values of the M pixels within a set threshold range. The acquisition unit 103 acquires the color values of the M pixels within the set threshold range according to the listeners and the M pixel coordinates in the following manner: periodically acquiring the color values of the M pixels within the set threshold range according to the sampling period.
[0138] In some possible implementations, the electronic device refreshes and synthesizes a new image frame based on a preset refresh rate, and the acquisition unit 103 acquires the color values of M pixels within a set threshold range in the following manner: in response to the synthesis of a new image frame, the color values of M pixels within a set threshold range are acquired.
[0139] In some possible implementations, the preset function includes a function performed in the system service layer based on the color values of M pixels within a set threshold range when displaying the display interface corresponding to the preset application; the acquisition unit 103 acquires the color values of M pixels within the set threshold range in the following manner: in response to the partial display interface corresponding to the preset application being covered by the first type layer, and the first type layer and the M pixels not overlapping within the set threshold range, the acquisition of the color values of M pixels within the set threshold range continues, and the first type layer is not set with a system sampling function.
[0140] In some possible implementations, the acquisition unit 103 is further configured to: stop acquiring the color values of the M pixels within the set threshold range in response to a partial display interface corresponding to a preset application being covered by a first type of layer, and the first type of layer and M pixels overlapping in at least a portion of a set threshold range.
[0141] In some possible implementations, the acquisition unit 103 is also used to: stop acquiring the color values of M pixels within a set threshold range in response to a partial display interface corresponding to a preset application being covered by a second type of layer, wherein the second type of layer is provided with a system sampling function.
[0142] In some possible implementations, the sampling period setting unit 105 is also used to: in response to disabling the preset function, destroy the registered M pixel coordinates and listeners at the system bottom layer, and set the sampling period to the default sampling period.
[0143] In some possible implementations, the preset function corresponds to a preset touch area. The execution unit 104 executes the preset function in the system service layer in the following manner: it simulates a touch operation for the preset touch area in the system service layer. The preset touch area includes or does not include the display area where M pixels are located.
[0144] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0145] Figure 18This is a block diagram illustrating an apparatus 200 for performing a function according to an exemplary embodiment. The apparatus 200 can be provided as a terminal. For example, the apparatus 200 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0146] Reference Figure 18 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.
[0147] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.
[0148] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0149] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.
[0150] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0151] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.
[0152] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0153] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0154] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0155] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0156] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0157] Figure 19 This is a block diagram illustrating an apparatus 300 for performing functions according to yet another exemplary embodiment. For example, apparatus 300 may be provided as a server. (Refer to...) Figure 19 The device 300 includes a processing component 322, which further includes one or more processors, and memory resources represented by memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in memory 332 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 322 is configured to execute instructions to perform the aforementioned functional execution methods.
[0158] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 358. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.
[0159] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.
[0160] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.
[0161] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.
[0162] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.
[0163] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.
[0164] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.
[0165] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A method for executing a function, characterized in that, include: In response to the electronic device activating a preset function, M pixels corresponding to the preset function are determined. The preset function includes a function executed by the system service layer based on the color values of M pixels collected from the system's underlying layer within a set threshold range, where M is a positive integer. The pixel coordinates corresponding to the M pixels and the listeners corresponding to the M pixels are registered to the system's underlying layer. The listeners are used to monitor the changes in the color values of the M pixels within the set threshold range. Based on the listener and the coordinates of the M pixels, collect the color values of the M pixels within a set threshold range; In response to the M pixels satisfying a preset color value condition within the set threshold range, the preset function is executed at the system service layer.
2. The method according to claim 1, characterized in that, The preset function corresponds to a time period for collecting pixel color values; After registering the M pixel coordinates and the listeners corresponding to the M pixels to the system's underlying layer, the method further includes: The time period corresponding to the preset function is set as the sampling period for acquiring the color values of the M pixels within the set threshold range; The step of acquiring the color values of the M pixels within a set threshold range based on the listener and the coordinates of the M pixels includes: According to the sampling period, the color values of the M pixels within the set threshold range are periodically collected.
3. The method according to claim 1, characterized in that, The electronic device refreshes and synthesizes new image frames based on a preset refresh rate, and the acquisition of color values of the M pixels within a set threshold range includes: In response to the synthesis of a new image frame, the color values of the M pixels within the set threshold range are acquired.
4. The method according to any one of claims 1-3, characterized in that, The preset function includes functions executed at the system service layer based on the color values of the M pixels within a set threshold range when displaying the display interface corresponding to the preset application; The process of collecting the color values of the M pixels within a set threshold range includes: In response to the fact that a portion of the display interface corresponding to the preset application is covered by the first type of layer, and the first type of layer and the M pixels do not overlap within a set threshold range, the color values of the M pixels within the set threshold range are continued to be collected, and the first type of layer is not set with system sampling function.
5. The method according to claim 4, characterized in that, The method further includes: In response to the fact that a portion of the display interface corresponding to the preset application is covered by a first type of layer, and the first type of layer and the M pixels overlap at least in a portion of a set threshold range, the acquisition of the color values of the M pixels within the set threshold range is stopped.
6. The method according to claim 4, characterized in that, The method further includes: In response to the fact that a portion of the display interface corresponding to the preset application is covered by the second type of layer, the collection of color values of the M pixels within a set threshold range is stopped. The second type of layer is equipped with a system sampling function.
7. The method according to claim 2, characterized in that, The method further includes: In response to disabling the preset function, the registered M pixels and the listener are destroyed at the system level, and the sampling period is set to the default sampling period.
8. The method according to claim 1, characterized in that, The preset functions correspond to preset touch areas. The execution of the preset function at the system service layer includes: The system service layer simulates touch operations for the preset touch area, which may or may not include the display area where the M pixels are located.
9. A functional execution device, characterized in that, include: A determining unit is configured to determine M pixels corresponding to a preset function in response to an electronic device activating a preset function. The preset function includes a function executed by a system service layer based on the color values of M pixels collected from the system's underlying layer within a set threshold range, where M is a positive integer. A registration unit is used to register the pixel coordinates corresponding to the M pixels and the listeners corresponding to the M pixels to the underlying system. The listeners are used to monitor the changes in the color values of the M pixels within the set threshold range. The acquisition unit is used to acquire the color values of the M pixels within a set threshold range based on the listener and the coordinates of the M pixels; An execution unit is configured to execute the preset function in the system service layer in response to the M pixels satisfying a preset color value condition within a set threshold range.
10. An electronic device, characterized in that, include: processor: Memory used to store processor-executable instructions; The processor is configured to execute the function execution method according to any one of claims 1 to 8.
11. A storage medium, characterized in that, The storage medium stores instructions that, when executed by a processor, enable the processor to perform the function execution method as described in any one of claims 1 to 8.