A load-reduced rendering method and electronic device

Abstract

Embodiments of the present application provide a load-reduced rendering method. During the period when a user's finger touches the screen, the electronic device can identify various levels of occlusion areas (finger touch area, finger occlusion area, palm occlusion area) based on touch information of the finger touch area, and perform differentiated load-reduced rendering on the various levels of occlusion areas. For example, the rendering method of rendering culling is implemented on the finger touch area, and the rendering methods of reducing resolution, reducing image texture, and feature point clipping are implemented on the finger occlusion area and the palm occlusion area. In addition, different degrees of reduction in resolution, reduction in image texture, and feature point clipping can be implemented on the finger occlusion area with different visibility. In this way, the load of GPU rendering of the game picture can be reduced, and the user experience is not affected.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to a rendering method and electronic device for reducing load. Background Technology

[0002] Game rendering places high demands on the performance of the graphics processing unit (GPU), and prolonged high-resolution game rendering can put a heavy rendering load on the GPU, which may lead to problems such as game stuttering, high power consumption and heat generation, and short battery life. Summary of the Invention

[0003] In a first aspect, embodiments of this application provide a load-reducing rendering method, which may include: an electronic device displaying a screen provided by a first application on a screen, detecting a user's finger touch operation on the screen, and acquiring touch information of the touch operation. Then, the electronic device determines a multi-level occlusion region based on the touch information. The multi-level occlusion region includes a finger touch area and a finger occlusion area, wherein the finger occlusion area surrounds the finger touch area. Furthermore, the electronic device performs load-reducing rendering on the screen of the multi-level occlusion region, wherein the load-reducing rendering is applied to the occlusion region closer to the touch center, where the touch center is the center of the finger touch area. The load-reducing rendering includes one or more of the following: rendering culling, reducing image texture, reducing image resolution, reducing feature points, and reducing shading rate.

[0004] In the first aspect, by implementing the method in the first aspect, electronic devices can more precisely determine the various areas with different degrees of occlusion and adapt to different rendering reduction methods, thereby reducing the GPU rendering load without affecting the user's gaming experience.

[0005] In conjunction with the first aspect, in some embodiments, the rendering load is reduced for the occluded area closer to the touch center. Specifically, this may include rendering culling of the area touched by the finger, and performing one or more of the following processing on the occluded area: reducing image texture, reducing image resolution, reducing feature points, and reducing shading rate. Specifically, for the occluded area closer to the touch center, the resolution can be reduced to a greater extent, the image texture reduced, and more feature points cropped.

[0006] In conjunction with the first aspect, in some embodiments, the first application may be a game application, and the screen also displays: a directional wheel and skill buttons. Touch operations are applied to the directional wheel or skill buttons, which are respectively located in the lower left and lower right corners of the screen, or respectively in the lower right and lower left corners of the screen.

[0007] The first application is not limited to games; it can also be any other application that provides a graphical user interface and receives user touch input. The approach in the first aspect also applies to such applications to reduce the rendering load of the graphical user interface when the user touches the screen with their finger.

[0008] In conjunction with the first aspect, in some embodiments, the finger touch area is an elliptical area, and the touch information includes the center coordinates, major and minor axes, and elliptical direction of the elliptical area, wherein the elliptical direction is the extension direction of the major axis.

[0009] There can be N finger-occluded regions, where N is a positive integer. These N finger-occluded regions can be constructed from N elliptical regions extended from the finger-touching region. The extended elliptical regions share the same elliptical direction as the finger-touching region, and their major axis vertices are located near the center of the screen. A finger-occluded region is the area remaining after removing the largest nested elliptical region from the extended elliptical region.

[0010] When N=1, the largest elliptical region within this expanded elliptical region is the area touched by the finger. For example, Figure 4 The finger-occluded area 212A is the area remaining after expanding the elliptical region S1 and removing the finger-touching area (i.e., the largest nested elliptical region within the elliptical region S1).

[0011] When N ≥ 2, the N elliptical regions that expand to form the left or right finger occlusion region can be nested, gradually increasing in size to construct multi-level finger occlusion regions with varying degrees of occlusion. The area remaining after removing the largest nested elliptical region from an elliptical region forms the first-level finger occlusion region. For example, Figure 4 The finger-occluded area 212A is the area left after the finger-touching area is removed from the expanded elliptical area S1, and the finger-occluded area 213A is the area left after the elliptical area S1 (i.e., the largest nested elliptical area within the elliptical area S2) is removed from the expanded elliptical area S2.

[0012] In this region, the sum of the two minor axes of the outermost elliptical region is equal to the width W of the first finger, or the difference between the two axes is less than a first value, such as 2 millimeters. The first finger can be the thumb.

[0013] When N≥2, load reduction rendering methods may include: using different shading rates for the N finger occlusion areas, where the game screen of the finger occlusion area closer to the touch center uses a lower shading rate.

[0014] In conjunction with the first aspect, in some embodiments, the multi-level occlusion area may further include: a left palm occlusion area. The left palm occlusion area is the area in the rectangular or triangular region at the lower left corner of the screen that is not covered by the left finger's touch area or the left finger's occlusion area. Specifically, the diagonal of the rectangular region at the lower left corner is the line segment connecting the center of the left finger's touch area to the lower left corner of the screen, and the hypotenuse of the triangular region at the lower left corner is the connecting line segment.

[0015] In conjunction with the first aspect, in some embodiments, the multi-level occlusion area further includes: a right palm occlusion area; the right palm occlusion area is the area in the rectangular or triangular area in the lower right corner of the screen that is not covered by the right finger touch area and the right finger occlusion area. Specifically, the diagonal of the rectangular area in the lower right corner is the line segment connecting the center of the right finger touch area to the lower right corner of the screen, and the hypotenuse of the triangular area in the lower right corner is the connecting line segment.

[0016] In a second aspect, this application provides an electronic device including one or more processors and one or more memories; wherein the memories are coupled to the processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method described in the first aspect or any possible implementation thereof.

[0017] Thirdly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0018] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0019] Fifthly, this application provides a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof. Attached Figure Description

[0020] Figure 1 An electronic device provided in an embodiment of this application is illustrated by way of example;

[0021] Figure 2 This example illustrates a scenario where a user holds an electronic device horizontally to play games;

[0022] Figure 3An exemplary embodiment of this application shows that the finger touch area is simulated as an elliptical region;

[0023] Figure 4 An exemplary embodiment of this application shows that multiple nested elliptical regions are extended from the finger touch area to construct a multi-level finger occlusion area;

[0024] Figure 5 An exemplary embodiment of this application shows a rectangular area extended from the finger touch area to construct a palm-covering area;

[0025] Figure 6 An exemplary illustration shows a palm-covering area constructed based on the finger touch area in an embodiment of this application;

[0026] Figure 7 An exemplary embodiment of this application provides a load-reduction rendering scheme;

[0027] Figure 8 An example is shown of the software system architecture of an electronic device;

[0028] Figure 9 It shows Figure 8 The software system architecture shown illustrates the interaction between various software modules.

[0029] Figure 10 The overall flow of the load reduction rendering method provided in the embodiments of this application is shown. Detailed Implementation

[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0031] Figure 1 An electronic device 100 provided in an embodiment of this application is shown.

[0032] Electronic device 100 can be a mobile phone, tablet computer, handheld computer, personal digital assistant (PDA) or other portable smart device.

[0033] like Figure 1As shown, the electronic device 100 may include: a processor 110, a memory 120, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0034] The sensor module 180 may include a gyroscope sensor 180B, an accelerometer sensor 180E, and a touch sensor 180K, etc.

[0035] The processor 110 can be a processor with powerful processing capabilities and abundant storage and peripheral resources, such as an application processor (AP), neural network processing unit (NPU), graphics processing unit (GPU), modem processor, etc., or it can be a system-on-a-chip (SoC) that integrates multiple processors.

[0036] The processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0037] The processor 110 may include a cache memory, which can be used to store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the cache memory, which can reduce the waiting time of the processor 110 and improve the program running efficiency.

[0038] The processor 110 can also be connected to a memory 120, which may include a program storage area and a user data storage area. The program storage area may store the operating system and one or more applications (such as games), while the data storage area may store data created by the user during use of the electronic device 100 (such as photos and contacts). The memory 120 can be a high-speed random access memory or a non-volatile memory, such as a hard disk, flash memory, or universal flash storage (UFS). The memory 120 can also be an external memory card, such as a Micro SD card.

[0039] The memory 120 may also store code instructions for the load-reducing rendering method provided in the embodiments of this application. When the processor 110 reads the code instructions from the memory 120 and runs the code instructions, the electronic device 100 may execute the load-reducing rendering method.

[0040] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0041] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0042] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0043] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0044] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0045] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0046] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0047] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be manufactured using organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), miniled, microled, micro-OLEDs, quantum dot light-emitting diodes (QLEDs), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0048] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0049] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0050] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0051] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0052] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0053] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0054] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0055] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0056] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0057] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0058] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0059] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0060] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0061] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0062] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0063] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100. Motor 191 can generate vibration prompts. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with electronic device 100.

[0064] Figure 1 The illustrated structure does not constitute a specific limitation on the electronic device 100. The electronic device 100 may include more or fewer components than illustrated, or combine some components, or separate some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0065] The electronic device 100 may have a game application installed. To reduce the load on the GPU when rendering game graphics, this application provides a load-reducing rendering method. This method will be described in detail below.

[0066] Identify occlusion areas at various levels

[0067] like Figure 2 As shown, when a user holds (e.g., in landscape mode) an electronic device 100 to play games, certain areas of the game screen will be obscured by the user's thumb or other hand parts. Generally, the lower left corner of the screen is designed as the control wheel area, and the lower right corner as the skill area. Users need to operate the control wheel and skill areas, thus obstructing the screen and making these parts of the game screen invisible or poorly visible. For example, the pad of the thumb touches and presses on the touchscreen, making the area pressed by the thumb invisible to the user. Simultaneously, the part of the thumb other than the pad hovering near the touchscreen results in poor visibility of the game screen area below it. Additionally, the base of the thumb and the connected palm part often touch or obscure the lower left and lower right corners of the screen, making these areas of the game screen invisible or poorly visible to the user.

[0068] The load-reducing rendering method provided in this application provides precise hierarchical identification of these occluded areas, laying the foundation for subsequent differentiated rendering. Each level of occlusion can include: finger touch area, finger occlusion area, and palm occlusion area. The method for accurately identifying each level of occlusion area is described below:

[0069] Finger touch area recognition

[0070] In this article, the finger-touched area refers to the screen area directly touched by the finger, where the game screen is not visible to the user and has the highest degree of obstruction. Here, "fingers" includes, but is not limited to, human fingers, and may also include robotic fingers that simulate human fingers, prosthetic fingers, etc.

[0071] Finger touch area recognition can be achieved based on touch events reported by the screen (specifically, a touchscreen). When a finger press is detected, the screen collects touch information, such as the location, shape, and size of the touch area, which can be carried in the touch event. Based on this touch information, the electronic device 100 can then recognize the finger touch area.

[0072] Specifically, such as Figure 3 As shown, the finger touch area can be simulated as an ellipse, with the coordinates of the ellipse's center (O'), major and minor axes, and the ellipse's direction used as touch information. The ellipse's direction can refer to the extension direction of the major axis, which can be specifically represented by the angle between the major axis and a side of the screen (e.g., the side XY). The ellipse's major axis has two vertices, one near the center of the screen (e.g., C). L C R ) and a vertex far from the center of the screen (such as F) L F R Of course, when implementing the embodiments of this application, the direction of the extension of the minor axis can also be regarded as the direction of the ellipse. However, knowing only the center coordinates and major and minor axes of the ellipse is not enough to uniquely determine the finger touch area, so it is necessary to introduce the direction of the ellipse.

[0073] Not limited to Figure 3 As shown, the finger touch area 211A of the left finger and the finger touch area 211B of the right finger do not necessarily have to appear together.

[0074] Finger occlusion area recognition

[0075] When the fingertip or pad of a finger directly touches the screen, making the area touched by the finger invisible to the user, other parts of the finger will also hover close to the screen, resulting in poor visibility of the screen area below it. This screen area will be referred to as the finger-occluded area in this article, and it does not include the aforementioned finger-touching area, in order to distinguish it from the finger-touching area.

[0076] Based on the finger-touching area, at least one elliptical region can be extended outwards to construct the finger-occluded area. The area remaining after subtracting the largest nested elliptical region from the extended elliptical region is the finger-occluded area. The extended elliptical region shares the elliptical direction and the major axis vertex near the center of the screen with the finger-touching area. This also means that the center of the extended elliptical region and the center of the finger-touching area are on the same straight line.

[0077] Specifically, such as Figure 4 As shown, based on the finger-touching area 211A of the left finger, elliptical areas S1 and S2 can be extended to form the occlusion area of ​​the left finger. "Based on the finger-touching area 211A" can mean that the extended elliptical areas S1 and S2 share the same elliptical direction and the same major axis vertex C as the finger-touching area 211A. L Vertex C of the major axis L It is the vertex of the long axis closest to the center of the screen, which reflects the projection position of the left fingertip on the screen.

[0078] Specifically, such as Figure 4 As shown, based on the finger-touching area 211B of the right finger, elliptical areas S1 and S2 can be extended to form the occlusion area of ​​the right finger. The extended elliptical areas S1 and S2 share the same elliptical direction and the major axis vertex C with the finger-touching area 211B. R Vertex C of the major axis R It is the vertex of the long axis closest to the center of the screen, which reflects the projection position of the right fingertip on the screen.

[0079] Due to the gradually rising three-dimensional structure of the fingertip, the vertical distance from each point on the fingertip to the screen is not uniform, with points or areas closer to the screen causing more severe occlusion. Therefore, the multiple elliptical regions that contribute to forming the left or right finger occlusion area can be nested, such as S1 nested within S2, gradually increasing in size to construct multi-level finger occlusion areas with varying degrees of occlusion. The area remaining after removing the largest nested elliptical region from an elliptical region forms the first-level finger occlusion area.

[0080] For example, the area remaining after removing the largest nested elliptical region (i.e., the finger touch area 211A(211B)) from the extended ellipse S1 forms a first-level occlusion area; the area remaining after removing the largest nested elliptical region (i.e., the elliptical region S1) from the extended ellipse S2 forms another level of occlusion area.

[0081] The closer the finger is to the center of the touch area, the less visible the area obscured by the finger. The touch center is the center of the area touched by the finger. For example, because it is closer to touch area 212A, Figure 4The visibility of the finger-occupied area 212A shown is worse than that of the finger-occupied area 213A. For example, because it is closer to the touch area 212B, Figure 4 The visibility of the finger-occluded area 212B shown is inferior to that of the finger-occluded area 213B.

[0082] Specifically, the finger-touched areas 211A (211B) can be considered as the first-level occlusion areas with the most severe occlusion.

[0083] based on Figure 4 The invisibility of regions 211A (211B), 212A (212B), and 213A (213B) shown in the figure gradually decreases, and they can be marked as first-level occlusion region, second-level occlusion region, and third-level occlusion region, respectively. Differentiated rendering can be applied to each level of occlusion region in the future.

[0084] The size of the outermost elliptical region used to construct the finger-occupying area can be constrained by the finger width W and the screen boundary. Specifically, the sum of the two minor axes of the outermost elliptical region can be equal to or approximately equal to the finger width W, and this elliptical region does not extend beyond the screen boundary. Here, "approximately" can mean smaller than the finger width W, but the difference is small, for example, within 2 millimeters. In particular, when only one elliptical region is extended to construct the finger-occupying area, this one elliptical region is the outermost elliptical region, and the sum of its two minor axes can be equal to or approximately equal to the finger width W. The finger width W can be the statistical average of the width of a human thumb. The finger width W can also be estimated based on the width of the finger touch area (such as the sum of the two minor axes of region 211A), and these two widths are directly proportional. This is because, generally, the wider the touch area formed by a finger contacting the screen, the wider the finger is to some extent. To improve the accuracy of the finger width W estimation, parameters such as user gender, age, and weight can be further considered. These parameters can be extracted from the personal information of the logged-in user of the electronic device 100. This application does not limit the method of obtaining these parameters or how to use these parameters to help estimate finger width.

[0085] Palm occlusion area recognition

[0086] When a finger touches the screen directly, making the area touched by the finger invisible to the user, in addition to the other parts of the finger being suspended close to the touch screen, the visibility of the screen area below it will be poor. The base of the thumb and the part of the palm connected to it may also block the lower left and lower right corners of the screen, which may result in poor visibility of the lower left and lower right corners of the game screen.

[0087] In this article, the screen area covered by the base of the thumb and the connected palm portion can be referred to as the palm-covered area. Here, the "palm" in "palm-covered area" is not limited to the palm as strictly defined in the field of human anatomy; this article only uses this terminology to distinguish it from the aforementioned finger-covered area.

[0088] Based on the finger-touched area, a rectangular area R or a triangular area T can be extended to construct the palm-occluded area. The palm-occluded area can be any area within the rectangular area R (or triangular area T) that is not covered by the finger-touched area or the finger-occluded area.

[0089] Specifically, such as Figure 5 As shown, based on the finger-touching area 211A of the left finger, a rectangular area R1 or a triangular area T1 can be extended. "Based on the finger-touching area 211A" can mean that the diagonal A of the rectangular area R1 originates from the center O of area 211A. L The line segment between the top left corner of the screen and the bottom left corner is formed by the line segment, or the hypotenuse of the triangular region T1 is formed by the line segment.

[0090] Specifically, such as Figure 5 As shown, based on the finger-touching area 211B of the right finger, a rectangular area R2 or a triangular area T2 can be extended. "Based on the finger-touching area 211B" can mean that the diagonal B of the rectangular area R2 originates from the center O of area 211B. R The line segment between the top right corner of the screen and the bottom right corner is formed by the line segment, or the hypotenuse of the triangular region T2 is formed by this line segment.

[0091] The rectangular regions R1 and R2 may not be strictly rectangular in the mathematical sense, but rather approximate rectangles. The sides a1 and b1, and sides a2 and b2 of the rectangles can be modeled or simulated as curved, taking into account the natural curvature of the base of the human thumb and its connection to the palm. Similarly, the triangular regions T1 and T2 may not be strictly triangular in the mathematical sense, but rather approximate triangles.

[0092] The palm-covered area can be the area within a rectangular region R (or a triangular region T) that is not covered by the area touched by the fingers or the area covered by the fingers.

[0093] like Figure 6 As shown, the palm-covered area 215A of the left palm is the area within the rectangular area R1 (or triangular area T1) that is not covered by the finger-touched areas 211A, 212A, and 213A of the left fingers. Similarly, the palm-covered area 215B of the right palm is the area within the rectangular area R2 (or triangular area T2) that is not covered by the finger-touched areas 211B, 212B, and 213B of the right fingers.

[0094] It can be seen that the center O L The closer to the bottom left corner of the screen, the smaller the area 215A obscured by the left palm; the center O R The closer to the bottom right corner of the screen, the smaller the area 215B that the right palm will cover.

[0095] When the distance from the center of the touch area to the bottom left or right corner of the screen is small enough, the area obstructed by the left or right palm can be disregarded. This indicates that the left or right fingers have not extended sufficiently towards the center of the screen, thus the left or right palm has not yet obstructed the screen. In specific implementation, when the center O L When the distance to the bottom left corner of the screen is less than a certain distance, the area 215A obstructed by the left palm can be disregarded. Similarly, when the center O... R When the distance to the bottom right corner of the screen is less than a certain distance, the area 215B obstructed by the right palm can be disregarded. This specific distance can be a statistical average of the projection length of a human thumb on the screen when it is naturally bent or straight.

[0096] The degree of occlusion in the palm-occluded area is lower than that in the finger-touched area, allowing for differentiated rendering of different areas. This application does not limit the degree of occlusion between the palm-occluded area and the finger-occluded area; it can be determined based on actual application needs.

[0097] Differentiated rendering

[0098] After classifying and identifying the aforementioned occluded areas, the load-reducing rendering method provided in this application embodiment can be further adapted to differentiated rendering reduction methods, reducing GPU rendering load without affecting the user's gaming experience.

[0099] De-rendering techniques include, but are not limited to: reducing resolution, reducing image texture, feature point clipping, and rendering culling. Reducing resolution decreases image sharpness, reducing image texture reduces image detail, feature point clipping reduces image feature points, and rendering culling avoids rendering culled areas. Additionally, de-rendering techniques can include variable rate shading (VRS) for areas with varying degrees of occlusion. VRS can shade images in groups of two or four pixels during rendering and reuse the shading results for surrounding pixels, reducing the GPU's computational load by selectively rendering and shading the image.

[0100] In this embodiment, differentiated rendering reduction refers to applying a lower rendering load to the occluded area closer to the touch center, where the touch center is the center of the finger-touched area. This allows for greater rendering reduction to be applied to occluded areas with higher degrees of occlusion. Rendering culling can be considered as maximizing rendering reduction, resulting in no content display, and is suitable for the finger-touched area. Specifically, rendering culling can be performed on the image in the finger-touched area, and one or more of the following processes can be applied to the image in the finger-occluded area: reducing image texture, reducing image resolution, reducing feature points, and reducing shading rate.

[0101] For example, such as Figure 7 As shown, the occlusion levels of the finger-touched area 211A, finger-occluded area 212A, and finger-occluded area 213A decrease progressively. It is also assumed that the occlusion level of the palm-occluded area 215A is lower than that of the finger-occluded area 213A. Therefore, as... Figure 7 As shown, a rendering reduction method with gradually decreasing degree can be implemented for the finger touch area 211A, finger occlusion area 212A, finger occlusion area 213A, and palm occlusion area 215A: rendering culling is implemented for the finger touch area 211A; VRS is implemented for the finger occlusion area 212A, finger occlusion area 213A, and palm occlusion area 215A, with 4×4 shading rate, 4×2 shading rate, and 2×2 shading rate respectively.

[0102] This differentiated rendering reduces the GPU rendering load while providing more refined rendering effects for different areas with varying degrees of occlusion, without affecting the user's gaming experience.

[0103] The load-reducing rendering method provided in this application embodiment can achieve seamless development of game applications, and is widely applicable to various game applications without requiring game application developers to change functionality. This is thanks to the software system architecture of the electronic device 100 provided in this application embodiment, which designs a path between touch information (carried in input events) and the image processing module.

[0104] Figure 8 An example of the software system architecture of electronic device 100 is shown.

[0105] like Figure 8 As shown, the software system architecture of electronic device 100 may include an application (APP) layer, an application framework (FWK) layer, and a kernel layer. The following sections elaborate on each part:

[0106] The application layer can include one or more applications, such as game applications. Game applications can provide game visual content.

[0107] The application framework layer may include: Input Management Service (IMS), Window Management Service (WMS), UI framework layer, and image processing module.

[0108] IMS includes an Event Hub, an InputReader, and an InputDispatcher. These components work together to transmit and distribute input events reported by the touchscreen to the appropriate window, such as window1 in a game application. The Event Hub listens to device nodes (e.g., / dev / input / eventX) via iNotify and Epoll mechanisms and reads the raw input events from these nodes. The InputReader continuously reads the raw input events from the Event Hub, processes them, and then passes them to the InputDispatcher. The InputDispatcher stores information about all windows in WMS (WMS updates this information in real-time), allowing it to dispatch input events to the appropriate window.

[0109] WMS can be used to manage all windows in the system, such as window1. window1 can be the window of a game application, which can be created (instantiated) when the game application is launched. Along with the instantiation of window1, a ViewRootImpl object can be instantiated in the UI framework layer. The ViewRootImpl object can be used to listen for input events dispatched to window1 by the InputDispatcher. This embodiment improves upon ViewRootImpl; the improved ViewRootImpl object, after listening to the input event dispatched to window1 by the InputDispatcher, can also pass the touch information carried in the input event to the image processing module. Furthermore, since the input event is listened to by the ViewRootImpl object, the game application also receives the input event and can determine whether a user's finger is touching the screen.

[0110] After receiving the touch information from the ViewRootImpl object, the image processing module can be used to intercept rendering commands, identify occlusion areas at various levels, and reduce rendering load. For details on the specific implementation of identifying occlusion areas at various levels and reducing rendering load, please refer to the preceding and following sections; these details will not be elaborated here.

[0111] In addition to IMS, WMS, and image processing modules, the application framework layer can also include a layer compositor (SurfaceFlinger). SurfaceFlinger can be used to perform layer compositing on the screen frames rendered by the GPU, generate an interface that includes the screen frames, and send the interface that includes the screen frames to the display screen for display.

[0112] The kernel layer can include touchscreen drivers and GPU drivers. The touchscreen driver controls the touchscreen's operation and receives touch signals reported by the touchscreen, while the GPU driver controls the GPU to perform rendering operations. When a user's finger touches the touchscreen, the touchscreen can simulate the area pressed by the finger as an elliptical region and report elliptical information to the touchscreen driver, such as the coordinates of the ellipse's center (O'), major and minor axes, and direction. This elliptical information is carried as touch information in the input event reported to the touchscreen driver.

[0113] In addition to the layers mentioned above, the system architecture of the electronic device 100 may also include a local service layer. The local service layer may include one or more graphics libraries, such as an open graphics library (OpenGL) or an open graphics library for embedded systems (OpenGL ES). The graphics drawing interfaces included in the graphics library can be called to configure the GPU to render screen frames.

[0114] based on Figure 8 The exemplary software system architecture is shown below. Figure 9 This illustrates the interoperability between various software modules, particularly the rendering process of the image processing module, which intercepts rendering commands, identifies occlusion regions at various levels, and reduces rendering load. The game application may include a main thread and a rendering thread. The image processing module may include a rendering command interception module, an occlusion region identification module, and a load-reducing rendering module.

[0115] The interaction flow between the modules can be as follows:

[0116] 1. The main thread of a game application can synchronize the game screen content to the rendering thread.

[0117] 2. After the rendering thread obtains the game screen content, it can call the first graphics drawing interface in the graphics library (such as OpenGL, OpenGL ES, etc.) to send rendering instructions.

[0118] Meanwhile, the game screen content to be rendered by the game application can be cached, for example, temporarily stored in cache 1, for later reading during GPU rendering. Cache 1 can be a portion of the cache area on the memory 120 in the electronic device 100.

[0119] 3. The first graphics drawing interface in the graphics library (such as OpenGL) is called to forward rendering instructions to the rendering instruction interception module.

[0120] Calling the first graphics drawing interface in the graphics library does not directly pass the rendering instructions from the rendering thread to the GPU. Instead, it forwards the rendering instructions to the rendering instruction interception module in the image processing module. This is the rendering instruction interception mentioned in this article.

[0121] Specifically, rendering command interception can occur during the period when the image processing module receives touch information from the ViewRootImpl object, during which time a user's finger touches the screen. That is, during the period when a user's finger touches the screen, the game application can call the first graphics drawing interface in the graphics library. Alternatively, when there is no user finger touching the screen, the game application can call the second graphics drawing interface in the graphics library to directly pass rendering commands to the GPU. This is the normal rendering mentioned in this article, as opposed to this rendering method that reduces the rendering load.

[0122] In the graphics library, the first graphics drawing interface is a graphics drawing interface introduced in this application embodiment to achieve rendering instruction interception, and the second graphics drawing interface is a traditional graphics drawing interface provided by the graphics library.

[0123] 4. After receiving the rendering instruction, the rendering instruction interception module can further pass the rendering instruction to the occlusion area recognition modules at each level.

[0124] The occlusion region recognition module can identify occlusion regions at various levels based on touch information, such as the coordinates of the ellipse's center (O'), major and minor axes, and the ellipse's direction: finger touch area, finger occlusion area, and palm occlusion area, laying the foundation for subsequent differentiated rendering. For details on how to identify occlusion regions at various levels, please refer to the relevant content above or below; it will not be repeated here.

[0125] Touch information can be passed from the ViewRootImpl object to the image processing module; see the details below. Figure 8 The relevant content in the embodiments will not be repeated here.

[0126] 5. After identifying the occlusion areas at each level, the occlusion area identification module at each level can inform the load-reducing rendering module of the identification results (i.e., the occlusion areas at each level).

[0127] Here, the recognition results can include indication information of various levels of occlusion areas (such as finger touch area, finger occlusion area, palm occlusion area), such as position, size, and shape, which indicate which pixels on the screen are covered by each level of occlusion area.

[0128] 6. After identifying the occlusion areas at each level, the load-reducing rendering module can adapt differentiated rendering reduction methods to each occlusion area, obtain a differentiated rendering reduction scheme, and send rendering instructions to the GPU. These rendering instructions can carry the differentiated rendering reduction scheme.

[0129] Differentiated down-rendering schemes can be used to apply a higher degree of down-rendering to occluded areas with a high degree of occlusion, and a lower degree of down-rendering to occluded areas with a low degree of occlusion. For example, for Figure 4 Region 211A is rendered with reduced rendering by culling, while regions 212A, 213A, and 215A are rendered with reduced rendering at variable color rates of 4×4, 4×2, and 2×2, respectively. For details on how to achieve differentiated reduced rendering, please refer to the relevant content above or below; it will not be repeated here.

[0130] 7. After receiving the rendering instruction from the load-reducing rendering module, the GPU can read the game screen content to be rendered from cache 1, render the game screen content according to the differentiated load-reducing rendering scheme, and finally store the rendered screen frames in cache 2.

[0131] Cache 2 can be a cache area specified by a rendering instruction, or it can be a portion of the cache area on the memory 120 in the electronic device 100.

[0132] 8. The SurfaceFlinger retrieves the rendered frames from buffer 2 and performs layer compositing. Then, SurfaceFlinger sends the composited frames into the frame buffer for display.

[0133] A frame buffer is a storage space that can be located in video memory or main memory. It stores rendering data that has been processed or is about to be retrieved by the graphics card chip. The contents of the frame buffer correspond to the interface displayed on the screen (194Hz). It can be simply understood as a cache corresponding to the content displayed on the screen (194Hz). Modifying the contents of the frame buffer is equivalent to modifying the content displayed on the screen (194Hz).

[0134] Specifically, after receiving a vertical sync (vsync) signal, SurfaceFlinger retrieves a frame from cache 2, composites the layers, and sends it to the display for refresh. The vsync signal is a synchronization signal generated by the hardware module corresponding to the display, with a period consistent with the screen refresh rate. The screen refresh rate indicates the number of times the display on an electronic device refreshes the displayed frame per second. Generally, the refresh rate of an electronic device's display is related to fixed parameters of the display and is a fixed value. For example, the refresh rate can be 60 Hz.

[0135] The software system architecture provided in this application provides a pathway between touch information (carried in input events) and the image processing module, and can also achieve low latency, so that there is no difference in user experience between load-reduced rendering and normal rendering.

[0136] Generally, games use triple buffering, requiring 3 frames to render to the display, with 2 frames buffered. For a 60fps game (with a frame interval of approximately 16.7 milliseconds), triple buffering introduces a latency of about 50 milliseconds. Taking an 80ms latency between user touch input and screen display as an example, this requires the latency of touch information transmission to the image processing module, plus the processing latency of the image processing module executing rendering command interception, various occlusion area identification, and outputting differentiated load-reducing rendering schemes, to not exceed 30 milliseconds in total. The fast path between touch information and the image processing module provided in this application's embodiments meets this requirement. This application's embodiments can achieve a touch information transmission time of less than 25 milliseconds to the image processing module, and the image processing module's processing time not exceeding 5 milliseconds. However, for low frame rate games, taking a 30fps game as an example, since the frame interval is already 33 milliseconds, rendering to the display under triple buffering takes about 100 milliseconds, resulting in significant latency. Considering user experience, electronic devices can choose not to perform load-reducing rendering and only perform normal rendering.

[0137] Figure 10 The overall flow of the load-reducing rendering method provided in the embodiments of this application is illustrated. Further details are provided below.

[0138] S11. Game begins.

[0139] Electronic devices can launch game applications due to user actions (such as clicking the desktop icon of a game application), and need to render the game screen provided by the game application and display the game screen provided by the game application.

[0140] As mentioned earlier, when a user holds (e.g., in landscape mode) an electronic device to play games, certain areas of the game screen are obscured by the user's thumb or other hand parts. Generally, the lower left corner of the screen is designed as the control wheel area, and the lower right corner as the skill area. Users need to operate these areas, thus obstructing the screen and making these parts of the game screen invisible or poorly visible. For example, the pad of the thumb touches and presses on the touchscreen, making the area it's pressing invisible to the user. Simultaneously, the part of the thumb other than the pad hovering near the touchscreen further reduces the visibility of the game screen area below it. Additionally, the base of the thumb and the connected part of the palm often touch or obscure the lower left and lower right corners of the screen, making these areas invisible or poorly visible to the user.

[0141] The load-reducing rendering method provided in this application embodiment accurately identifies these occluded areas in a hierarchical manner, laying the foundation for subsequent differentiated rendering. Not limited to game applications, this application embodiment can also be applied to other applications that provide a graphical user interface and receive user touch input, thereby reducing the rendering load of the graphical user interface when the user touches the screen.

[0142] S12. The electronic device can determine whether a touch operation is detected on the touch screen. If a touch operation is detected, S13-S15 can be executed; otherwise, S16 can be executed. That is, while the user's finger is touching the screen, the electronic device performs load-reducing rendering; while the user's finger is not touching the screen, the electronic device performs normal rendering.

[0143] The game screen may also display a directional wheel and skill buttons. The directional wheel is used to control the movement of characters or other entities in the game, while the skill buttons are used to execute game skills. The directional wheel and skill buttons can be located in the lower left and lower right corners of the screen, or in the lower right and lower left corners, respectively.

[0144] The touch operation can specifically be an operation performed on the steering wheel or skill button. That is, the electronic device can trigger the execution of S13-S15 when it detects a touch operation performed on the steering wheel or skill button, or when it detects a touch operation in the lower right or lower left corner of the screen.

[0145] In addition, before determining whether to execute S13-S15 or S16, the electronic device can also determine whether the game application is a low frame rate game application, such as a 30 frame rate game. If it is not a low frame rate game application, then it will execute S13-S15; otherwise, it will execute S16.

[0146] In this article, the latency requirement between the user's touch operation and the display of the game screen (maximum latency T) can be used as the basis. maxTo determine whether a game application is a low frame rate game application, specifically, if a game application's triple buffering latency has reached or exceeded T... max This can be considered a low frame rate game application. Or, more strictly, if a game application's latency for triple buffering exceeds T... max -T reserve Then it can be considered a low frame rate game application. Among them, T reserve This represents the latency reserved for transmitting touch information to the image processing module and the processing latency of the image processing module, for example, 30 milliseconds. Its length is affected by the processing performance of the relevant hardware and can be adjusted according to the actual implementation scenario. The latency of the triple buffer executed by the game application is determined by the frame rate of the game application, R, which is equal to 3*(1000 / R) milliseconds.

[0147] S13. Electronic devices can acquire touch information.

[0148] Touch information can include, for example, the location, shape, and size of the area touched by a finger. This information can be carried in touch events. Based on this touch information, the electronic device can recognize the area touched by the finger. Specifically, for example... Figure 3 As shown, the finger touch area can be simulated as an ellipse, and the coordinates of the ellipse's center (O'), major and minor axes, and the direction of the ellipse are used as touch information. The direction of the ellipse can refer to the extension direction of the major axis, which can be specifically represented by the angle between the major axis and a side of the screen (such as the XY side).

[0149] S14. Electronic devices can identify occlusion areas at various levels based on touch information.

[0150] The occlusion areas at each level can include: the finger touch area, the finger occlusion area, and the palm occlusion area.

[0151] The recognition of the finger touch area can be achieved based on touch events reported by the screen (specifically, a touchscreen). The finger touch area is the screen area directly touched by the finger; the game screen displayed within this area is not visible to the user, exhibiting the highest degree of obstruction and the worst visibility. The finger touch area can be simulated as an ellipse.

[0152] There can be N (N is a positive integer) finger-occluded regions, which can be constructed from N elliptical regions extending outward from the finger-touching region. The extended elliptical regions have the same elliptical direction as the finger-touching region, and their major axis vertices near the screen center coincide. A finger-occluded region is the area remaining after removing the largest nested elliptical region from the extended elliptical region.

[0153] When N=1, the largest elliptical region within this expanded elliptical region is the area touched by the finger. For example, Figure 4The finger-occluded area 212A is the area left after the finger-touched area (i.e., the largest nested elliptical area within the elliptical area S1) is removed from the expanded elliptical area S1.

[0154] When N ≥ 2, the N elliptical regions that expand to form the left or right finger occlusion region can be nested, gradually increasing in size to construct multi-level finger occlusion regions with varying degrees of occlusion. The area remaining after removing the largest nested elliptical region from an elliptical region forms the first-level finger occlusion region. For example, Figure 4 The finger-occluded area 212A is the area left after the finger-touching area is removed from the expanded elliptical area S1, and the finger-occluded area 213A is the area left after the elliptical area S1 (i.e., the largest nested elliptical area within the elliptical area S2) is removed from the expanded elliptical area S2.

[0155] The closer to the center of the touch, the less visible the area obscured by the finger. The center of the touch is the center of the area touched by the finger.

[0156] Specifically, such as Figure 4 As shown, the elliptical regions S1 and S2, extended from the left finger's touch area 211A, after removing their respective nested largest ellipses, leave regions 212A and 213A, which represent the two levels of left finger occlusion regions with different degrees of occlusion. The extended elliptical regions S1 and S2 share the same elliptical direction and the same major axis vertex C as the touch area 211A. L .

[0157] Specifically, such as Figure 4 As shown, the elliptical regions S1 and S2, extended from the right finger's touch area 211B, after removing their respective nested largest ellipses, leave regions 212B and 213B, which represent the two levels of right finger occlusion regions with different degrees of occlusion. The extended elliptical regions S1 and S2 share the same elliptical direction and the same major axis vertex C as the touch area 211B. R .

[0158] The size of the outermost elliptical region used to construct the finger-occluding area can be constrained by the finger width W and the screen boundary. Specifically, the sum of the two minor axes of the outermost elliptical region can be equal to or approximately equal to the finger width W, and this elliptical region does not extend beyond the screen boundary. In particular, when only one elliptical region is extended to construct the finger-occluding area, this single elliptical region is the outermost elliptical region, and the sum of its two minor axes can be equal to or approximately equal to the finger width W.

[0159] The palm-occluded area can be constructed from a rectangular area R or a triangular area T extended from the finger-touching area. The palm-occluded area can be the area within the rectangular area R (or triangular area T) that is not covered by the finger-touching area or the finger-occluded area.

[0160] Specifically, such as Figure 5 As shown, in the rectangular region R1 or triangular region T1 extended from the left finger's touch area 211A, the area not covered by the left finger's touch area 211A, the left finger's occlusion areas 212A, and 213A is the left palm's occlusion area 215A. "Based on the finger touch area 211A" can mean that the diagonal A of the rectangular region R1 originates from the center O of region 211A. L The line segment between the top left corner of the screen and the bottom left corner is formed by the line segment, or the hypotenuse of the triangular region T1 is formed by the line segment.

[0161] Specifically, such as Figure 5 As shown, in the rectangular region R2 or triangular region T2 extended from the right finger's touch area 211B, the area not covered by the left finger's touch area 211A, the left finger's occlusion area 212A, and 213A is the right palm's occlusion area 215B. "Based on the finger touch area 211B" can mean that the diagonal B of the rectangular region R2 originates from the center O of region 211B. R The line segment between the top right corner of the screen and the bottom right corner is formed by the line segment, or the hypotenuse of the triangular region T2 is formed by this line segment.

[0162] When the distance from the center of the touch area to the bottom left or right corner of the screen is small enough that the area covered by the left or right palm is not considered, the left or right fingers do not extend sufficiently towards the center of the screen, thus the left or right palm does not yet cover the screen.

[0163] S15. Electronic devices can render images in reduced load for areas with varying levels of obstruction.

[0164] Specifically, electronic devices can employ less load rendering for the occluded area closer to the touch center. Less load rendering techniques may include, but are not limited to, one or more of the following: rendering culling, reducing image texture, reducing image resolution, reducing feature points, and reducing shading rate.

[0165] For example, electronic devices can reduce resolution, image texture, and crop more feature points in the occluded area closer to the touch center.

[0166] Render culling can be considered as maximally reducing rendering load, resulting in no content being displayed. Specifically, electronic devices can apply this load-reducing rendering method of rendering culling to the finger-touched area, performing one or more of the following processing on the image in the finger-occluded or palm-occluded areas: reducing resolution, reducing image texture, and cropping feature points. The closer the finger-occluded area is to the center of the touch, the greater the reduction in resolution, reduction in image texture, and cropping of more feature points can be achieved.

[0167] In addition, load reduction rendering techniques can also include applying VRS (Dynamic Color Scheme) to areas with different degrees of occlusion. For example, finger-occluded areas with different degrees of occlusion can use different shading rates, with lower shading rates applied to finger-occluded areas closer to the touch center and higher shading rates applied to finger-occluded areas further out. If the occlusion level of the palm occluded area is considered to be higher than that of the finger occluded area, then the shading rate used for the palm occluded area can be lower than that used for the finger occluded area.

[0168] Thus, users can observe that after load reduction rendering is implemented, the images in occluded areas at various levels are more blurry compared to other screen areas that have not undergone load reduction rendering. Moreover, the images are even blurrier in areas closer to the touch center. This is because these occluded areas have undergone load reduction rendering, and their images are inferior to other screen areas that have not undergone load reduction rendering in one or more aspects such as image texture, resolution, number of feature points, and shading rate.

[0169] S16. Electronic devices can perform normal rendering.

[0170] For standard rendering, please refer to the relevant descriptions in the previous examples; they will not be repeated here.

[0171] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0172] This application also provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0173] This application also provides a chip system, which includes a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0174] The term "user interface (UI)," or simply "interface," used in the specification and accompanying drawings of this application, refers to the medium through which an application or operating system interacts and exchanges information with the user. It facilitates the conversion between the internal form of information and a form acceptable to the user. The user interface of an application is written in source code using specific computer languages ​​such as Java or Extensible Markup Language (XML). This source code is parsed and rendered on the terminal device, ultimately presenting user-recognizable content, such as images, text, and buttons. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, images, and text. The attributes and content of controls in the interface are defined using tags or nodes, such as XML tags. <textview> 、 <imgview> 、 <videoview>Nodes define the controls contained in the interface. A node corresponds to a control or property in the interface, and after parsing and rendering, the node is presented as the content visible to the user. In addition, many applications, such as hybrid applications, often contain web pages within their interfaces. A web page, also known as a webpage, can be understood as a special control embedded in the application interface. Web pages are source code written in a specific computer language, such as Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), JavaScript (JS), etc. Web page source code can be loaded and displayed as user-readable content by a browser or a web page display component with browser-like functionality. The specific content contained in a webpage is also defined through tags or nodes in the webpage source code; for example, HTML uses tags or nodes to define the content. 、 、 <video> 、 <canvas>Used to define the elements and attributes of a webpage.

[0175] The most common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0176] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0178] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A method of rendering with reduced load, characterized by, The method comprises the following steps: An electronic device displays a picture provided by a first application on a screen; The electronic device detects a touch operation of a user's finger on the screen and acquires touch information of the touch operation; The electronic device determines a multi-level shielding area based on the touch information, wherein the multi-level shielding area comprises a finger touch area and a finger shielding area, and the finger shielding area is wrapped around the periphery of the finger touch area; The electronic device performs load-reduced rendering on the picture of the multi-level shielding area, wherein the picture of the shielding area closer to the touch center from the inside is rendered with a lower load, the touch center is the center of the finger touch area, and the load-reduced rendering comprises one or more of the following: rendering rejection, reduced image texture, reduced image resolution, reduced feature points, and reduced shading rate.

2. The method of claim 1, wherein, The picture of the shielding area closer to the touch center from the inside is rendered with a lower load, which comprises the following steps: performing rendering rejection on the picture of the finger touch area, and performing one or more of the following processes on the picture of the finger shielding area: reducing image texture, reducing image resolution, reducing feature points, and reducing shading rate.

3. The method of claim 2, wherein, The first application comprises a game application, and the picture further displays a direction wheel and a skill key, the touch operation is performed on the direction wheel or the skill key, and the direction wheel and the skill key are respectively distributed at the lower left corner and the lower right corner of the screen or are respectively distributed at the lower right corner and the lower left corner of the screen.

4. The method of claim 3, wherein, The finger touch area is an elliptical area, and the touch information comprises the center coordinates, the long and short axes, and the elliptical direction of the elliptical area, wherein the elliptical direction is the extension direction of the long axis.

5. The method of claim 4, wherein, The finger shielding area comprises N, wherein N is a positive integer, the N finger shielding areas are constructed based on N extended elliptical areas of the finger touch area, one finger shielding area is the area left after the largest elliptical area nested in one extended elliptical area is removed, the extended elliptical area and the finger touch area have the same elliptical direction, and the long axis vertex close to the center of the screen.

6. The method of claim 5, wherein, When N is equal to 1, the largest elliptical area nested in the one extended elliptical area is the finger touch area.

7. The method of claim 5, wherein, When N is greater than or equal to 2, the N extended elliptical areas are nested layer by layer and gradually become larger.

8. The method of claim 6, wherein, When N is greater than or equal to 2, the N extended elliptical areas are nested layer by layer and gradually become larger.

9. The method of claim 7, wherein, The sum of the two short axes of the outermost extended elliptical area is equal to the width W of a first finger, or the difference between the sum and the width W of the first finger is less than a first value; the first finger comprises a thumb.

10. The method of claim 8, wherein, The sum of the two short axes of the outermost extended elliptical area is equal to the width W of a first finger, or the difference between the sum and the width W of the first finger is less than a first value; the first finger comprises a thumb.

11. The method of claim 7, wherein, The N finger shielding areas adopt different shading rates, wherein the game picture of the finger shielding area closer to the touch center from the inside adopts a lower shading rate.

12. The method of claim 8, wherein, The N finger blocking areas adopt different shading rates, wherein the game picture of the finger blocking area closer to the touch center adopts a lower shading rate.

13. The method of claim 9, wherein, The N finger blocking areas adopt different shading rates, wherein the game picture of the finger blocking area closer to the touch center adopts a lower shading rate.

14. The method of claim 10, wherein, The N finger blocking areas adopt different shading rates, wherein the game picture of the finger blocking area closer to the touch center adopts a lower shading rate.

15. The method of any one of claims 1-14, wherein, The multi-level blocking area further comprises a left palm blocking area. The left palm blocking area is a region in a rectangular region or a triangular region at the lower left corner of the screen which is not covered by the finger touch region of the left finger and the finger blocking region of the left finger; wherein a diagonal line of the rectangular region at the lower left corner is a line segment from the center of the finger touch region of the left finger to the lower left corner of the screen, and an oblique side of the triangular region is the line segment.

16. The method of any one of claims 1-14, wherein, The multi-level blocking area further comprises a right palm blocking area. The right palm blocking area is a region in a rectangular region or a triangular region at the lower right corner of the screen which is not covered by the finger touch region of the right finger and the finger blocking region of the right finger; wherein a diagonal line of the rectangular region at the lower right corner is a line segment from the center of the finger touch region of the right finger to the lower right corner of the screen, and an oblique side of the triangular region is the line segment.

17. The method of claim 15, wherein, The multi-level blocking area further comprises a right palm blocking area. The right palm blocking area is a region in a rectangular region or a triangular region at the lower right corner of the screen which is not covered by the finger touch region of the right finger and the finger blocking region of the right finger; wherein a diagonal line of the rectangular region at the lower right corner is a line segment from the center of the finger touch region of the right finger to the lower right corner of the screen, and an oblique side of the triangular region is the line segment.

18. An electronic device, comprising: The device comprises one or more processors and one or more memories; wherein the one or more memories are coupled with the one or more processors, and the one or more memories are configured to store computer program codes, the computer program codes comprising computer instructions, when the one or more processors execute the computer instructions, the method of any one of claims 1-17 is performed.

19. A chip system applied to an electronic device, the chip system comprising one or more processors, characterized in that, The processor is configured to invoke the computer instructions to perform the method of any one of claims 1-17.

20. A computer-readable storage medium comprising instructions, wherein: When the instructions are run on the electronic device, the method of any one of claims 1-17 is performed.

Images