A scene recognition method and electronic device based on rendering data

Abstract

This application provides a scene recognition method and electronic device based on rendering data. Before performing a rendering operation on a first image frame, the method acquires real-time rendering data of the first image frame. Based on the real-time rendering data, it identifies at least one of the following: the image scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene lighting complexity. Based on at least one of the following, it determines an optimization strategy for the first image frame. Based on the optimization strategy, it performs a rendering operation on the first image frame. In this way, the scene that can affect the optimization strategy can be identified in advance before performing the rendering operation on the image. Subsequent optimization algorithms can then use the most suitable optimization strategy based on the pre-identified scene conditions to perform the rendering operation on the first image frame, achieving precise rendering resource supply and improving the user experience.

Description

Technical Field

[0001] This application belongs to the field of terminal technology, and in particular relates to a scene recognition method and electronic device based on rendering data. Background Technology

[0002] To meet users' growing demands for enhanced gaming experiences, game applications on mobile devices are increasingly featuring richer rendering scenes and higher rendering quality. Correspondingly, the rendering power consumption of these devices is also increasing, requiring them to support higher-demand rendering resource provision.

[0003] However, current mobile phones and other electronic devices cannot accurately provide rendering resources for game applications, resulting in a poor user gaming experience. Summary of the Invention

[0004] This application provides a scene recognition method and electronic device based on rendering data, which can achieve accurate rendering resource supply and improve user experience.

[0005] In a first aspect, this application provides a scene recognition method based on rendering data, the method being applied to an electronic device, the method comprising: acquiring real-time rendering data of the first image frame before performing a rendering operation on the first image frame; identifying at least one of the following based on the real-time rendering data: the screen scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene light complexity; determining an optimization strategy for the first image frame based on at least one of the following: the screen scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene light complexity; the optimization strategy including at least one of the following: a system scheduling strategy, a frame interpolation strategy, an image super-resolution strategy, an image post-processing strategy, and a screen adjustment strategy; and performing a rendering operation on the first image frame based on the optimization strategy.

[0006] In this way, before performing rendering operations on the image, scene scenarios that can affect the optimization strategy can be identified in advance through rendering data. For example, special cases such as sudden zooming in, game scene changes, vehicle types used by game characters, and high scene lighting complexity can be identified. Subsequent optimization algorithms can then use the most appropriate optimization strategy based on the pre-identified scene conditions, performing rendering operations on the first image frame to achieve precise rendering resource allocation and improve the user experience.

[0007] In one possible implementation, identifying the screen scaling type corresponding to the first image frame based on the real-time rendering data includes: determining a first distance between the game character and the camera corresponding to the first image frame based on the real-time rendering data; obtaining a second distance, wherein the second distance is the distance between the game character and the camera corresponding to a second image frame, and the second image frame is the image frame preceding the first image frame; determining that the screen scaling type corresponding to the first image frame is a sudden screen scaling change when the difference between the first distance and the second distance is greater than a distance threshold; and determining that the screen scaling type corresponding to the first image frame is a stable screen scaling when the difference between the first distance and the second distance is less than or equal to the distance threshold.

[0008] In this way, before performing rendering operations on the first image frame, it is possible to identify sudden changes in image scaling. This allows for the matching of appropriate optimization strategies based on the image scaling in advance.

[0009] In one possible implementation, determining the first distance between the game character corresponding to the first image frame and the camera based on the real-time rendering data includes: obtaining the spatial coordinates of the game character and the camera corresponding to the first image frame based on the real-time rendering data; determining the spatial coordinates of the center of a sphere around the game character based on the spatial coordinates of the game character and a first offset based on the spatial coordinates of the game character corresponding to the first image frame; the first offset being the distance offset of the spatial coordinates of the game character relative to the spatial coordinates of the center of the sphere corresponding to the camera; and determining the first distance based on the spatial coordinates of the center of the sphere and the spatial coordinates of the camera corresponding to the first image frame.

[0010] In this way, determining the first distance based on the spatial coordinates of the game character and the camera can avoid occlusion by different objects in the first image frame, which could lead to recognition errors.

[0011] In one possible implementation, identifying the vehicle type used by the game character corresponding to the first image frame based on the real-time rendering data includes: when the first distance and the second distance are the same, determining the set of spatial coordinates of the center of the sphere around the game character based on the spatial coordinates of the game character corresponding to the first image frame and a second offset set; wherein, the second offset set includes the distance offset of the spatial coordinates of the game character relative to the spatial coordinates of the center of the sphere corresponding to the camera when the game character uses different vehicles; determining a set of candidate distances based on the set of spatial coordinates of the center of the sphere and the spatial coordinates of the camera; determining a target distance from the set of candidate distances based on a first correspondence relationship; the first correspondence relationship is a correspondence relationship between vehicle type and reference distance, the reference distance being the distance between the camera and the center of the sphere around the game character when the game character uses a vehicle; the target distance belongs to the reference distance in the first correspondence relationship; determining the vehicle type used by the game character based on the first correspondence relationship and the target distance.

[0012] In this way, the type of vehicle used by the game character in each image frame can be matched in real time through the pre-established first correspondence.

[0013] In one possible implementation, identifying the game scene corresponding to the first image frame based on the real-time rendering data includes: obtaining feature information of static objects in the first image frame based on the real-time rendering data; determining the game scene corresponding to the first image frame based on the feature information of the static objects and a scene recognition database; the scene recognition database includes multiple game scenes and scene recognition features corresponding to each game scene.

[0014] In this way, the game scene corresponding to each image frame can be matched in real time through a pre-established scene recognition database.

[0015] In one possible implementation, the method further includes: acquiring historical rendering data, the historical rendering data including rendering data corresponding to multiple game scenes; extracting feature information of static objects corresponding to the multiple game scenes based on the historical rendering data; filtering out destructible feature information from the feature information of the static objects to obtain candidate feature information; determining scene recognition features corresponding to each game scene from the candidate feature information; and establishing a scene recognition database based on each game scene and the scene recognition features corresponding to each game scene.

[0016] In this way, by first filtering out destructible static objects in each game scene, the feature information of candidate static objects is obtained. Then, the scene recognition features corresponding to each game scene are further determined from the feature information of the candidate static objects. This can avoid the occurrence of scene misidentification due to the destruction of destructible static objects.

[0017] The scene recognition database can be stored in the electronic device's system library. When using the game scene switching recognition unit to identify game scenes later, the scene recognition database can be directly used to identify and match game scenes without recalculation.

[0018] In one possible implementation, identifying the scene lighting complexity corresponding to the first image frame based on the real-time rendering data includes: determining the light source distance between each light source and the camera in the first image frame based on the real-time rendering data; determining the rendering weight corresponding to each light source based on the light source type, light source distance, and a second correspondence relationship of each light source in the first image frame; the second correspondence relationship being the correspondence relationship between light source type, light source distance, and rendering weight; and determining the scene lighting complexity of the first image frame based on the rendering weight of each light source in the first image frame.

[0019] In this way, the second correspondence can be pre-calculated using historical rendering data, and the scene light complexity corresponding to each image frame can be calculated in real time.

[0020] In one possible implementation, the method further includes: acquiring historical rendering data, the historical rendering data including historical light source information, historical camera position information, and historical rendering load information corresponding to each lighting scene; the historical light source information including the number of historical light sources, the type of historical light sources, and the position information of historical light sources; determining the historical light source distance between each historical light source and the historical camera based on the position information of the historical light sources and the position information of the historical cameras; and determining the second correspondence between light source type, light source distance, and rendering weight based on the influence relationship between the historical light source distance and the historical rendering load information.

[0021] In one possible implementation, obtaining the real-time rendering data of the first image frame includes: obtaining a real-time rendering instruction; and if the rendering instruction includes preset rendering instruction features, obtaining the rendering data corresponding to the rendering instruction.

[0022] In this way, by pre-setting rendering instruction features, only the rendering data needed for scene recognition can be obtained, thereby reducing the workload of obtaining real-time rendering data.

[0023] In one possible implementation, an optimization strategy for the first image frame is determined based on at least one of the following: the screen scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene lighting complexity. This includes:

[0024] If the first image frame satisfies the first condition, the optimization strategy for the first image frame is determined to include core binding processing of the application thread corresponding to the first image frame; the first condition includes at least one of the following: the game scene corresponding to the first image frame is the same as the game scene corresponding to the second image frame, the vehicle type used by the game character is the target vehicle type, and the scene optical complexity is greater than the optical complexity threshold, wherein the second image frame is the previous image frame of the first image frame.

[0025] Different vehicle types, game scenes, and scene lighting complexity can all affect the rendering complexity of image rendering operations. The target vehicle can be a type of vehicle with high rendering overhead.

[0026] In this way, the system scheduling algorithm can adjust the resource allocation based on vehicle type, game scene, and lighting complexity before rendering is executed. For example, if the first image frame is identified as including the target vehicle and / or the lighting complexity of the current scene is high, core binding and frequency boosting operations can be performed on the game thread in advance. Thus, when the actual rendering operation is executed, the running speed of high-load game scenes can be optimized based on the pre-set core binding and frequency boosting operations.

[0027] If the first image frame satisfies the second condition, the optimization strategy for the first image frame is determined to include rendering the first image frame using a frame interpolation algorithm and / or an image super-resolution algorithm; the second condition includes at least one of the following: the screen scaling type is stable screen scaling, the game scene corresponding to the first image frame is the same as the game scene corresponding to the second image frame, and the second image frame is the previous image frame of the first image frame.

[0028] In this way, the frame interpolation algorithm can know the difference between the current image frame and the previous image frame before performing rendering processing. Therefore, the algorithm can decide whether to perform frame interpolation before rendering. Thus, during the actual rendering operation, the algorithm can decide whether to perform frame interpolation based on this decision. For example, if the image scaling type issued by the image scaling recognition unit is a sudden change in scaling, the frame interpolation algorithm can decide not to perform frame interpolation on the current image frame to avoid obtaining a poor-quality predicted frame due to significant differences between the two frames. In this way, no predicted frame will be drawn during the actual rendering operation, thus saving the computational cost of the frame interpolation algorithm.

[0029] When the game scene corresponding to the first image frame is a low-light game scene, the optimization strategy for the first image frame is determined to include increasing the screen brightness of the electronic device; when the game scene corresponding to the first image frame has a responsiveness requirement greater than a responsiveness threshold, the optimization strategy for the first image frame is determined to include reducing the screen refresh rate; when the game scene corresponding to the first image frame has a responsiveness requirement less than or equal to a responsiveness threshold, the optimization strategy for the first image frame is determined to include increasing the screen refresh rate.

[0030] In this way, post-processing algorithms or screen adjustment algorithms can dynamically adjust the screen brightness based on the identified game scene. For example, if the game scene switching recognition unit identifies the current image frame as a dark underground scene, the screen brightness can be increased using post-processing algorithms or screen adjustment algorithms. Screen adjustment algorithms can also dynamically adjust the screen refresh rate based on the game scene to provide users with a better gaming experience.

[0031] In one possible implementation, the method further includes: obtaining configuration data of the game application corresponding to the first image frame; and determining one or more parameters based on the configuration data, the parameters including a first offset, a second offset set, a first correspondence, a scene recognition database, and a second correspondence.

[0032] In this way, this application can support scene recognition for multiple games, making it more universal.

[0033] In one possible implementation, the configuration data includes a game name; determining one or more of the following parameters based on the configuration data includes: determining one or more of the parameters based on the game name.

[0034] In a second aspect, this application provides an electronic device including a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any one of the first aspects.

[0035] Thirdly, this application provides a chip system, characterized in that the chip system includes a processor; the processor is coupled to a memory for storing computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, the method as described in any one of the first aspects is executed.

[0036] Fourthly, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program or instructions that, when the computer program or instructions are run on a computer, cause the computer to perform the method as described in any one of the first aspects. Attached Figure Description

[0037] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0038] Figure 2 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0039] Figure 3 A flowchart illustrating a scene recognition method based on rendered data provided in this application embodiment;

[0040] Figure 4 A flowchart illustrating a method for intercepting rendering data provided in an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of the structure of a scene recognition module provided in an embodiment of this application;

[0042] Figure 6 This application provides a schematic diagram illustrating the movement of a camera around the main character in a game world.

[0043] Figure 7 This is a scene diagram of a sudden change in screen scaling according to an embodiment of this application;

[0044] Figure 8 A flowchart illustrating a scene recognition method based on screen scaling type provided in this application embodiment;

[0045] Figure 9 A flowchart illustrating a vehicle identification method provided in this application embodiment;

[0046] Figure 10 A flowchart illustrating a vehicle identification method provided in this application embodiment;

[0047] Figure 11 A flowchart illustrating the process of a game scene recognition method provided in this application embodiment;

[0048] Figure 12 A flowchart illustrating a scene optical complexity recognition method provided in an embodiment of this application;

[0049] Figure 13 A flowchart illustrating a scene recognition method based on rendered data provided in this application embodiment;

[0050] Figure 14A flowchart illustrating yet another scene recognition method based on rendered data provided in this application embodiment;

[0051] Figure 15 This is a structural block diagram of a chip provided in an embodiment of this application. Detailed Implementation

[0052] To meet users' growing demands for enhanced gaming experiences, game applications on mobile phones and other electronic devices are increasingly featuring richer rendering scenes and higher rendering quality. This leads to increased memory usage and power consumption in these devices. However, due to power consumption limitations or the capabilities of CPUs and GPUs, high refresh rates and high frame rates often cause overheating or stuttering, negatively impacting the user experience.

[0053] Taking mobile games as an example, with the development and promotion of large-scale mobile games, the rendering pipeline of modern mobile games is becoming increasingly complex, and the resource load is also increasing. However, the increase in computing power of modern mobile phones has not met the needs of modern large-scale mobile games, and mobile phones are limited by limited battery capacity and heat dissipation capabilities. Therefore, in order to improve the user's gaming experience, many optimization algorithms have emerged, such as system resource scheduling algorithms, frame interpolation algorithms, and image super-resolution algorithms.

[0054] However, current optimization algorithms are all based on rendered image frames. Taking frame interpolation as an example, these algorithms generate predicted frames based on two adjacent rendered image frames. This allows for an increase in refresh rate by inserting predicted frames between two adjacent image frames. However, when the two adjacent rendered image frames differ significantly, the generated predicted frames may exhibit image distortion and other problems.

[0055] To address the aforementioned technical problems, this application provides a scene recognition method based on rendering data. Before performing rendering operations on an image, the method identifies the scene corresponding to the current game application in advance using rendering data. This allows subsequent optimization algorithms to execute optimal optimization strategies based on the pre-identified scene changes, thereby improving the user experience.

[0056] For example, if a significant difference is detected between the image frame to be rendered and the previous image frame before rendering, the frame interpolation algorithm can decide not to perform frame interpolation before rendering. This avoids generating predicted frames with image distortion or deformation, and also saves computational resources for the frame interpolation algorithm.

[0057] The scene recognition method based on rendering data provided in this application can be applied to electronic devices with display functions. These electronic devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on.

[0058] In this application embodiment, the terminal device may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc.

[0059] One or more game applications can be installed in an electronic device. When the electronic device runs a game application, it processes and displays the corresponding game interface based on the scene recognition method based on rendering data according to the embodiments of this application.

[0060] For example, when an electronic device receives a user's click operation on a game application, the electronic device starts running the game application, and the scene recognition method based on rendering data according to the embodiments of this application processes and displays the corresponding game interface.

[0061] The structure of the electronic device in this application embodiment will be described below using a mobile phone as an example.

[0062] Figure 1 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Figure 1As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0063] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0064] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0065] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0066] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0067] In some embodiments, processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface 130, etc.

[0068] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0069] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0070] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0071] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0072] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0073] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0074] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0075] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0076] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0077] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0078] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0079] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0080] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0081] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0082] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0083] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0084] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0085] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0086] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0087] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0088] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0089] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0090] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0091] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0092] Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0093] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer (HAL), and the kernel layer.

[0094] The application layer can include a series of application packages.

[0095] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, game, WLAN, Bluetooth, music, video, and SMS.

[0096] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0097] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0098] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0099] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0100] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0101] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0102] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0103] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0104] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0105] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0106] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0107] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0108] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0109] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0110] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0111] A 2D graphics engine is a graphics engine for 2D drawing.

[0112] In this embodiment of the application, the system library may also include optimization algorithms such as scene recognition module, scene recognition service, system scheduling algorithm, frame interpolation algorithm, image super-resolution algorithm, post-processing algorithm, and screen control algorithm.

[0113] The scene recognition module can be used to identify, based on rendering data, the game scene, sudden changes in screen scaling, vehicle type, light complexity, and other conditions corresponding to the image frame that is about to be rendered.

[0114] Optimization algorithms such as frame interpolation, image super-resolution, post-processing, and screen adjustment can subscribe to corresponding scene recognition services. Then, the scene recognition module can distribute the corresponding recognition results to each optimization algorithm based on their subscription status. In this way, each optimization algorithm can make algorithmic decisions such as resource adjustments in advance based on the recognition results.

[0115] HAL (Hardware Abstraction Layer) is the abstract interface for the device kernel driver, providing application programming interfaces (APIs) for accessing the underlying device to higher-level Java API frameworks. The Hardware Abstraction Layer can include multiple library modules, such as display, audio, Bluetooth, and Wi-Fi modules. Each module can implement an interface for a specific type of hardware component. When the framework API requests access to the device hardware, the Android system loads the library module for that hardware component.

[0116] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0117] The following describes the scene recognition method based on rendering data provided in the embodiments of this application.

[0118] like Figure 3 As shown in the embodiments of this application, the scene recognition method based on rendering data mainly includes two parts. The first part is to calculate the basic parameters required by the scene recognition module based on historical rendering data. The second part is to use the scene recognition module to recognize the scene based on real-time rendering data and the calculated basic parameters when running the game application.

[0119] In some embodiments, the basic parameters required by the scene recognition module can be calculated based on historical rendering data in the following way: various game scenes, interaction modes, game effects, etc. in the game can be run in advance to obtain various rendering data required for calculating the basic parameters.

[0120] In some embodiments, when running a game application in the past, the electronic device can save the corresponding rendering data to the electronic device's system library (e.g., local system library) to obtain historical rendering data.

[0121] For example, such as Figure 4As shown, in response to user actions such as clicking and swiping on the game application interface, the game application at the application layer calls the rendering engine at the framework layer and sends rendering instructions to the system library through the rendering engine. Based on the received rendering instructions, the system library prepares rendering resources corresponding to those instructions. Then, the system library sends the rendering resources to the GPU, which renders the game screen based on the rendering resources. After receiving the rendered game screen, the GPU can output it to the display screen for game display.

[0122] This application embodiment can intercept resources during the rendering resource preparation stage to obtain rendering data for GPU rendering of game screens. The obtained rendering data can be stored in the system library of the electronic device.

[0123] It should be noted that, in the embodiments of this application, both historical rendering data and real-time rendering data can be obtained using the above-described interception method.

[0124] Historical rendering data can include rendering data corresponding to multiple game frames. The rendering data for each game frame can include a wide range of data. For example, rendering data can include texture information, vertex data, camera space coordinates, static object space coordinates, dynamic object space coordinates, light source space coordinates, number of light sources, and light source type.

[0125] This application embodiment can calculate the rendering data required for the basic parameters of the scene recognition module, i.e., the target rendering data, based on historical rendering data and statistical analysis. Then, the target rendering data can be extracted from a large amount of historical rendering data.

[0126] In some embodiments, the characteristics of rendering instructions corresponding to each target rendering data can also be statistically analyzed. This allows for the interception of corresponding target rendering data based on the characteristics of the rendering instructions during subsequent real-time game execution.

[0127] For example, a correspondence can be established between rendering instruction features and target rendering data. This correspondence might include feature tags corresponding to the rendering instruction features and the target rendering data. Thus, when a rendering instruction includes a feature corresponding to a feature tag, the target rendering data corresponding to that rendering instruction can be intercepted.

[0128] In this way, after obtaining the target rendering data, the basic parameters corresponding to the scene recognition module can be calculated based on the target rendering data.

[0129] like Figure 5 As shown, the scene recognition module in this embodiment may include four recognition units: screen scaling recognition unit, game scene switching recognition unit, vehicle recognition unit, and scene light complexity recognition unit.

[0130] The image scaling recognition unit can be used to identify the degree of image magnification or reduction based on rendering data. If the degree of image magnification or reduction is too large, it can be considered an abrupt change in image scaling.

[0131] The game scene switching recognition unit can be used to identify the game scene corresponding to each image frame based on rendering data, and then determine whether the game scene has changed relative to the previous image frame. For example, the game scene may include a lobby game scene, a land game scene, an underwater game scene, etc.

[0132] The vehicle identification unit can be used to identify the moving vehicle driven or ridden by a character in a game, based on rendered data. For example, vehicles can include cars, boats, airplanes, tanks, etc.

[0133] The scene light complexity recognition unit can be used to identify the light complexity of each image frame based on rendering data.

[0134] Among them, the target rendering data and the corresponding basic parameters to be calculated are different for each of the four recognition units in the scene recognition module, namely the screen scaling recognition unit, vehicle recognition unit, game scene switching recognition unit and scene light complexity recognition unit.

[0135] The following sections explain the calculation methods for the basic parameters of the four recognition units and the methods for scene recognition using the four recognition units.

[0136] The image scaling recognition unit provided in the embodiments of this application will be described first.

[0137] like Figure 5 As shown, the target rendering data required to calculate the basic parameters corresponding to the screen scaling recognition unit includes: the spatial coordinates of the main character and the spatial coordinates of the camera.

[0138] The main character's spatial coordinates refer to the main character's location information in the game world. The main character can be understood as the character controlled by the player.

[0139] In a game environment, a camera (also known as a following camera) is a virtual camera that reflects the player's perspective within the game screen. This perspective automatically follows and moves or rotates around the player-controlled main character.

[0140] For example, such as Figure 6As shown, in the game design, camera 10 will always revolve around the main character 20 in a spherical motion. The spatial coordinates of the center C1 of this spherical motion are usually not the same as the spatial coordinates P1 of the main character 20. The distance R1 between the center C1 and camera 10 represents the radius of the rotating sphere. A larger R1 value indicates a wider game world range and a farther field of view for camera 10, but relatively poorer detail display of the game character. Conversely, a smaller R1 value indicates a smaller game world range and a closer field of view for camera 10, highlighting more detail of the game character.

[0141] In some embodiments, the value of R1 can change in response to a user's zooming in or zooming out of the game screen. For example, when a user zooms in on the game screen, the value of R1 increases. When a user zooms out on the game screen, the value of R1 decreases.

[0142] In some embodiments, the user cannot control the zoom in or out of the game screen. That is, the distance between the camera 10 and the main character 20 in the game remains constant, unless in a camera collision scenario, in which case the distance between the camera and the main character will automatically decrease.

[0143] For example, in combination Figure 6 and Figure 7 As shown, when the camera is located at M1, the frame rate of the game screen is... Figure 7 The scene shown in (a) depicts the camera 10 rotating around the main character 20 as the main character 20 rotates. When the camera's perspective is... Figure 7 When the wall in (a) blocks the view, the main character 10 will not be visible. In this case, the distance between camera 10 and the main character 20 can be reduced to enlarge the field of view of camera 10, thus ensuring that the main character 20 is in the frame. For example, camera 10 can be moved forward from M1 to M1', and the view can be switched to... Figure 7 The image shown in (b) is shown in the middle. Among them, by Figure 7 The screen shown in (a) switches to Figure 7 The scene shown in (b) is a type of scene with a sudden change in screen scaling. This type of scene with a sudden change in screen scaling can also be called a camera collision scene.

[0144] In this embodiment of the application, the scene of sudden change in screen scaling refers to a scene in which the distance between the camera 10 and the main character 20 suddenly changes significantly, which results in a large difference in scaling between the current frame of the game screen and the previous frame of the game screen.

[0145] Figure 8 This is a flowchart illustrating a scene recognition method for screen scaling provided in an embodiment of this application. Figure 8 As shown, the following steps may be included:

[0146] S201: Obtain the spatial coordinates of the main character and the four non-coplanar cameras from the historical rendering data, with the main character's position remaining unchanged at the furthest viewpoint.

[0147] For example, please continue to refer to Figure 6 This method obtains the spatial coordinates of the main character 20 and the camera 10 at four different times (t1, t2, t3, and t4) from the farthest viewpoint, assuming the main character 20's position remains unchanged. At times t1, t2, t3, and t4, the main character 20's position is the same; for example, its spatial coordinates are all P1(x0, y0, z0). At times t1, t2, t3, and t4, the camera 10 rotates around the main character 20 on the same sphere, meaning its position is different. For example, at time t1, the camera 10's spatial coordinates are M1(x1, y1, z1); at time t2, they are M2(x2, y2, z2); at time t3, they are M3(x3, y3, z3); ​​and at time t4, they are M4(x4, y4, z4). The positions of the camera 10 at times t1, t2, t3, and t4 are not coplanar.

[0148] S202, based on the spatial coordinates of four non-coplanar cameras, calculate the spatial coordinates C1 of the center of the largest sphere and its radius R1.

[0149] For example, the equation of the sphere can be obtained by using the following formula (1) based on the spatial coordinates M1, M2, M3 and M4 of the camera, so as to obtain the spatial coordinates C1(a, b, c) of the center of the sphere and the radius R1.

[0150] (xa) 2 +(yb) 2 +(zc) 2 =R1 2 Formula (1)

[0151] Where x, y, z represent the spatial coordinates of the camera, a, b, c represent the spatial coordinates of the sphere's center, and R1 represents the radius of the largest sphere.

[0152] S203, based on the spatial coordinates C1 of the sphere's center and the spatial coordinates P1 of the main character, calculate the offset1 between the sphere's center and the main character.

[0153] It should be understood that after the main character's position in the game world changes, the main character's spatial coordinates P1 will change, but the offset 1 between the center of the sphere and the main character will not change.

[0154] In this way, the maximum sphere radius R1 and offset1 obtained by the above method can be used as the basic parameters of the scene change recognition unit. When using the scene scaling recognition unit to identify whether the game scene has changed abruptly, these basic parameters can be used directly without recalculation. For example, these basic parameters can be saved to the system library of the electronic device.

[0155] It should be noted that the above embodiments are illustrative only, using the calculation of a set of basic parameters as an example, and do not imply a limitation on the number of basic parameters. For example, this application embodiment can calculate the maximum sphere radius R1 and offset offset1 for each of multiple games. Then, the calculated multiple sets of sphere radii R1 and offset1 are stored as basic data in the system library of the electronic device for subsequent use by the screen scaling recognition unit to identify whether the game screen has a sudden scaling change. In this way, this application embodiment can support the recognition of screen scaling change scenes in multiple games, making it more universal.

[0156] For the method of image scaling recognition using the above basic parameters, sphere radius R1 and offset1, please refer to the description of steps S204 to S210.

[0157] S204, intercept the rendering data of the current image frame to obtain the spatial coordinates P2 of the main character and the spatial coordinates M5 of the camera.

[0158] In this embodiment, the current image frame refers to the image frame that is about to be rendered. For example, the previous image frame refers to the image frame currently displayed on the screen, and the current image frame refers to the next image frame to be displayed.

[0159] S205, based on the spatial coordinates P2 of the main character and the offset1 in the basic parameters, determine the spatial coordinates C2 of the center of the sphere around which the camera rotates.

[0160] S206, Based on the spatial coordinates C2 of the sphere's center and the spatial coordinates M5 of the camera, determine the radius R_D of the sphere corresponding to the current image frame.

[0161] S207, Based on the sphere radius R_D corresponding to the current image frame and the sphere radius corresponding to the previous image frame, determine the change A of the camera relative to the sphere center.

[0162] For example, such as Figure 6As shown, the spatial coordinates of the main character in the current image frame are P2 and the spatial coordinates of the camera are M5. Although the positions of the main character, the camera, and the center of the sphere around which the camera rotates will change with the position of the main character, the offset between the center of the sphere C2 around which the camera rotates and the spatial coordinates of the main character, P2, remains constant.

[0163] Therefore, based on the main character's spatial coordinates P2 and the offset 1 in the basic parameters, the spatial coordinates C2 of the sphere's center in the current image frame can be determined. Furthermore, based on the sphere's center coordinates C2 and the camera's spatial coordinates M5, the sphere's radius R_D in the current image frame can be determined. The sphere radius R_D refers to the distance between the camera and the main character in the current image frame.

[0164] In some embodiments, for games where the user cannot control the zoom in or out of the game screen, the radius of the sphere corresponding to the previous image frame in step S207 is always the maximum sphere radius R1.

[0165] In some embodiments, for games where the value of R1 can change in response to the user's zooming in or out of the game screen, typically, the distance between the camera and the main character is greatest when the game is first entered. Therefore, the radius of the sphere corresponding to the first image frame after entering the game is the maximum sphere radius R1. Thus, if the current image frame is the second image frame after the first image frame, the change A of the current camera relative to the center of the sphere is obtained based on the difference between the sphere radius R_D corresponding to the current image frame and the maximum sphere radius R1.

[0166] If the current image frame is the third image frame, then the change A of the current camera relative to the center of the sphere is obtained based on the difference between the sphere radius R_D corresponding to the current image frame and the sphere radius R_D corresponding to the second image frame.

[0167] Similarly, the above method can be used to determine the change A corresponding to subsequent image frames, which will not be elaborated here.

[0168] S208, determine whether the change A is greater than the distance threshold.

[0169] S209, when the change A is less than or equal to the distance threshold, the current scene is identified as a stable zoom scene.

[0170] For example, in games where the value of R1 changes in response to the user's zooming in or out of the game screen, the zooming in or out operation is a smooth scaling change scenario, and there is usually no sudden zooming in. Therefore, for such games, the change A is usually less than or equal to the distance threshold, meaning the current scene is a stable scaling scenario.

[0171] S210, if the change A is greater than the distance threshold, identify the current scene as a scene with sudden changes in screen scaling.

[0172] For example, in games where the user cannot control the zoom in or out of the game screen, when a situation arises... Figure 7 In the camera collision scenario shown, the R1 value suddenly decreases, causing the change A to exceed the distance threshold. From the user's perspective, the game screen appears to be instantly magnified. This situation can be identified as a sudden change in screen scaling.

[0173] Thus, the screen scaling recognition unit provided in this application embodiment can determine whether the screen scaling change type is a stable screen scaling scene or a screen scaling change scene based on real-time rendering data.

[0174] It should be noted that the change A in the above embodiments is an instantaneous value, that is, the change corresponding to one image frame. This application embodiment can also calculate whether the sphere radius R_D corresponding to the current image frame is a stable value based on the method provided in the above embodiments.

[0175] For example, if the sphere radius R_D corresponding to multiple consecutive image frames preceding the current image frame is the same, then the sphere radius R_D corresponding to the current image frame can be determined to be a stable value.

[0176] The vehicle identification unit provided in the embodiments of this application will be described below.

[0177] First, it's important to note that the main character's field of view changes before and after driving or riding in a vehicle. However, once the main character is driving or riding in a vehicle, their field of view stabilizes within the range corresponding to that vehicle. Therefore, if a change in the sphere's radius R_D is detected and stabilizes at the changed radius R_D, the vehicle being driven or ridden by the main character can be further identified.

[0178] For example, such as Figure 9 As shown, if a change in the sphere radius R_D of the i-th image frame is detected, and the sphere radius R_D of the N subsequent image frames is the same as that of the i-th image frame, then the sphere radius R_D can be determined to be a stable value. Thus, when the sphere radius R_D is a stable value, the vehicle identification unit can identify the vehicle type. For example, it can identify vehicle types such as vehicle 1, vehicle 2, vehicle 3, etc. Finally, the vehicle identification unit can output the identified vehicle type. Here, i is a positive integer, and N is a positive integer greater than 0.

[0179] The method for calculating the sphere radius R_D for each image frame can be found in the description of steps S204 to S210, and will not be repeated here.

[0180] Secondly, it should be noted that when the main character is driving or riding in a vehicle, the camera still rotates around the main character. However, the perspective differs depending on the vehicle the main character is driving or riding in; therefore, the center and radius of the sphere corresponding to the main character driving or riding in different vehicles are different.

[0181] Based on this, the target rendering data required for calculating the basic parameters of the vehicle identification unit in this application embodiment may include: the spatial coordinates of the main character when the main character drives or rides in different vehicles, the spatial coordinates of the camera, and the spatial coordinates of the vehicle.

[0182] Figure 10 This is a flowchart illustrating a vehicle identification method provided in an embodiment of this application. Figure 10 As shown, the following steps may be included:

[0183] S301: Obtain the spatial coordinates of the main character and the spatial coordinates of four non-coplanar cameras from historical rendering data when the main character is driving or riding in a vehicle.

[0184] S302, based on the spatial coordinates of four non-coplanar cameras, calculate the spatial coordinates C3 of the center of the sphere corresponding to the current vehicle and the radius R2.

[0185] S303, based on the spatial coordinates C3 of the current vehicle's center of gravity and the spatial coordinates P3 of the main character, calculate the offset 'offset2' between the spatial coordinates C3 of the center of gravity and the main character's P3.

[0186] By iterating through all the vehicles that need to be identified in the game and repeating steps S301 to S303 above, we can obtain the spatial coordinates C3 of the center of the sphere for each vehicle, as well as the offset 2 between the spatial coordinates C3 and the main character P3.

[0187] The specific implementation process of steps S301 to S303 can be found in the description of steps S201 to S203, and will not be repeated here.

[0188] In this way, the sphere radius R2 and offset offset2 corresponding to each vehicle obtained by the above method can be used as the basic parameters of the vehicle detection and recognition unit. When using the vehicle detection and recognition unit to identify vehicles in the game, these basic parameters can be used directly without recalculation. For example, these basic parameters can be saved to the system library of the electronic device.

[0189] It should be noted that the above embodiments are illustrative only, using the calculation of a set of basic parameters as an example, and do not imply a limitation on the number of basic parameters. For example, this application embodiment can calculate the sphere radius R2 and offset offset2 corresponding to each vehicle in multiple games. Then, the calculated sphere radii R2 and offset2 corresponding to multiple games are stored as basic data in the system library of the electronic device for subsequent use by the vehicle detection and recognition unit to identify vehicles in the game. In this way, this application embodiment can support vehicle recognition for multiple games, making it more universal.

[0190] It should also be noted that the above embodiments are only used as examples to illustrate the basic data, with the offset 2 between the spatial coordinates C3 of the center of the sphere corresponding to each vehicle and the main character P3 as the basic data. This does not imply any limitation on the basic data corresponding to the vehicle detection and recognition unit.

[0191] As the camera rotates with the main character, the offset 2 between the spatial coordinates C3 of the vehicle's center and the main character P3 remains unchanged, as does the offset 3 between the vehicle's center and the vehicle itself. Therefore, in this embodiment, the offset 3 between the spatial coordinates C3 of the vehicle's center and the vehicle can also be used as basic data. In this case, in step S301, the spatial coordinates of the vehicle and the spatial coordinates of the four non-coplanar cameras are obtained when the main character is driving or riding in the vehicle. Then, in step S302, based on the spatial coordinates C3 of the current vehicle's center and the current vehicle's spatial coordinates, the offset 3 between the spatial coordinates C3 of the center and the current vehicle is calculated.

[0192] Next, the vehicle can be identified using the basic parameters mentioned above (the sphere radius R2 and offset2 corresponding to each vehicle). For the specific vehicle identification method, please refer to the description of steps S304 to S306.

[0193] S304 intercepts the rendering data of the current image frame to obtain the spatial coordinates P4 of the main character and the spatial coordinates M6 of the camera when the main character is driving or riding in a vehicle.

[0194] S305, based on the spatial coordinates P4 of the main character and the offsets offset2 in the basic parameters, determine the spatial coordinates C4 of the center of the sphere around the main character for multiple cameras.

[0195] S306, based on the spatial coordinates C4 of each sphere's center and the spatial coordinates M6 of the camera, determine the radii R_D of multiple spheres.

[0196] Steps S304 to S306 can be found in the description of steps S301 to S303, and will not be repeated here.

[0197] S307, based on the correspondence between each vehicle and the sphere radius R2 and the sphere radius R_D, determine the type of vehicle used by the main character in the current image frame.

[0198] Since the sphere radius R2 corresponding to each vehicle is obtained in advance based on historical rendering data, the sphere radius R_D corresponding to the current image frame can be matched with the sphere radius R2 corresponding to each vehicle to determine the vehicle corresponding to the current image frame.

[0199] For example, as shown in Table 1, the vehicles in the game include cars, airplanes, boats, and tanks. The radius of the sphere corresponding to the main character driving a car is R2-1, the radius of the sphere corresponding to the main character driving an airplane is R2-2, the radius of the sphere corresponding to the main character driving a boat is R2-3, and the radius of the sphere corresponding to the main character driving a tank is R2-4. Thus, if the sphere radius R_D calculated in step S306 includes R2-1, then the vehicle driven by the main character is determined to be a car. If the sphere radius R_D calculated in step S306 includes R2-2, then the vehicle driven by the main character is determined to be an airplane. If the sphere radius R_D calculated in step S306 includes R2-3, then the vehicle driven by the main character is determined to be a boat. If the sphere radius R_D calculated in step S306 includes R2-4, then the vehicle driven by the main character is determined to be a tank.

[0200] Table 1. Correspondence between vehicle and sphere radius

[0201] Vehicle type sphere radius car R2-1 airplane R2-2 Boat R2-3 tank R2-4

[0202] The game scene switching recognition unit provided in the embodiments of this application will be described below.

[0203] A complete game typically contains multiple game scenes, each corresponding to a different game environment. Game environments usually include static objects such as buildings, trees, vegetation, and terrain. This application's embodiments can identify game scenes by analyzing the characteristics of these static objects.

[0204] Thus, the target rendering data required to calculate the basic parameters corresponding to the game scene switching recognition unit includes: the spatial coordinates of static objects.

[0205] Figure 11 This is a flowchart illustrating a game scene switching recognition method provided in an embodiment of this application. Figure 11 As shown, the following steps may be included:

[0206] S401 retrieves the spatial coordinates of static objects in all game scenes from historical rendering data.

[0207] S402, filter the spatial coordinates of destructible static objects to obtain the spatial coordinates of candidate static objects.

[0208] S403, determine the scene recognition features corresponding to each game scene from the spatial coordinates of the candidate static objects.

[0209] This application embodiment can first filter out destructible static objects in each game scene to obtain the spatial coordinates of candidate static objects. Then, it further determines the scene recognition features corresponding to each game scene from the spatial coordinates of the candidate static objects. In this way, it can avoid the occurrence of scene misidentification due to the destruction of destructible static objects.

[0210] In some embodiments, the most representative static object in each game scene can be determined from the spatial coordinates of candidate static objects. Then, the spatial coordinates of each most representative static object are determined as scene recognition features for each game scene.

[0211] Among them, the most representative static objects refer to static objects that are unique to each game scene and can be distinguished from other game scenes. For example, the most representative static objects can be one or more of the following: buildings, trees, vegetation, and terrain.

[0212] In this way, a scene recognition database can be obtained based on each game scene and the scene recognition features corresponding to each game scene. This scene recognition database can be stored in the electronic device's system library. When using the game scene switching recognition unit to identify game scenes subsequently, the scene recognition database can be directly used to identify and match game scenes without recalculation.

[0213] It should be noted that the above embodiments are only illustrative examples of constructing a scene recognition database for one game. Multiple scene recognition databases corresponding to various games can also be constructed using the above method and stored in the system library of the electronic device. In this way, the embodiments of this application can support scene recognition for multiple games, making them more universally applicable.

[0214] Furthermore, real-time game scene recognition can be performed based on the established scene recognition database. Please refer to the descriptions of steps S404 to S405 for details.

[0215] S404 intercepts the rendering data of the current image frame to obtain the spatial coordinates of the static object.

[0216] S405, based on the spatial coordinates of the matching static objects corresponding to the current image frame and the scene recognition database, determine the game scene corresponding to the current image frame.

[0217] For example, the spatial coordinates of the static object corresponding to the current image frame can be compared with the scene recognition features corresponding to each game scene in the scene recognition database. If the spatial coordinates of the static object corresponding to the current image frame match the scene recognition features corresponding to a certain game scene, then the game scene of the current image frame can be confirmed as the matching game scene.

[0218] Furthermore, it is also possible to determine whether the game scene has been switched based on the game scene corresponding to the current image frame and the game scene corresponding to the previous image frame.

[0219] The scene optical complexity recognition unit provided in the embodiments of this application is described below.

[0220] In game design, a light source is a virtual light source used to simulate and render lighting effects. Designers can place different numbers and types of light sources in a scene according to their needs to create ideal lighting and atmosphere effects.

[0221] The target rendering data required for calculating the basic parameters of the scene light complexity recognition unit in this embodiment may include: the spatial coordinates of the light source corresponding to each lighting scene, the number of light sources, the type of light source, the spatial coordinates of the camera, and rendering load information.

[0222] Figure 12 This is a flowchart illustrating a scene optical complexity recognition method provided in an embodiment of this application. Figure 12 As shown, the following steps may be included:

[0223] S501 obtains light source information, rendering load information, and camera spatial coordinates corresponding to the lighting scene from historical rendering data.

[0224] Light source information can include the number of light sources, the type of light source, and the spatial coordinates of the light sources. Among them, the type of light source can include point light sources, parallel light sources, spotlights, etc.

[0225] Rendering load information refers to the rendering load metrics corresponding to rendering a scene under lighting conditions. For example, rendering load metrics may include: GPU idle level, load level, frequency, etc., and may also include CPU frequency and memory frequency, etc.

[0226] In this embodiment, rendering load information does not need to be obtained by intercepting rendering resources. During game runtime, the electronic device automatically saves the rendering load information in the system library. Therefore, the rendering load information corresponding to each lighting scene can be directly obtained from the system library.

[0227] S502, based on the spatial coordinates of the light source and the spatial coordinates of the camera, determines the distance S between each light source and the camera in the lighting scene.

[0228] Specifically, the difference between the spatial coordinates of each light source and the spatial coordinates of the camera in the same lighting scene can be calculated to obtain the distance S between each light source and the camera in that lighting scene.

[0229] Similarly, the distance S between each light source and the camera can be calculated for all lighting scenarios.

[0230] S503 uses statistical analysis to determine the relationship between light source type, light source distance S and rendering weight by analyzing the impact of light source type and light source distance S on rendering load.

[0231] For the same light source, the rendering load may differ depending on the distance S between the light source and the camera. Conversely, for different light sources, the rendering load may also differ even when the distance S between the light source and the camera is the same.

[0232] Therefore, the embodiments of this application can analyze the impact of each light source on the rendering load when it is at different light source distances S from the camera, based on the correspondence between the light source distance S between each light source and the camera and the rendering load information, thereby determining the rendering weight of each light source at different light source distances S.

[0233] In other words, in the embodiments of this application, the rendering weight refers to the rendering weight value corresponding to each light source type at different light source distances S.

[0234] For example, as shown in Table 2, taking point light sources, parallel lights, and spotlights as examples, the rendering weight of the same light source varies depending on the distance S between it and the camera. Different types of light sources may also have different rendering weights when the distance S between them and the camera is the same or different. For instance, when the distance S between a point light source and the camera satisfies S1 < S ≤ S2, the rendering weight of that point light source is W1. When the distance S between the point light source and the camera satisfies S2 < S ≤ S3, the rendering weight of that point light source is W2. When the distance S between the point light source and the camera satisfies S3 < S ≤ S4, the rendering weight of that point light source is W3. Where S1 < S2 < S3 < S4. As another example, when the distance S between a parallel light source and the camera satisfies S1 < S ≤ S2, the rendering weight of that point light source is W4. When the distance S between a spotlight and the camera satisfies S1 < S ≤ S2, the rendering weight of that point light source is W7.

[0235] Table 2. Correspondence between light source type, distance S, and rendering weight

[0236]

[0237]

[0238] In some embodiments, a light source weight model can be obtained based on the correspondence between light source type, light source distance S, and rendering weight. The input to this light source weight model can be the type of light source corresponding to the lighting scene, the number of light sources, and the light source distance S between each light source and the camera. The output of the light source weight model can be the rendering load corresponding to the lighting scene. The rendering weight can be used as a constant in the light source weight model. Thus, the light source weight model can calculate the rendering load corresponding to the lighting scene based on the type of light source, the number of light sources, the light source distance S between each light source and the camera, and the rendering weight.

[0239] The following describes the method for identifying scene lighting complexity using basic parameters (determining the rendering weights corresponding to each light source) or a light source weight model. For details, please refer to the descriptions of steps S504 to S506.

[0240] S504 intercepts the rendering data of the current image frame and obtains the number of light sources, the type of light source, the spatial coordinates of the light source, and the spatial coordinates of the camera corresponding to the current image frame.

[0241] S505, determine the distance between each light source and the camera in the current image frame.

[0242] S506, based on the light source type, light source distance of each light source in the current image frame, and the correspondence between the light source type, light source distance S obtained in step S503 and the rendering weight, determine the rendering weight corresponding to each light source.

[0243] S507, based on the rendering weight of each light source in the current image frame, determines the rendering load of the current image frame.

[0244] In some embodiments, the number of light sources corresponding to the current image frame, the type of light source, and the distance between each light source and the camera can be input into the light source weight model. In this way, the light source weight model can calculate the rendering load corresponding to the current image frame.

[0245] For example, the light source weighting model can calculate the rendering load corresponding to each light source in the current image frame. Then, the rendering loads corresponding to all light sources are summed or averaged to obtain the rendering load corresponding to the current image frame.

[0246] S508 determines the optical complexity of the current image frame based on the rendering load corresponding to the current image frame.

[0247] For example, the rendering load can be pre-classified into optical complexity levels. For instance, the greater the rendering load, the higher the corresponding optical complexity level.

[0248] In this way, the optical complexity of the current image frame can be determined based on the pre-defined optical complexity levels.

[0249] Among them, the higher the level of optical complexity of an image frame, the heavier the rendering load when rendering it.

[0250] Based on the above description, such as Figure 5 As shown in the embodiments of this application, the scene recognition module's image scaling recognition unit can identify whether the scaling type corresponding to the current image frame is stable scaling or abrupt scaling. The vehicle recognition unit can identify the type of vehicle driven or ridden by the main character in the current image frame. The game scene switching recognition unit can identify the game scene corresponding to the current image frame. The scene light complexity recognition unit can identify the light complexity corresponding to the current image frame.

[0251] Thus, the scene recognition module provided in this application embodiment can identify the screen scaling type, vehicle type, game scene, and lighting complexity of the current image frame before rendering it, based on the intercepted rendering data corresponding to the current image frame. This allows subsequent optimization algorithms to execute optimal optimization strategies based on the pre-identified scene conditions, thereby improving the user experience.

[0252] In some embodiments, such as Figure 13 As shown, the services and processes corresponding to each optimization algorithm can subscribe to the corresponding recognition service from the scene recognition service. For example, each optimization algorithm can subscribe to the recognition service of any one or more of the four recognition units. In this way, each recognition unit in scene recognition can send its recognition result to the scene recognition service. Then, based on the subscription status of each optimization algorithm, the scene recognition service distributes the corresponding recognition result to each optimization algorithm.

[0253] The following section uses optimization algorithms, including system scheduling algorithms, frame interpolation algorithms, super-resolution algorithms, post-processing algorithms, and screen control algorithms, as examples to explain how each optimization algorithm executes the best optimization strategy based on the recognition results of the scene recognition module.

[0254] In this embodiment, the system scheduling algorithm can dynamically adjust the resource supply based on the rendering complexity of each image frame.

[0255] Among them, different vehicle types, game scenes, and scene light complexity can all affect the rendering complexity of performing rendering operations on images.

[0256] For example, the system scheduling algorithm can subscribe to the recognition services provided by the vehicle recognition unit, game scene switching recognition unit, and scene light complexity recognition unit in the scene recognition module. In this way, the system scheduling algorithm can adjust the resource allocation based on vehicle type, game scene, and light complexity before executing rendering processing. For instance, if the current image frame is identified as containing vehicles and / or the current lighting scene has high light complexity, core binding and frequency boosting operations can be performed on the game thread in advance. Thus, when the actual rendering operation is executed, the running speed of high-load game scenes can be optimized based on the pre-set core binding and frequency boosting operations.

[0257] In this embodiment, the frame interpolation algorithm can draw a rendering prediction frame based on the current image frame when the game scene is the same in two adjacent image frames and there is no sudden change in screen scaling; otherwise, no rendering prediction frame is drawn.

[0258] For example, the frame interpolation algorithm can subscribe to the recognition services provided by the screen scaling recognition unit and the game scene switching recognition unit. This allows the algorithm to know the difference between the current image frame and the previous image frame before performing rendering processing. Consequently, the algorithm can decide whether to perform frame interpolation before rendering. Thus, during the actual rendering operation, frame interpolation can be performed or not based on this decision. For instance, if the screen scaling image type issued by the screen scaling recognition unit is a sudden screen scaling change, the algorithm can decide not to perform frame interpolation on the current image frame to avoid obtaining a poor-quality predicted frame due to significant differences between the two frames. Therefore, during the actual rendering operation, the predicted frame will not be drawn, thus saving computational resources for the frame interpolation algorithm.

[0259] In this embodiment, the super-resolution algorithm can perform image super-resolution processing on the current image frame when the game scene is the same in multiple consecutive image frames and there is no sudden change in screen scaling; otherwise, image super-resolution processing is not performed.

[0260] For example, the super-resolution algorithm can also subscribe to the recognition services provided by the screen scaling recognition unit and the game scene switching recognition unit. In this way, the super-resolution algorithm can know the difference between the current image frame and the previous image frame before performing rendering processing. For instance, if the game scene changes as indicated by the game scene switching recognition unit, the game super-resolution algorithm can decide not to perform game super-resolution processing on the current image frame to avoid obtaining a poor-quality image due to significant differences between the two frames. This also saves computational resources for the game super-resolution algorithm.

[0261] The post-processing algorithms in this application include processing to enhance the visual effects of the image. For example, image enhancement processing can improve the contrast, brightness, and sharpness of the image. Another example is image restoration processing such as denoising, deblurring, and occlusion repair.

[0262] For example, the post-processing algorithm can subscribe to the recognition services provided by the game scene switching recognition unit and the scene light complexity recognition unit. This allows for the use of a post-processing algorithm with lower overhead when the scene light complexity is high, and a post-processing algorithm with higher overhead when the scene light complexity is low. This enables the selection of the appropriate post-processing algorithm based on the scene light complexity to balance the overall load on the electronic device. As another example, the post-processing algorithm can dynamically adjust the screen brightness based on the game scene identified by the game scene switching recognition unit. For instance, if the game scene switching recognition unit identifies the current image frame as a dark underground scene, the post-processing algorithm can increase the screen brightness to provide a better gaming experience for the user.

[0263] In this embodiment, the screen control algorithm can dynamically adjust the screen's refresh rate, brightness, etc., based on the game scene.

[0264] For example, the screen adjustment algorithm can subscribe to the recognition service provided by the game scene switching recognition unit. In this way, the screen adjustment algorithm adjusts the screen's refresh rate, brightness, and other settings based on the game scene identified by the game scene switching recognition unit. For instance, for game scenes where responsiveness is not critical, the screen refresh rate can be reduced; for game scenes where responsiveness is critical, the screen refresh rate can be increased. As another example, for darker underground scenes, the screen adjustment algorithm can increase the screen brightness to provide the user with a better gaming experience.

[0265] Figure 14 This is a flowchart illustrating a scene recognition method based on rendered data, provided in an embodiment of this application. Figure 14 As shown, the following steps may be included:

[0266] S601, before performing a rendering operation on the first image frame (also known as the current image frame), obtain the real-time rendering data of the first image frame.

[0267] In one possible implementation, the real-time rendering data of the first image frame can be obtained by: obtaining a real-time rendering instruction; and if the rendering instruction includes preset rendering instruction features, obtaining the rendering data corresponding to the rendering instruction.

[0268] Real-time rendering data can include the spatial coordinates of game characters, the spatial coordinates of cameras, the spatial coordinates of vehicles, the spatial coordinates of static objects, and light source information.

[0269] The method for obtaining the real-time rendering data of the first image frame can be found in [reference needed]. Figure 4 The description will not be repeated here.

[0270] S602, based on real-time rendering data, identifies at least one of the following: the screen scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene light complexity.

[0271] In one possible implementation, identifying the screen scaling type corresponding to the first image frame based on real-time rendering data may include: determining a first distance between the game character (also known as the main character) and the camera corresponding to the first image frame based on the real-time rendering data; obtaining a second distance, where the second distance is the distance between the game character and the camera corresponding to the second image frame, and the second image frame is the image frame preceding the first image frame; if the difference between the first distance and the second distance (also known as the change A) is greater than a distance threshold, determining that the screen scaling type corresponding to the first image frame is a sudden screen scaling change; if the difference between the first distance and the second distance is less than or equal to the distance threshold, determining that the screen scaling type corresponding to the first image frame is a stable screen scaling change.

[0272] In one possible implementation, determining the first distance between the game character and the camera corresponding to the first image frame based on real-time rendering data includes: obtaining the spatial coordinates of the game character and the camera corresponding to the first image frame based on real-time rendering data; determining the spatial coordinates of the center of the sphere around the game character (equivalent to the spatial coordinates C2 of the center of the sphere in the above embodiment) based on the spatial coordinates of the game character and the first offset (also called offset1) of the game character; the first offset is the distance offset of the spatial coordinates of the game character relative to the spatial coordinates of the center of the sphere corresponding to the camera; and determining the first distance (equivalent to the sphere radius R_D determined in step S206 in the above embodiment) based on the spatial coordinates of the center of the sphere corresponding to the first image frame and the spatial coordinates of the camera.

[0273] The process of identifying the image scaling type corresponding to the first image frame can be found in the description of the image scaling identification unit in the above embodiments, and will not be repeated here.

[0274] In one possible implementation, based on real-time rendering data, identifying the vehicle type used by the game character corresponding to the first image frame includes: when the first distance and the second distance are the same, determining the set of spatial coordinates of the center of the sphere around the game character (equivalent to the spatial coordinates C4 of multiple sphere centers determined in step S305 in the above embodiment) based on the spatial coordinates of the game character corresponding to the first image frame and the second offset set (the second offset set includes the offset offset2 corresponding to each vehicle); wherein, the second offset set includes the distance between the spatial coordinates of the game character and the spatial coordinates of the sphere center corresponding to the camera when the game character uses different vehicles. Offset; Based on the set of spatial coordinates of the sphere's center and the spatial coordinates of the camera, determine a set of candidate distances (equivalent to the multiple sphere radii R_D determined in step S306 in the above embodiment); Based on the first correspondence, determine the target distance from the set of candidate distances; The first correspondence is the correspondence between vehicle type and reference distance (equivalent to the sphere radius R2 corresponding to each vehicle determined in the above embodiment), and the reference distance is the distance between the camera and the center of the sphere around which the camera rotates when the game character uses a vehicle; The target distance belongs to the reference distance in the first correspondence; Based on the first correspondence and the target distance, determine the type of vehicle used by the game character.

[0275] The identification of the vehicle type used by the game character corresponding to the first image frame can be found in the description of the vehicle identification unit in the above embodiments, and will not be repeated here.

[0276] In one possible implementation, identifying the game scene corresponding to the first image frame based on real-time rendering data includes: obtaining feature information of static objects in the first image frame (e.g., spatial coordinates of static objects) based on the real-time rendering data; determining the game scene corresponding to the first image frame based on the feature information of the static objects and a scene recognition database; the scene recognition database includes multiple game scenes and scene recognition features corresponding to each game scene.

[0277] In one possible implementation, the scene recognition database can be established as follows: acquiring historical rendering data, which includes rendering data corresponding to multiple game scenes; extracting feature information of static objects corresponding to multiple game scenes based on the historical rendering data; filtering out destructible feature information from the feature information of static objects to obtain candidate feature information; determining scene recognition features corresponding to each game scene from the candidate feature information; and establishing a scene recognition database based on each game scene and the scene recognition features corresponding to each game scene.

[0278] The identification of the game scene corresponding to the first image frame can be found in the description of the game scene switching identification unit in the above embodiments, and will not be repeated here.

[0279] In one possible implementation, the scene light complexity corresponding to the first image frame is identified based on real-time rendering data, including: determining the light source distance between each light source and the camera in the first image frame based on real-time rendering data; determining the rendering weight corresponding to each light source based on the light source type, light source distance, and a second correspondence in the first image frame; the second correspondence is the correspondence between light source type, light source distance, and rendering weight; and determining the scene light complexity of the first image frame based on the rendering weight of each light source in the first image frame.

[0280] In one possible implementation, the second correspondence can be obtained as follows: acquiring historical rendering data, which includes historical light source information, historical camera position information, and historical rendering load information corresponding to each lighting scene; the historical light source information includes the number of historical light sources, the type of historical light sources, and the position information of historical light sources; determining the historical light source distance between each historical light source and the historical camera based on the position information of the historical light sources and the position information of the historical cameras; and determining the second correspondence between the light source type, the light source distance, and the rendering weight based on the influence relationship between the historical light source distance and the historical rendering load information.

[0281] The identification of the scene light complexity corresponding to the first image frame can be found in the description of the scene light complexity identification unit in the above embodiments, and will not be repeated here.

[0282] S603, based on at least one of the following: the screen scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene light complexity, determine an optimization strategy for the first image frame; the optimization strategy includes at least one of the following: system scheduling strategy, frame interpolation strategy, image super-resolution strategy, image post-processing strategy, and screen control strategy;

[0283] In one possible implementation, an optimization strategy for a first image frame is determined based on at least one of the following: the screen scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene optical complexity. This includes: if the first image frame satisfies a first condition, determining the optimization strategy for the first image frame includes performing core binding processing on the application thread corresponding to the first image frame; the first condition includes at least one of the following: the game scene corresponding to the first image frame is the same as the game scene corresponding to the second image frame, the vehicle type used by the game character is the target vehicle type, and the scene optical complexity is greater than a optical complexity threshold (e.g., the scene optical complexity level is greater than a preset optical complexity level); if the first image frame satisfies a second condition, determining the optimization strategy for the first image frame includes using interpolation. The first image frame is rendered using a frame algorithm and / or an image super-resolution algorithm; the second condition includes at least one of the following: the screen scaling type is stable scaling, the game scene corresponding to the first image frame is the same as the game scene corresponding to the second image frame, and the second image frame is the previous image frame of the first image frame; if the game scene corresponding to the first image frame is a low-light game scene, the optimization strategy for the first image frame is determined to include increasing the screen brightness of the electronic device; if the game scene corresponding to the first image frame has a responsiveness requirement greater than a responsiveness threshold, the optimization strategy for the first image frame is determined to include reducing the screen refresh rate; if the game scene corresponding to the first image frame has a responsiveness requirement less than or equal to a responsiveness threshold, the optimization strategy for the first image frame is determined to include increasing the screen refresh rate.

[0284] The method for determining the optimization strategy for the first image frame based on at least one of the following: the image scaling type corresponding to the first image frame, the vehicle type used by the game character, the game scene, and the scene lighting complexity, can be found in the above embodiments. Figure 13 The description will not be repeated here.

[0285] S604, based on the optimization strategy, performs a rendering operation on the first image frame.

[0286] In one possible implementation, the method further includes: obtaining configuration data of the game application corresponding to the first image frame; and determining one or more of the following parameters based on the configuration data, including a first offset, a second offset set, a first correspondence, a scene recognition database, and a second correspondence.

[0287] For example, the configuration data includes the game name; thus, one or more parameters can be determined based on the game name.

[0288] The various method embodiments described herein can be independent solutions or combinations based on internal logic, and all such solutions fall within the protection scope of this application.

[0289] It is understood that the methods and operations implemented by electronic devices in the above-described method embodiments can also be implemented by components (such as chips or circuits) that can be used in electronic devices.

[0290] The above embodiments describe the scene recognition method based on rendered data provided in this application. It is understood that, in order to achieve the above functions, the electronic device includes hardware structures and / or software modules corresponding to each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0291] This application also provides a processing apparatus, which includes at least one processor and a communication interface. The communication interface is used to provide information input and / or output to the at least one processor, which is used to execute the methods described in the above method embodiments.

[0292] It should be understood that the aforementioned processing device can be a chip. For example, see Figure 15 , Figure 15 This is a structural block diagram of a chip provided in an embodiment of this application. Figure 15 The chip shown can be a general-purpose processor or a special-purpose processor. The chip 700 may include at least one processor 701. The at least one processor 701 can be used to support the technical solutions corresponding to any of the above embodiments.

[0293] Optionally, the chip 700 may further include a transceiver 702, which is used to receive control from the processor 701 and to support the technical solutions corresponding to any of the above embodiments. Optionally, Figure 15 The chip 700 shown may also include a storage medium 703. Specifically, the transceiver 702 may be replaced by a communication interface that provides information input and / or output to the at least one processor 701.

[0294] It should be noted that, Figure 15The chip 700 shown can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), application-specific integrated circuits (ASICs), system-on-chips (SoCs), central processors (CPUs), network processors (NPs), digital signal processors (DSPs), microcontrollers (MCUs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various functions described throughout this application.

[0295] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0296] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0297] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0298] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer program product, which includes: a computer program or instructions, which, when run on a computer, cause the computer to perform the method of any one of the embodiments of the method embodiments.

[0299] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer storage medium storing a computer program or instructions, which, when run on a computer, causes the computer to perform the method of any one of the embodiments of the method.

[0300] According to the method provided in the embodiments of this application, the embodiments of this application also provide an electronic device, including a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, the electronic device performs the method of any one of the embodiments of the method embodiments.

[0301] According to the method provided in the embodiments of this application, this application also provides a chip system. The chip system includes a processor coupled to a memory for executing computer programs or instructions stored in the memory. When the computer program or instructions are executed, the chip system can implement all or part of the steps in the method embodiments. This chip system may be composed of chips or may include chips and other discrete devices.

[0302] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0303] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0304] The computer storage medium, computer program product, and electronic device provided in the embodiments of this application are all used to execute the methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects corresponding to the methods provided above, and will not be repeated here.

[0305] It should be understood that in the various embodiments of this application, the execution order of each step should be determined by its function and internal logic, and the size of each step number does not mean the order of execution, and does not constitute a limitation on the implementation process of the embodiments.

[0306] The various parts of this specification are described in a progressive manner. Similar or identical parts between the various embodiments can be referred to interchangeably. Each embodiment focuses on the differences from other embodiments. In particular, the embodiments for devices, computer storage media, computer program products, and electronic devices are basically similar to the method embodiments, so the descriptions are relatively simple; relevant details can be found in the descriptions within the method embodiments.

[0307] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0308] The embodiments described above do not constitute a limitation on the scope of protection of this application.

Claims

1. A scene recognition method based on rendered data, characterized in that, The method is applied to an electronic device, and the method includes: Before performing rendering operations on the first image frame, obtain the real-time rendering data of the first image frame; Based on the real-time rendering data, the vehicle type used by the game character corresponding to the first image frame is identified; this includes: determining a first distance between the game character and the camera corresponding to the first image frame based on the real-time rendering data; obtaining a second distance, the second distance being the distance between the game character and the camera corresponding to a second image frame, the second image frame being the image frame preceding the first image frame; when the first distance and the second distance are the same, determining a set of spatial coordinates of the center of the sphere around the game character based on the spatial coordinates of the game character corresponding to the first image frame and a second offset set, the second offset set including the distance offset of the game character's spatial coordinates relative to the spatial coordinates of the center of the sphere corresponding to the camera when the game character uses different vehicles; determining a set of candidate distances based on the set of spatial coordinates of the center of the sphere and the spatial coordinates of the camera; determining a target distance from the set of candidate distances based on a first correspondence relationship; the first correspondence relationship being a correspondence between vehicle type and reference distance, the reference distance being the distance between the camera and the center of the sphere around the game character when the game character uses a vehicle; the target distance being a reference distance in the first correspondence relationship; determining the vehicle type used by the game character based on the first correspondence relationship and the target distance. Based on the vehicle type used by the game character corresponding to the first image frame, an optimization strategy is determined for the first image frame; the optimization strategy includes at least one of the following: system scheduling strategy, frame interpolation strategy, image super-resolution strategy, image post-processing strategy, and screen control strategy; Based on the optimization strategy, a rendering operation is performed on the first image frame.

2. The scene recognition method based on rendering data according to claim 1, characterized in that, The step of identifying the vehicle type used by the game character corresponding to the first image frame based on the real-time rendering data includes: Based on the real-time rendering data, identify the type of vehicle used by the game character corresponding to the first image frame, and identify at least one of the screen scaling type, game scene, and scene light complexity corresponding to the first image frame; The step of determining an optimization strategy for the first image frame based on the vehicle type used by the game character corresponding to the first image frame includes: Based on the vehicle type used by the game character corresponding to the first image frame, and at least one of the screen scaling type, game scene, and scene light complexity, the optimization strategy for the first image frame is determined.

3. The method according to claim 2, characterized in that, The step of identifying the type of vehicle used by the game character corresponding to the first image frame based on the real-time rendering data, and identifying at least one of the screen scaling type, game scene, and scene light complexity corresponding to the first image frame, includes: If the difference between the first distance and the second distance is greater than the distance threshold, the image scaling type corresponding to the first image frame is determined to be a sudden change in image scaling. If the difference between the first distance and the second distance is less than or equal to the distance threshold, the image scaling type corresponding to the first image frame is determined to be stable image scaling.

4. The method according to claim 3, characterized in that, Determining the first distance between the game character and the camera corresponding to the first image frame based on the real-time rendering data includes: Based on the real-time rendering data, the spatial coordinates of the game character and the camera corresponding to the first image frame are obtained. Based on the spatial coordinates of the game character corresponding to the first image frame and the first offset, the spatial coordinates of the center of the sphere around which the camera rotates are determined; the first offset is the distance offset of the spatial coordinates of the game character relative to the spatial coordinates of the center of the sphere corresponding to the camera. The first distance is determined based on the spatial coordinates of the sphere's center corresponding to the first image frame and the spatial coordinates of the camera.

5. The method according to claim 2, characterized in that, The step of identifying the type of vehicle used by the game character corresponding to the first image frame based on the real-time rendering data, and identifying at least one of the screen scaling type, game scene, and scene light complexity corresponding to the first image frame, includes: Based on the real-time rendering data, the feature information of static objects in the first image frame is obtained; Based on the feature information of the static object and the scene recognition database, the game scene corresponding to the first image frame is determined; the scene recognition database includes multiple game scenes and scene recognition features corresponding to each game scene.

6. The method according to claim 5, characterized in that, The method further includes: Acquire historical rendering data, which includes rendering data corresponding to multiple game scenes; Based on the historical rendering data, feature information of static objects corresponding to the multiple game scenes is extracted; Filter the destructible feature information from the feature information of the static object to obtain alternative feature information; Determine the scene recognition features corresponding to each game scene from the candidate feature information; A scene recognition database is established based on each game scene and the scene recognition features corresponding to each game scene.

7. The method according to claim 2, characterized in that, The step of identifying the type of vehicle used by the game character corresponding to the first image frame based on the real-time rendering data, and identifying at least one of the screen scaling type, game scene, and scene light complexity corresponding to the first image frame, includes: Based on the real-time rendering data, the distance between each light source and the camera in the first image frame is determined; Based on the light source type, light source distance, and second correspondence of each light source in the first image frame, the rendering weight corresponding to each light source is determined; the second correspondence is the correspondence between light source type, light source distance, and rendering weight. The scene lighting complexity of the first image frame is determined based on the rendering weight of each light source in the first image frame.

8. The method according to claim 7, characterized in that, The method further includes: Acquire historical rendering data, which includes historical light source information, historical camera position information, and historical rendering load information for each lighting scene; the historical light source information includes the number of historical light sources, the type of historical light sources, and the position information of historical light sources. Based on the location information of the historical light source and the location information of the historical camera, the historical light source distance between each historical light source and the historical camera is determined; Based on the influence relationship between the historical light source distance and the historical rendering load information, the second correspondence between light source type, light source distance and rendering weight is determined.

9. The method according to claim 1, characterized in that, The step of obtaining the real-time rendering data of the first image frame includes: Obtain real-time rendering instructions; If the rendering instruction includes preset rendering instruction features, the rendering data corresponding to the rendering instruction is obtained.

10. The method according to claim 2, characterized in that, The step of determining the optimization strategy for the first image frame based on the vehicle type used by the game character corresponding to the first image frame, and at least one of the following: screen scaling type, game scene, and scene lighting complexity, includes: If the first image frame satisfies the first condition, the optimization strategy for the first image frame is determined to include core binding processing of the application thread corresponding to the first image frame; the first condition includes at least one of the following: the game scene corresponding to the first image frame is different from the game scene corresponding to the second image frame, the vehicle type used by the game character is the target vehicle type, the scene optical complexity is greater than the optical complexity threshold, and the second image frame is the previous image frame of the first image frame. If the first image frame satisfies the second condition, the optimization strategy for the first image frame is determined to include rendering the first image frame using a frame interpolation algorithm and / or an image super-resolution algorithm; the second condition includes at least one of the following: the screen scaling type is stable screen scaling, the game scene corresponding to the first image frame is the same as the game scene corresponding to the second image frame, and the second image frame is the previous image frame of the first image frame. When the game scene corresponding to the first image frame is a low-light game scene, the optimization strategy for the first image frame is determined to include increasing the screen brightness of the electronic device; If the game scene corresponding to the first image frame has a responsiveness requirement greater than a responsiveness threshold, the optimization strategy for the first image frame is determined to include reducing the screen refresh rate; if the game scene corresponding to the first image frame has a responsiveness requirement less than or equal to the responsiveness threshold, the optimization strategy for the first image frame is determined to include increasing the screen refresh rate.

11. The method according to claim 1, characterized in that, The method further includes: Obtain the configuration data of the game application corresponding to the first image frame; Based on the configuration data, one or more of the following parameters are determined, including a first offset, a second offset set, a first correspondence, a scene recognition database, and a second correspondence.

12. The method according to claim 11, characterized in that, The configuration data includes the game name; the determination of one or more of the following parameters based on the configuration data includes: Based on the game name, determine one or more of the parameters.

13. An electronic device, characterized in that, The device includes a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any one of claims 1-12.

14. A chip system, characterized in that, The chip system includes a processor; the processor is coupled to a memory for storing computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, the method as described in any one of claims 1-12 is performed.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-12.

Images