Data processing method and electronic device

Abstract

The present application discloses a data processing method and electronic device, the method comprising: obtaining configuration information of a first image, the configuration information comprising first image data, data format information, algorithm association information and scene information; the scene information comprising multiple algorithm modules corresponding to the first image; generating a first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information; the first algorithm path being a transmission path of the first image data between the multiple algorithm modules; and processing the first image data using a processor in the first algorithm path to obtain second image data. Through the present application, the algorithm path can be adaptively optimized, which is conducive to reducing power consumption and improving system performance.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular to a data processing method and an electronic device. Background Art

[0002] In the process of image data processing, an algorithm path is usually created. Here, the algorithm path refers to the path for image data to be transmitted between one or more algorithm modules. However, with the continuous increase in the types of algorithms, the algorithm paths in different scenarios are becoming more and more complex, resulting in increased power consumption and reduced system performance. Summary of the invention

[0003] The embodiments of the present application provide a data processing method and an electronic device. Based on the method described in the present application, the algorithm path can be adaptively optimized, which is beneficial to reducing power consumption and improving system performance.

[0004] In a first aspect, the present application provides a data processing method, the method comprising: obtaining configuration information of a first image, the configuration information comprising first image data, data format information, algorithm association information and scene information; the scene information comprising multiple algorithm modules corresponding to the first image; generating a first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information; the first algorithm path is a transmission path for the first image data between the multiple algorithm modules; and processing the first image data using a processor in the first algorithm path to obtain second image data.

[0005] Based on the method described in the first aspect, since the data format information and algorithm association information are combined when creating and configuring the first algorithm path, the number of image data format conversions can be better reduced, which is beneficial to reducing system power consumption, improving system performance, and realizing adaptive optimization of the algorithm path.

[0006] In a possible implementation, the scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the method further includes: determining a processor based on the scene information. Based on this method, the processor can be accurately selected, the system power consumption can be reduced, and the efficiency and effectiveness of data processing can be improved.

[0007] In a possible implementation, the scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the configuration information also includes system status information; the method further includes: determining a processor based on the system status information, or determining a processor based on the system status information and the scene information. Based on this method, the processor can be accurately selected, the system power consumption can be reduced, and the efficiency and effectiveness of data processing can be improved.

[0008] In a possible implementation, determining a processor based on the system state information and the scene information includes: determining priority information corresponding to the first image based on the system state information and the scene information; the priority information is used to indicate that the priority of system power consumption is greater than the priority of system time consumption, or the priority information is used to indicate that the priority of system time consumption is greater than the priority of system power consumption; determining a processor based on the priority information. Based on this method, due to the combination of the priority information corresponding to the first image, the processor can be selected more accurately, the system power consumption can be reduced, and the efficiency and effectiveness of data processing can be improved.

[0009] In a possible implementation manner, the data format information includes a data format of the first image data, a data format of the second image data, and an image data format supported by each algorithm module in the multiple algorithm modules.

[0010] In a possible implementation, determining the first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information, and the algorithm association information includes: sorting the multiple algorithm modules based on the data format information to obtain a first sequence; or sorting the multiple algorithm modules based on the algorithm association information to obtain a second sequence; adjusting the first sequence based on the algorithm association information to obtain a third sequence; or adjusting the second sequence based on the data format information to obtain a fourth sequence; processing the multiple algorithm modules according to the third sequence or the fourth sequence to generate the first algorithm path corresponding to the first image. Based on this method, the best algorithm path can be determined to achieve optimization of the algorithm path.

[0011] In a possible implementation, there are multiple third sequences, or there are multiple fourth sequences; before the multiple algorithm modules are processed according to the third sequence or the fourth sequence to generate the first algorithm path corresponding to the first image, the method further includes: determining a target sequence from the multiple third sequences or the multiple fourth sequences; the target sequence is the sequence with the lowest number of image format conversions between the multiple algorithm modules in the multiple third sequences or the multiple fourth sequences; processing the multiple algorithm modules according to the third sequence or the fourth sequence to generate the first algorithm path corresponding to the first image includes: processing the multiple algorithm modules according to the target sequence to generate the first algorithm path corresponding to the first image. Based on this method, the best algorithm path can be determined, and the optimization of the algorithm path can be further realized.

[0012] In a possible implementation, the scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the configuration information also includes system status information; determining the target sequence from multiple third sequences or multiple fourth sequences includes: determining the target sequence from multiple third sequences or multiple fourth sequences based on the scene information and / or the system status information. Based on this method, the best algorithm path can be determined, and the algorithm path can be further optimized.

[0013] In a second aspect, the present application provides a data processing device, which may be an electronic device, or a device in an electronic device, or a device that can be used in combination with an electronic device; wherein the data processing device may also be a chip system, and the data processing device may execute the method executed by the electronic device in the first aspect. The functions of the data processing device may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. The unit may be software and / or hardware. The operations and beneficial effects performed by the data processing device may refer to the method and beneficial effects of the first aspect above, and the repeated parts will not be repeated.

[0014] In a third aspect, the present application provides a data processing device, which includes a processor. When the processor calls a computer program in a memory, the method in the first aspect is executed.

[0015] In a fourth aspect, the present application provides a data processing device, which includes a processor and a memory, wherein the processor and the memory are coupled; the processor is used to implement the method as described in the first aspect.

[0016] In a fifth aspect, the present application provides a data processing device, which includes a processor, a memory and a transceiver, and the processor and the memory are coupled; the transceiver is used to send and receive data, and the processor is used to implement the method as in the first aspect.

[0017] In a sixth aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the data processing method in any possible implementation of the first aspect.

[0018] In a seventh aspect, the present application provides a chip comprising a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method of the first aspect is executed.

[0019] In an eighth aspect, the present application provides a data processing system, which includes an electronic device; wherein the electronic device is used to execute the method of the first aspect.

[0020] In a ninth aspect, the present application provides a data processing device, which includes functions or units for executing any one of the methods in the first aspect.

[0021] In the tenth aspect, the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on a data processing device, the data processing device executes the data processing method in any possible implementation of the first aspect above.

[0022] In an eleventh aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the data processing method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1A is a schematic diagram of an algorithm path A provided in an embodiment of the present application;

[0024] Figure 1B is a schematic diagram of an algorithm path B provided in an embodiment of the present application;

[0025] Figure 2 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0026] Figure 3 It is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0027] Figure 4A It is a flowchart of a data processing method provided in an embodiment of the present application;

[0028] Figure 4B It is a flowchart of a dynamic calling processor provided by an embodiment of the present application;

[0029] Figure 5 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0030] Fig. 6A is a schematic diagram of a flow chart for generating a first algorithm path provided by an embodiment of the present application;

[0031] Figure 6B It is a schematic diagram of a first order, a third order and a first algorithm path corresponding to an algorithm module provided in an embodiment of the present application;

[0032] Figure 6C is a schematic diagram of a first order, a third order and a first algorithm path corresponding to another algorithm module provided in an embodiment of the present application;

[0033] Fig. 7A is another schematic diagram of a flow chart for generating a first algorithm path provided in an embodiment of the present application;

[0034] Figure 7B It is a schematic diagram of a second order, a fourth order and a first algorithm path corresponding to an algorithm module provided in an embodiment of the present application;

[0035] Figure 7C is a schematic diagram of a second order, a fourth order and a first algorithm path corresponding to another algorithm module provided in an embodiment of the present application;

[0036] Fig. 8A is a schematic diagram of determining a processor based on scene information provided by an embodiment of the present application;

[0037] Figure 8B is a schematic diagram of determining a processor based on system status information provided by an embodiment of the present application;

[0038] Figure 8C It is a flowchart of determining a processor based on system status information and scene information provided by an embodiment of the present application;

[0039] Fig.8D is a schematic diagram of determining priority information corresponding to a first image based on system status information and scene information provided by an embodiment of the present application;

[0040] Fig. 9 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0041] Fig.10 It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0043] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0044] The term "user interface (UI)" in the following embodiments of the present application refers to the medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. A control, also called a widget, is a basic element of the user interface. Typical controls include a toolbar, menu bar, text box, button, scrollbar, picture and text. The properties and contents of the controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>The controls contained in the interface are specified by nodes such as . A node corresponds to a control or attribute in the interface. After parsing and rendering, the node is presented as user-visible content. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hypertext markup language (GTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code. For example, GTML is defined by 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0045] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0046] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application:

[0047] 1. Central processing unit (CPU), graphics processing unit (GPU)

[0048] The CPU is one of the most important components in a computer system, responsible for executing the instructions of computer programs and controlling various operations of the computer. As the computing and control core of a computer system, it is the final execution unit for information processing and program operation. Since its creation, the CPU has made great progress in logical structure, operating efficiency, and functional extension.

[0049] GPU is a processor specially designed for processing computer graphics and images. It can accelerate computer graphics rendering and processing operations, and improve the performance and quality of computer graphics and images. GPU has more processing units and higher parallel processing capabilities, so it can process large amounts of graphics and image data faster. The main functions of GPU include graphics rendering, image processing, computing acceleration, etc. In the fields of games, animation, visual effects, etc., GPU is a necessary component for achieving high-quality graphics and images. At the same time, in the fields of scientific computing, deep learning, etc., GPU can also be used as a computing accelerator, which can greatly improve computing speed and efficiency.

[0050] Although CPU and GPU are both processing devices, they work differently. CPU mainly uses a single processing unit to handle various computing tasks, such as browsing the web, running applications and managing files. GPU uses thousands of small processing units to handle a large number of computing tasks, such as drawing graphics, rendering animations frame by frame and processing video encoding. Since GPU needs to process a large amount of graphics and video data, it requires more processing units to achieve it, which makes GPU power consumption higher than CPU; in addition, because GPU has efficient parallel processing capabilities and can handle multiple tasks at the same time, GPU is more efficient and time-saving than CPU when processing large amounts of data.

[0051] 2. Image data format

[0052] Image data formats are specific data types used to store and transmit image data. There are many common image data formats according to different needs and uses. Some common image data formats are introduced below:

[0053] (1) RAW format: RAW format image data is the original data output by the imaging element (sensor), generally available in raw8, raw10, raw12, etc., indicating that one pixel has 8-bit, 10-bit, and 12-bit data respectively. It can be considered as the original record of the level when the imaging element converts the light signal into an electrical signal. It is simply image data without any processing, that is, it is obtained by digital processing of the electrical signal directly obtained by the imaging element.

[0054] (2) JPEG (joint photographic experts group) format: JPEG format is a widely used lossy compression format suitable for storing photos and color images. It can reduce the file size while maintaining relatively high image quality.

[0055] (3) PNG (portable network graphics) format: PNG format is a lossless compression format suitable for storing icons, transparent images, and simple graphics. It supports transparency and higher image quality, but the file size is usually larger than the JPEG format.

[0056] (4) YUV format: YUV format is a color encoding method, which refers to a pixel format in which the brightness parameter and the chrominance parameter are expressed separately. Among them, "Y" represents brightness (Luma), which is the grayscale value; and "U" and "V" represent chrominance (Chrominance or Chroma). It is mainly used in television systems and analog video fields. It separates brightness information (Y) from color information (UV). It can display a complete image without UV information, but it is black and white. This design solves the compatibility problem between color TVs and black and white TVs. In addition, unlike RGB, YUV does not require three independent video signals to be transmitted simultaneously. Through some compression methods, YUV reduces the occupied bandwidth a lot when it is transmitted in YUV mode.

[0057] (5) BMP (bitmap) format: BMP format is an uncompressed image format suitable for storing bitmap images. It can maintain the original quality of the image, but the file size is usually larger.

[0058] In addition to the image data formats mentioned above, there are many other formats, such as TIFF (tagged image file format), SVG (scalable vector graphics) and many other formats. Each format has its own applicable scenarios and characteristics. Choosing a suitable image data format can be determined based on the image content, purpose and requirements.

[0059] 3. Algorithm Path

[0060] An algorithmic path is a path for data to be transmitted between one or more algorithmic modules (plungins). In the process of image data processing, an algorithmic path is usually created. Figure 1A As shown, Figure 1A A schematic diagram of an algorithm path A is given, and the algorithm path A includes an algorithm module 1 and an algorithm module 2. Among them, the data format of the source image data is format A, and the data format of the target image data is format A; the data format supported by the algorithm module 1 is format A, and the data format supported by the algorithm module 2 is format A and B. The source image data is first processed by the algorithm module 1 in the algorithm path A to obtain image data 1, and the data format of image data 1 is format A; since the data formats supported by the algorithm module 2 are formats A and B, there is no need to convert the format of image data 1, and it can be directly passed to the algorithm module 2 for processing to obtain image data 2, and the data format of image data 2 is format A; since the data format of the target image data is required to be format A, there is no need to convert the format of image data 2, and the final target image data is directly output.

[0061] However, the types of algorithms are constantly increasing, the image data formats supported by internal algorithm modules are becoming more and more diverse, and the correlation and dependency between algorithm modules are also increasing, resulting in increasingly complex algorithm pathways in different scenarios.

[0062] like Figure 1B As shown, Figure 1B A schematic diagram of an algorithm path B is given, in which the algorithm path A includes algorithm module 1, algorithm module 2, algorithm module 3, algorithm module 4, algorithm module 5 and algorithm module 6. Among them, the data format of the source image data is format A, and the data format of the target image data is format C; the data format supported by algorithm module 1 is format A, the data format supported by algorithm module 2 is format A and B, the data format supported by algorithm module 3 is format B and format C, the data format supported by algorithm module 4 is format A, the data format supported by algorithm module 5 is format B, and the data format supported by algorithm module 6 is format C; and, algorithm module 3 depends on the output of algorithm module 2 (that is, it is necessary to ensure that algorithm module 2 is in front of algorithm module 3).

[0063] The source image data is first processed by algorithm module 1 in algorithm path B to obtain image data 1, and the data format of image data 1 is format A; since the data formats supported by algorithm module 2 are formats A and B, there is no need to convert the format of image data 1, and it can be directly passed to algorithm module 2 for processing to obtain image data 2, and the data format of image data 2 is format A; since the data formats supported by algorithm module 3 are formats B and C, it is necessary to convert the format of image data 2 and pass it to algorithm module 3 for processing to obtain image data 3, and the data format of image data 3 is format B; since the data format supported by algorithm module 4 is format A, it is necessary to convert the format of image data 2 After data 3 is format-converted, it is passed to algorithm module 4 for processing to obtain image data 4, and the data format of image data 4 is format A; since the data format supported by algorithm module 5 is format B, image data 4 needs to be format-converted and passed to algorithm module 5 for processing to obtain image data 5, and the data format of image data 5 is format B; since the data format supported by algorithm module 6 is format C, image data 5 needs to be format-converted to obtain image data 6, and the data format of image data 6 is format C; since the data format of the target image data is required to be format C, there is no need to format-convert image data 6, and the final target image data is directly output. In general, the image data format is converted four times in algorithm path B.

[0064] It can be seen that with the continuous increase in the types of algorithms, the diversity of image data formats supported by internal algorithm modules and the correlation between algorithm modules will cause the algorithm path to become more and more complex. Moreover, with the increase in the number of image data format conversions, power consumption will increase, reducing system performance.

[0065] In order to be able to adaptively optimize the algorithm path, which is conducive to reducing power consumption and improving system performance, the present application provides a data processing method and an electronic device. In a specific implementation, the above-mentioned data processing method can be performed by an electronic device 100. Among them, the electronic device 100 can be a mobile phone, a tablet computer, or a wearable electronic device with wireless communication function (such as a smart watch), etc., but is not limited to this. The electronic device 100 is configured with a display screen and can be installed with a preset application (application, APP), such as a camera APP, etc., which is not limited here. For example, a user can take an image or preview an image through the camera APP on the display screen.

[0066] The hardware structure of the electronic device 100 is introduced below. Figure 2 , Figure 2 It is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0067] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0068] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0069] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0070] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0071] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. The processor 110 calls the instructions or data stored in the memory, so that the electronic device 100 executes the data processing method executed by the electronic device in the following method embodiment.

[0072] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0073] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0074] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110.

[0075] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0076] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0077] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0078] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0079] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0080] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0081] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0082] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0083] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs.

[0084] NPU is a neural-network (NN) computing processor. It can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0085] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0086] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area may store data (such as image data) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc.

[0087] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0088] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0089] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The receiver 170B, also called a "earpiece", is used to convert an audio electrical signal into a sound signal. The microphone 170C, also called a "microphone", is used to convert a sound signal into an electrical signal. The headphone jack 170D is used to connect a wired headphone. The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal.

[0090] In some embodiments, the pressure sensor 180A may be provided on the display screen 194. The gyroscope sensor 180B may be used to determine the motion posture of the electronic device 100. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The acceleration sensor 180E may detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The distance sensor 180F is used to measure the distance. The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector. The ambient light sensor 180L is used to sense the brightness of ambient light. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K, also known as a "touch panel". The touch sensor 180K may be provided on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M may obtain a vibration signal. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0091] In addition, an operating system is running on the above components. For example, operating systems such as iOS and Android. The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 100. It should be noted that although the embodiment of the present application is described by taking the Android system as an example, its basic principles are also applicable to electronic devices of other operating systems.

[0092] Figure 3 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The software structure adopts a layered architecture, which divides the system into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom, namely, the application layer (application, APP), the application framework layer (framework, FWK), the hardware abstraction layer (hardware abstraction layer, HAL), the driver layer and the hardware layer.

[0093] The application layer may include a series of application packages. Figure 3 As shown, the application package may include applications such as camera, gallery, etc.

[0094] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a camera access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and a programming framework for camera applications.

[0095] The hardware abstraction layer is an interface layer between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. Figure 3 As shown, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0096] Among them, the camera hardware abstraction layer can provide virtual hardware of camera device 1, camera device 2 or more camera devices. Taking the virtual hardware of camera device 1 as an example, the virtual hardware of camera device 1 includes an algorithm path (pipeline), which includes one or more nodes (node, which can also be called a plug-in or the smallest entity unit for processing data). The so-called algorithm path can be regarded as a pipeline for managing one or more nodes; it is a way of task processing, which realizes a complex workflow by dividing the task into multiple stages (i.e., nodes) and processing them in sequence. The so-called node is each node (plug-in) in the pipeline, which is used to process and transmit data. The pipeline is a whole composed of multiple nodes, which is used to complete a complete task. In the embodiment of the present application, the node here can be a perception engine, a processing engine, etc. Specifically, the perception engine can obtain the configuration information of the image, and the configuration information of the image can include image data, scene information, data format information, algorithm association information, system status information, etc.; the processing engine can dynamically load the pipeline, dynamically call the processor, etc. It should be noted that the node can be processed before the target image data is generated, or it can be processed before the target image data is displayed, which is not limited here.

[0097] The camera algorithm library may include operating codes and data for implementing the data processing method provided in the embodiments of the present application.

[0098] The driver layer is a layer between hardware and software. The driver layer includes drivers for various hardware. The driver layer may include camera device drivers, digital signal processor drivers, and image processor drivers.

[0099] The camera device driver is used to drive the camera sensor to collect images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0100] The hardware layer includes sensors, image signal processors, digital signal processors, image processors, etc. Among them, the sensors may include sensor 1, sensor 2, time of flight (TOF) sensors, multispectral sensors, etc. A TOF sensor is a sensor that measures the flight time of light and the distance to an object, and is also called a TOF camera or a 3D camera. A multispectral sensor is a device used to sense energy in multiple wavelength ranges such as infrared, visible light, and ultraviolet light. It can detect the reflectivity and absorptivity of light of different wavelengths, obtain information about the target substance, and then analyze and process it.

[0101] In combination with the above system structure, a data processing method provided by an embodiment of the present application is described in detail below:

[0102] like Figure 4A As shown, the perception engine statically reads the configuration information of the first image, and the configuration information includes the first image data, data format information, algorithm association information, scene information and system status information.

[0103] The first image data refers to the data of the first image, that is, the data of the first image itself, which can be considered as a set of grayscale values ​​of each pixel (pixel) represented by numerical values.

[0104] The scene information may include multiple algorithm modules (algorithm types), module information, modes, sub-modes, etc. corresponding to the first image. The algorithm module (algorithm type) may be a beauty algorithm module, a filter algorithm module, a correction algorithm module, etc., which are not limited here; the module information may be parameter information corresponding to the lens, sensor, soft board, image processing chip, etc.; the mode may be a photo mode, a preview mode, a portrait mode, etc., and the sub-modes of the photo mode include a normal mode, a professional mode, etc.

[0105] The data format information may include the data format of the first image data (ie, the data format of the source image data), the data format of the second image data (ie, the data format of the target image data), and image data formats supported by multiple algorithm modules.

[0106] Algorithm association information is used to indicate the dependency between algorithms, for example, algorithm module 3 depends on the output of algorithm module 2. Specifically, one algorithm module may be associated with another algorithm module, one algorithm module may be associated with multiple algorithm modules, or multiple algorithm modules may be associated with one algorithm module, which is not limited here.

[0107] The system status information is used to indicate the current status of the system, such as the working status of the GPU, the working status of the CPU, the system power consumption (power consumption can be understood as power consumption), the system time consumption (time consumption can be understood as time consumption), etc.

[0108] Then, the processing engine dynamically loads the first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information (i.e., generates a pipeline link, including pipeline configuration and loading); the first algorithm path is the transmission path of the first image data between the multiple algorithm modules. Specifically, please refer to the following Figure 5 The description in will not be repeated here.

[0109] Finally, the processing engine uses the conversion module to dynamically call the processor (e.g., CPU, GPU) in the first algorithm path to process the first image data to obtain the second image data (i.e., the target image data). Figure 4B As shown, the perception engine can confirm the priority information based on the scene information and / or system status information (for example, the priority of system power consumption is higher than the priority of system time consumption, or the priority of system time consumption is higher than the priority of system power consumption); the processing engine then uses the conversion module to dynamically select the processor to be called according to the priority information.

[0110] Based on the above, the electronic device is regarded as the execution subject, and another data processing method provided by the embodiment of the present application is further described in detail. Figure 5 As shown, the data processing method includes the following steps S501 to S503. Figure 5 The method execution subject shown can be the electronic device mentioned above. Or, Figure 5 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 5 The method is described by taking an electronic device as an example of an execution subject.

[0111] S501. An electronic device obtains configuration information of a first image, where the configuration information includes first image data, data format information, algorithm association information, and scene information; the scene information includes multiple algorithm modules corresponding to the first image.

[0112] In the embodiment of the present application, the user can take photos, preview photos, etc. through the camera APP in the electronic device. Taking taking photos as an example, after the user takes a first image through the camera APP in the electronic device, the electronic device will obtain the configuration information of the first image, which may include the first image data, data format information, algorithm association information and scene information, and may also include system status information, etc., which is not limited here. Specifically, the perception engine in the electronic device may statically read the configuration information of the first image.

[0113] In a possible implementation manner, the data format information includes a data format of the first image data, a data format of the second image data, and an image data format supported by each algorithm module in the multiple algorithm modules.

[0114] Specifically, various information included in the configuration information of the first image is described below.

[0115] (1) The first image data may refer to the data of the first image, that is, the data of the first image itself, which may be considered as a set of grayscale values ​​of each pixel (pixel) represented by numerical values.

[0116] (2) The scene information may be used to indicate the scene attributes of the first image, and may include multiple algorithm modules (algorithm types), module information, modes, sub-modes, etc. corresponding to the first image, which are not limited here. The algorithm module (algorithm type) may be a beauty algorithm module, a filter algorithm module, a correction algorithm module, etc., which are not limited here; the module information may be parameter information corresponding to a lens, a sensor, a soft board, an image processing chip, etc.; the mode may be a photo mode, a preview mode, etc., and the photo mode may also include sub-modes such as a normal mode, a professional mode, and a portrait mode, which are not limited here.

[0117] (3) The data format information may include the data format of the first image data (i.e., the data format of the source image data), the data format of the second image data (i.e., the data format of the target image data), and the image data formats supported by multiple algorithm modules. Among them, the image data format (i.e., the data format of the image data) may be the RAW format, JPEG format, PNG format, YUV format, BMP format, etc. mentioned above, which are not limited here. The data format of the second image data may be the data format that the user sets to be generated, or it may be the data format generated by default according to the current shooting mode, which are not limited here. In addition, since the image data formats supported by the algorithm modules are diverse, for the first image, the image data formats supported by the multiple algorithm modules contained in the scene information may be the same or different.

[0118] For example, it is assumed that the multiple algorithm modules include algorithm module 1, algorithm module 2, algorithm module 3, algorithm module 4, algorithm module 5, and algorithm module 6. The data format information may include: the data format of the first image data is format A, the data format of the second image data is format C, and the image data format supported by algorithm module 1 is format A, the image data format supported by algorithm module 2 is format A and B, the image data format supported by algorithm module 3 is format B and format C, the image data format supported by algorithm module 4 is format A, the image data format supported by algorithm module 5 is format B, and the image data format supported by algorithm module 6 is format C.

[0119] (4) Algorithm association information is used to indicate the dependency between algorithms, for example, algorithm module 3 depends on the output of algorithm module 2, or, it is necessary to ensure that algorithm module 2 is in front of algorithm module 3. Specifically, one algorithm module may be associated with another algorithm module, one algorithm module may be associated with multiple algorithm modules, or multiple algorithm modules may be associated with one algorithm module, which is not limited here. It should be noted that the algorithm association information may be pre-set by the developer when configuring the algorithm module, or may be determined by the electronic device by analyzing the internal structure of the algorithm module, or may be determined by other methods, which is not limited here.

[0120] (5) System status information is used to indicate the current status of the system, such as the working status of the GPU, the working status of the CPU, the system power consumption, the system time consumption, etc., which are not limited here.

[0121] S502. The electronic device generates a first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information; the first algorithm path is a transmission path for the first image data between the multiple algorithm modules.

[0122] In an embodiment of the present application, after the electronic device obtains the multiple algorithm modules corresponding to the first image, the data format information and the algorithm association information, it can determine the data format requirements of the first algorithm path and the image data format supported by each of the multiple algorithm modules according to the data format information, and can also determine the dependencies between the algorithm modules according to the algorithm association information, sort the multiple algorithm modules in combination with these requirements and dependencies, and finally generate the first algorithm path corresponding to the first image. In addition, if there are multiple sorting methods, the final sorting method can also be selected in combination with the number of image format conversions between the multiple algorithm modules (for example, the sorting method with the lowest number of image format conversions can be selected), so as to obtain the optimal first algorithm path. Specifically, the processing engine of the electronic device can dynamically load the first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information.

[0123] In a possible implementation, when the electronic device generates the first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information, the specific implementation may adopt any one of the following two methods. The two methods are described in detail below. Of course, the electronic device may also adopt other methods to generate the first algorithm path, which is not limited here.

[0124] Method 1: The electronic device first sorts the multiple algorithm modules according to the data format information, and then adjusts the first sequence according to the algorithm association information. Fig. 6A As shown, the specific steps include the following steps s11 to s13. Based on this method, the best algorithm path can be determined to achieve optimization of the algorithm path.

[0125] s11. The electronic device sorts the multiple algorithm modules based on the data format information to obtain a first sequence.

[0126] s12. The electronic device adjusts the first sequence based on the algorithm association information to obtain a third sequence.

[0127] s13. The electronic device processes the multiple algorithm modules according to the third sequence to generate a first algorithm path corresponding to the first image.

[0128] Exemplarily, it is assumed that the multiple algorithm modules include: algorithm module 1, algorithm module 2, algorithm module 3, algorithm module 4, algorithm module 5 and algorithm module 6. The data format information includes: the data format of the first image data (source image data) is format A, and the data format of the second image data (target image data) is format C; the data format supported by algorithm module 1 is format A, the data format supported by algorithm module 2 is format A and B, the data format supported by algorithm module 3 is format B and format C, the data format supported by algorithm module 4 is format A, the data format supported by algorithm module 5 is format B, and the data format supported by algorithm module 6 is format C. The algorithm association information includes: algorithm module 3 depends on the output of algorithm module 2 (that is, it is necessary to ensure that algorithm module 2 is in front of algorithm module 3).

[0129] First, if Figure 6B As shown in (a), the data formats supported by each algorithm module in the data format information are classified. Since the data format of the first image data (source image data) is format A and the data format of the second image data (target image data) is format C, the multiple algorithm modules can be classified and sorted according to format A, format B and format C to obtain the first order. For example, Figure 6B The first order shown in (a) (algorithm module 1, algorithm module 4, algorithm module 3, algorithm module 2, algorithm module 5 and algorithm module 6, respectively) is of course just an example, and there may be other first orders, which are not limited here.

[0130] Then, if Figure 6B As shown in (b), according to the algorithm association information, it is known that the algorithm module 2 needs to be in front of the algorithm module 3, so the first order needs to be further adjusted to obtain the third order. For example, it can be obtained Figure 6B The third order shown in (b) (algorithm module 1, algorithm module 4, algorithm module 2, algorithm module 5, algorithm module 3 and algorithm module 6, respectively) is of course just an example, and there may be other third orders, which are not limited here.

[0131] Finally, if Figure 6B As shown in (c), the electronic device processes the multiple algorithm modules according to the third sequence, generates the first algorithm path corresponding to the first image (i.e., generates a pipeline link), and implements the configuration and loading of the first algorithm path corresponding to the first image. It can be seen that the image data format will only be converted twice in the first algorithm path, that is, when algorithm module 2 transfers data to algorithm module 5, the image data format will be converted once (format A is converted to format B), and when algorithm module 3 transfers data to algorithm module 6, the image data format will be converted once (format B is converted to format C). Compared to Figure 1B For algorithm path B in the example, the first algorithm path reduces the conversion of image data formats twice, which is beneficial to reducing system power consumption, improving system performance, and realizing adaptive optimization of the algorithm path.

[0132] Optionally, there are multiple third sequences, and before the electronic device processes the multiple algorithm modules according to the third sequence to generate a first algorithm path corresponding to the first image, the method also includes: determining a target sequence from the multiple third sequences; the target sequence is a sequence with the lowest number of image format conversions between multiple algorithm modules in the multiple third sequences; when the electronic device processes the multiple algorithm modules according to the third sequence to generate the first algorithm path corresponding to the first image, a specific implementation method may be: the electronic device processes the multiple algorithm modules according to the target sequence to generate the first algorithm path corresponding to the first image.

[0133] Exemplarily, it is assumed that the multiple algorithm modules include: algorithm module 1, algorithm module 2, algorithm module 3, algorithm module 4, algorithm module 5 and algorithm module 6. The data format information includes: the data format of the first image data (source image data) is format A, and the data format of the second image data (target image data) is format C; the data format supported by algorithm module 1 is format A, the data format supported by algorithm module 2 is format A and B, the data format supported by algorithm module 3 is format B and format C, the data format supported by algorithm module 4 is format A, the data format supported by algorithm module 5 is format B, and the data format supported by algorithm module 6 is format C. The algorithm association information includes: algorithm module 3 depends on the output of algorithm module 2 (that is, it is necessary to ensure that algorithm module 2 is in front of algorithm module 3).

[0134] First, if Figure 6C As shown in (a), the data formats supported by each algorithm module in the data format information are classified. Since the data format of the first image data (source image data) is format A and the data format of the second image data (target image data) is format C, the multiple algorithm modules can be classified and sorted according to format A, format B and format C to obtain the first order. For example, Figure 6B The first order shown (algorithm module 1, algorithm module 4, algorithm module 3, algorithm module 2, algorithm module 5 and algorithm module 6 in sequence) is of course just an example, and there may be other first orders, which are not limited here.

[0135] Then, if Figure 6C As shown in (b), according to the algorithm association information, it is known that the algorithm module 2 needs to be in front of the algorithm module 3, so the first order needs to be further adjusted to obtain two third orders. For example, it can be obtained Figure 6C The third order 1 shown in (b) (algorithm module 1, algorithm module 4, algorithm module 2, algorithm module 5, algorithm module 3 and algorithm module 6 in sequence) and the third order 2 (algorithm module 1, algorithm module 4, algorithm module 2, algorithm module 6, algorithm module 5 and algorithm module 3 in sequence). Of course, this is just an example, and there may be other ways of the third order, which is not limited here.

[0136] Further, the electronic device determines the sequence with the lowest number of image format conversions between the multiple algorithm modules from the two third sequences, that is, the target sequence. For example, the number of image format conversions between the multiple algorithm modules in the third sequence 1 is 2 times, and the number of image format conversions between the multiple algorithm modules in the third sequence 2 is 3 times. It can be seen that the number of image format conversions between the multiple algorithm modules in the third sequence 1 is the lowest, and therefore, the third sequence 1 is determined as the target sequence.

[0137] Finally, if Figure 6C As shown in (c), the electronic device processes the multiple algorithm modules according to the target sequence (i.e., the third sequence 1), generates the first algorithm path corresponding to the first image (i.e., generates a pipeline link), and implements the configuration and loading of the first algorithm path corresponding to the first image. It can be seen that the image data format will only be converted twice in the first algorithm path, that is, when algorithm module 2 transfers data to algorithm module 5, the image data format will be converted once (format A is converted to format B), and when algorithm module 6 outputs, the image data format will be converted once (format B is converted to format C). Compared to Figure 1B For algorithm path B in the example, the first algorithm path reduces the conversion of image data formats twice, which is beneficial to reducing system power consumption, improving system performance, and realizing adaptive optimization of the algorithm path.

[0138] Further optionally, the scene information also includes module information corresponding to the first image and / or the mode adopted by the first image; the configuration information also includes system status information; when the electronic device determines the target sequence from multiple third sequences, the specific implementation method may be: the electronic device determines the target sequence from multiple third sequences based on the scene information and / or the system status information.

[0139] It can be understood that it is necessary not only to determine the sequence with the lowest number of image format conversions between multiple algorithm modules from multiple third sequences, but also to screen out the final target sequence in combination with the scene information and / or the system state information. Based on this approach, it is more conducive to improving system performance.

[0140] For example, it is assumed that the electronic device determines three third sequences, namely, third sequence a, third sequence b, and third sequence c. Among them, the number of image format conversions corresponding to the third sequence a and the number of image format conversions corresponding to the third sequence b are both the lowest, the algorithm modules in the third sequence a are algorithm module 1, algorithm module 3, and algorithm module 2, respectively, and the algorithm modules in the third sequence b are algorithm module 2, algorithm module 1, and algorithm module 3, respectively. At this time, the scene information of the first image indicates that the mode used by the first image is the camera shooting mode. In the camera shooting mode, algorithm module 2 is preferentially used for better processing effect. Therefore, the third sequence b is determined as the target sequence in combination with the current scene information.

[0141] For another example, assume that the electronic device determines three third sequences, namely, third sequence a, third sequence b, and third sequence c. Among them, the number of image format conversions corresponding to the third sequence a and the number of image format conversions corresponding to the third sequence b are both the lowest, the algorithm modules in the third sequence a are algorithm module 1, algorithm module 3, and algorithm module 2, respectively, and the algorithm modules in the third sequence b are algorithm module 2, algorithm module 1, and algorithm module 3, respectively. At this time, the system status information of the first image indicates that the system power consumption is greater than the preset threshold. When the system power consumption is greater than the preset threshold, the algorithm module 3 is finally used for processing, and the effect is better. Therefore, the third sequence b is determined as the target sequence in combination with the current system status information.

[0142] For another example, assume that the electronic device determines three third sequences, namely, third sequence a, third sequence b, and third sequence c. Among them, the number of image format conversions corresponding to the third sequence a and the number of image format conversions corresponding to the third sequence b are both the lowest, the algorithm modules in the third sequence a are algorithm module 1, algorithm module 3, and algorithm module 2, respectively, and the algorithm modules in the third sequence b are algorithm module 2, algorithm module 1, and algorithm module 3, respectively. At this time, the scene information of the first image indicates that the mode adopted by the first image is the camera shooting mode, and in the camera shooting mode, algorithm module 2 is preferentially used for processing with better effect; at this time, the system status information of the first image indicates that the system power consumption is greater than the preset threshold, and when the system power consumption is greater than the preset threshold, algorithm module 3 is finally used for processing with better effect, so the third sequence b is determined as the target sequence in combination with the current scene information and the current system status information.

[0143] Of course, determining the target sequence from multiple third sequences based on the scene information and / or the system status information can also be achieved in other ways. For example, the weight values ​​of the scene information and the system status information in each third sequence can be calculated, and the final target sequence can be selected according to the weight values, etc., which is not limited here.

[0144] Method 2: The electronic device first sorts the multiple algorithm modules according to the algorithm association information, and then adjusts the first sequence according to the data format information. Fig. 7A As shown, the specific steps include the following steps s21 to s23. Based on this method, the best algorithm path can be determined to achieve optimization of the algorithm path.

[0145] s21. The electronic device sorts the multiple algorithm modules based on the algorithm association information to obtain a second sequence.

[0146] s22. The electronic device adjusts the second sequence based on the data format information to obtain a fourth sequence.

[0147] s23. The electronic device processes the multiple algorithm modules according to the fourth sequence to generate a first algorithm path corresponding to the first image.

[0148] Exemplarily, it is assumed that the multiple algorithm modules include: algorithm module 1, algorithm module 2, algorithm module 3, algorithm module 4, algorithm module 5 and algorithm module 6. The data format information includes: the data format of the first image data (source image data) is format A, and the data format of the second image data (target image data) is format C; the data format supported by algorithm module 1 is format A, the data format supported by algorithm module 2 is format A and B, the data format supported by algorithm module 3 is format B and format C, the data format supported by algorithm module 4 is format A, the data format supported by algorithm module 5 is format B, and the data format supported by algorithm module 6 is format C. The algorithm association information includes: algorithm module 3 depends on the output of algorithm module 2 (that is, it is necessary to ensure that algorithm module 2 is in front of algorithm module 3).

[0149] First, if Figure 7B As shown in (a), according to the algorithm association information, it is known that algorithm module 2 needs to be in front of algorithm module 3, and the multiple algorithm modules are preliminarily sorted to obtain a second sequence. For example, it can be obtained Figure 7B The second order shown in (a) (algorithm module 1, algorithm module 5, algorithm module 2, algorithm module 3, algorithm module 4 and algorithm module 6, respectively) is of course just an example, and there may be other second orders, which are not limited here.

[0150] Then, if Figure 7B As shown in (b), the adjustment is made according to the data formats supported by each algorithm module in the data format information. Specifically, the adjustment can be made according to the format type. Since the data format of the first image data (source image data) is format A and the data format of the second image data (target image data) is format C, the second order can be adjusted according to format A, format B and format C to obtain the fourth order (algorithm module 1, algorithm module 4, algorithm module 2, algorithm module 5, algorithm module 3 and algorithm module 6, respectively). Of course, this is just an example, and there can be other fourth orders, which are not limited here.

[0151] Finally, if Figure 7B As shown in (c), the electronic device processes the multiple algorithm modules according to the fourth sequence, generates the first algorithm path corresponding to the first image (i.e., generates a pipeline link), and implements the configuration and loading of the first algorithm path corresponding to the first image. It can be seen that in the first algorithm path, only two image data format conversions are performed, that is, when algorithm module 2 transfers data to algorithm module 5, an image data format conversion is performed once (format A is converted to format B), and when algorithm module 3 transfers data to algorithm module 6, an image data format conversion is performed once (format B is converted to format C). Compared to Figure 1B For algorithm path B in the example, the first algorithm path reduces the conversion of image data formats twice, which is beneficial to reducing system power consumption, improving system performance, and realizing adaptive optimization of the algorithm path.

[0152] Optionally, there are multiple fourth sequences, and before the electronic device processes the multiple algorithm modules according to the fourth sequence to generate a first algorithm path corresponding to the first image, the method also includes: determining a target sequence from the multiple fourth sequences; the target sequence is a sequence with the lowest number of image format conversions between multiple algorithm modules in the multiple fourth sequences; when the electronic device processes the multiple algorithm modules according to the fourth sequence to generate the first algorithm path corresponding to the first image, a specific implementation method may be: the electronic device processes the multiple algorithm modules according to the target sequence to generate the first algorithm path corresponding to the first image.

[0153] Exemplarily, it is assumed that the multiple algorithm modules include: algorithm module 1, algorithm module 2, algorithm module 3, algorithm module 4, algorithm module 5 and algorithm module 6. The data format information includes: the data format of the first image data (source image data) is format A, and the data format of the second image data (target image data) is format C; the data format supported by algorithm module 1 is format A, the data format supported by algorithm module 2 is format A and B, the data format supported by algorithm module 3 is format B and format C, the data format supported by algorithm module 4 is format A, the data format supported by algorithm module 5 is format B, and the data format supported by algorithm module 6 is format C. The algorithm association information includes: algorithm module 3 depends on the output of algorithm module 2 (that is, it is necessary to ensure that algorithm module 2 is in front of algorithm module 3).

[0154] First, if Figure 7C As shown in (a), according to the algorithm association information, it is known that algorithm module 2 needs to be in front of algorithm module 3, and the multiple algorithm modules are preliminarily sorted to obtain a second sequence. For example, it can be obtained Figure 7C The second order shown in (a) (algorithm module 1, algorithm module 5, algorithm module 2, algorithm module 3, algorithm module 4 and algorithm module 6, respectively) is of course just an example, and there may be other second orders, which are not limited here.

[0155] Then, if Figure 7C As shown in (b), the adjustment is made according to the data formats supported by each algorithm module in the data format information. Specifically, it can be classified according to the format type. Therefore, the second order can be adjusted according to format A, format B and format C to obtain two fourth orders, namely fourth order 1 (algorithm module 1, algorithm module 4, algorithm module 2, algorithm module 5, algorithm module 3 and algorithm module 6, respectively) and fourth order 2 (algorithm module 1, algorithm module 4, algorithm module 2, algorithm module 6, algorithm module 5 and algorithm module 3, respectively). Of course, this is just an example, and there may be other fourth orders, which are not limited here.

[0156] Further, the electronic device determines the sequence with the lowest number of image format conversions between the multiple algorithm modules from the two fourth sequences, that is, the target sequence. For example, the number of image format conversions between the multiple algorithm modules in the fourth sequence 1 is 2 times, and the number of image format conversions between the multiple algorithm modules in the fourth sequence 2 is 3 times. It can be seen that the number of image format conversions between the multiple algorithm modules in the fourth sequence 1 is the lowest, and therefore, the fourth sequence 1 is determined as the target sequence.

[0157] Finally, if Figure 7C As shown in (c), the electronic device processes the multiple algorithm modules according to the target sequence (i.e., the fourth sequence 1), generates the first algorithm path corresponding to the first image (i.e., generates a pipeline link), and implements the configuration and loading of the first algorithm path corresponding to the first image. It can be seen that the image data format will only be converted twice in the first algorithm path, that is, when algorithm module 2 transfers data to algorithm module 5, the image data format will be converted once (format A is converted to format B), and when algorithm module 6 outputs, the image data format will be converted once (format B is converted to format C). Compared to Figure 1B For algorithm path B in the example, the first algorithm path reduces the conversion of image data formats twice, which is beneficial to reducing system power consumption, improving system performance, and realizing adaptive optimization of the algorithm path.

[0158] Further optionally, the scene information also includes module information corresponding to the first image and / or the mode adopted by the first image; the configuration information also includes system status information; when the electronic device determines the target sequence from multiple fourth sequences, the specific implementation method may be: the electronic device determines the target sequence from multiple fourth sequences based on the scene information and / or the system status information.

[0159] It can be understood that it is necessary not only to determine the sequence with the lowest number of image format conversions between multiple algorithm modules from multiple fourth sequences, but also to screen out the final target sequence in combination with the scene information and / or the system state information. Based on this method, it is more conducive to improving system performance. For details, please refer to the description of "the electronic device determines the target sequence from multiple third sequences based on the scene information and / or the system state information" in the above method 1, which will not be repeated here.

[0160] S503: The electronic device processes the first image data using a processor in the first algorithm path to obtain second image data.

[0161] In an embodiment of the present application, the electronic device can dynamically call the processor in real time to process the first image data in the first algorithm path to obtain the second image data. The processor here can be a CPU and / or a GPU. Specifically, the processing engine of the electronic device can dynamically call the processor (e.g., CPU, GPU) in the first algorithm path using a conversion module to process the first image data to obtain the second image data (i.e., the target image data).

[0162] In a possible implementation, the electronic device may specifically select a processor in the following three ways, which are described in detail below. Of course, the electronic device may also select a processor in other ways, which are not limited here.

[0163] Method 1: The electronic device determines the processor according to scene information.

[0164] The scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the method also includes: the electronic device determines a processor based on the scene information.

[0165] It can be understood that a corresponding processor is required to perform the image format conversion operation for different scene information, so the electronic device can select a corresponding processor according to the scene information of the current first image. Based on this method, the processor can be accurately selected, the system power consumption can be reduced, and the efficiency and effectiveness of data processing can be improved.

[0166] Optionally, when the electronic device determines the processor based on the scene information, illustratively, Fig. 8A As shown, the specific implementation method may be:

[0167] (1) When the scene information indicates that the mode adopted by the first image is the camera preview mode, the electronic device determines that the processor is a GPU.

[0168] In the specific implementation, since the camera preview mode needs to ensure timeliness and reduce processing delay, the GPU can be used to process image data, which is more efficient. Of course, this is just an example, and other processors can also be set to process image data in the camera preview mode, which is not limited here.

[0169] (2) When the scene information indicates that the mode adopted by the first image is a camera shooting mode, the electronic device determines that the processor is a CPU and / or a GPU.

[0170] In a specific implementation, in the camera photo mode, the CPU can be used to process image data, the GPU can be used to process image data, or the CPU and GPU can be used together to process image data. There is no limitation here. The specific processor to be used can be determined according to user settings or developer settings, and can also be further determined in combination with other information.

[0171] Of course, the processor to be called may also be determined based on other scene information, which is not limited here.

[0172] Method 2: Determine the processor based on system status information.

[0173] The configuration information also includes system status information; the method also includes: the electronic device determines a processor based on the system status information.

[0174] It can be understood that a corresponding processor is required to perform the image format conversion operation for different system status information, so the electronic device can select a corresponding processor according to the system status information of the current first image. The system status information can be system power consumption, system time consumption, or the working state of the CPU and / or GPU, which is not limited here. Based on this method, the processor can be accurately selected, the system power consumption can be reduced, and the efficiency and effectiveness of data processing can be improved.

[0175] Optionally, when the electronic device determines the processor based on the system state information, illustratively, Figure 8B As shown, the specific implementation method may be:

[0176] (1) When the system power consumption in the system state information is less than or equal to a first preset threshold, the electronic device determines that the processor is a CPU or a GPU.

[0177] In a specific implementation, when the system power consumption in the system status information is less than or equal to the first preset threshold, it is considered that the system power consumption of the electronic device is relatively low at this time, and the image data processing can be performed using either the CPU or the GPU.

[0178] (2) When the system power consumption in the system state information is greater than a first preset threshold, the electronic device determines that the processor is a CPU.

[0179] In a specific implementation, when the system power consumption in the system status information is greater than a first preset threshold, it is considered that the system power consumption of the electronic device is relatively large. In order to reduce the system power consumption, the CPU needs to be used to perform image data processing.

[0180] (3) When the system time consumed in the system state information is less than or equal to a second preset threshold, the electronic device determines that the processor is a CPU or a GPU.

[0181] In a specific implementation, when the system time consumption in the system status information is less than or equal to the second preset threshold, it is considered that the system time consumption of the electronic device is relatively small at this time, and the image data processing can be performed using either the CPU or the GPU.

[0182] (4) When the system time consumption in the system state information is greater than a second preset threshold, the electronic device determines that the processor is a GPU.

[0183] In a specific implementation, when the system time consumption in the system status information is greater than the second preset threshold, it is considered that the system time consumption of the electronic device is large at this time. In order to reduce the system time consumption, the GPU needs to be used to perform image data processing at this time.

[0184] Of course, the processor to be called may also be determined based on other system status information, which is not limited here.

[0185] Method 3: Determine the processor based on scene information and system status information.

[0186] The scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the configuration information also includes system status information; the method also includes: the electronic device determines a processor based on the system status information and the scene information.

[0187] It can be understood that for different scene information and different system status information, a corresponding processor needs to be used to perform the image format conversion operation, so the electronic device can select a corresponding processor in combination with the scene information of the current first image and the system status information. Based on this approach, the efficiency and effectiveness of data processing are improved.

[0188] Optionally, when the electronic device determines the processor based on the system state information and the scene information, Figure 8C As shown, the specific implementation method may include the following steps s31 and s32. Based on this method, due to the combination of the priority information corresponding to the first image, the processor can be selected more accurately, the system power consumption can be reduced, and the efficiency and effectiveness of data processing can be improved.

[0189] s31. The electronic device determines the priority information corresponding to the first image based on the system status information and the scene information; the priority information is used to indicate that the priority of system power consumption is greater than the priority of system time consumption, or the priority information is used to indicate that the priority of system time consumption is greater than the priority of system power consumption.

[0190] It can be understood that the electronic device first needs to determine the priority information corresponding to the first image based on the system status information and the scene information, that is, determine whether the system status corresponding to the current first image indicates system power consumption priority or system time consumption priority. Specifically, the perception engine of the electronic device can confirm the priority information based on the system status information and the scene information (for example, the priority of system power consumption is higher than the priority of system time consumption, or the priority of system time consumption is higher than the priority of system power consumption).

[0191] Further optionally, when the electronic device determines the priority information corresponding to the first image based on the system state information and the scene information, illustratively, Fig.8D As shown, specific implementation methods may include:

[0192] (1) When the scene information indicates that the mode adopted by the first image is the camera preview mode, and the system power consumption in the system state information is less than or equal to a first preset threshold, the electronic device determines that the priority information indicates that the priority of the system time consumption is greater than the priority of the system power consumption.

[0193] In a specific implementation, in the camera preview mode, the latency is kept as low as possible, and the system power consumption is relatively small, so the priority information corresponding to the first image can be determined as system time priority.

[0194] (2) When the scene information indicates that the mode adopted by the first image is the camera preview mode, and the system power consumption in the system status information is greater than a first preset threshold, the electronic device determines that the priority information indicates that the priority of the system power consumption is greater than the priority of the system time consumption.

[0195] In a specific implementation, in the camera preview mode, low latency is ensured as much as possible, but the system power consumption is also relatively large, so it can be determined that the priority information corresponding to the first image is system power consumption first.

[0196] (3) When the scene information indicates that the mode adopted by the first image is the camera preview mode, and when the system time consumption in the system status information is less than or equal to the second preset threshold, the electronic device determines that the priority information indicates that the priority of the system time consumption is greater than the priority of the system power consumption.

[0197] In a specific implementation, in the camera preview mode, the latency is kept as low as possible, and the system time consumption is relatively small, so the priority information corresponding to the first image can be determined as system time consumption priority.

[0198] (4) When the scene information indicates that the mode adopted by the first image is the camera preview mode, and when the system time consumption in the system status information is greater than the second preset threshold, the electronic device determines that the priority information indicates that the priority of the system time consumption is greater than the priority of the system power consumption.

[0199] In a specific implementation, in the camera preview mode, the latency is kept as low as possible, and the system time consumption is relatively large, so the priority information corresponding to the first image can be determined as system time consumption priority.

[0200] (5) When the scene information indicates that the mode adopted by the first image is a camera shooting mode, and the system power consumption in the system state information is less than or equal to a first preset threshold, the electronic device determines that the priority information indicates that the priority of the system power consumption is greater than the priority of the system time consumption.

[0201] In a specific implementation, in order to ensure the photo effect as much as possible in the camera preview mode, power consumption needs to be guaranteed. Since the system power consumption is relatively small at this time, the priority information corresponding to the first image can be determined as system power consumption priority.

[0202] (6) When the scene information indicates that the mode adopted by the first image is a camera shooting mode, and the system power consumption in the system state information is greater than a first preset threshold, the electronic device determines that the priority information indicates that the priority of the system time consumption is greater than the priority of the system power consumption.

[0203] In a specific implementation, in order to ensure the photo effect as much as possible in the camera preview mode, power consumption needs to be guaranteed. However, since the system power consumption is relatively large at this time, the priority information corresponding to the first image can be determined as system time priority.

[0204] (7) When the scene information indicates that the mode adopted by the first image is a camera shooting mode, and when the system time consumption in the system status information is less than or equal to a second preset threshold, the electronic device determines that the priority information indicates that the priority of the system power consumption is greater than the priority of the system time consumption.

[0205] In a specific implementation, in order to ensure the photo effect as much as possible in the camera preview mode, power consumption needs to be guaranteed, and the system power consumption is relatively small at this time, so the priority information corresponding to the first image can be determined as system power consumption priority.

[0206] (8) When the scene information indicates that the mode adopted by the first image is a camera shooting mode, and when the system time consumption in the system state information is greater than a second preset threshold, the electronic device determines that the priority information indicates that the priority of the system power consumption is greater than the priority of the system time consumption.

[0207] In a specific implementation, in order to ensure the photo effect as much as possible in the camera preview mode, power consumption needs to be guaranteed, and the system time consumption is relatively large at this time, so the priority information corresponding to the first image can be determined as system power consumption priority.

[0208] s32. The electronic device determines a processor based on the priority information.

[0209] Further optionally, when the electronic device determines the processor based on the priority information, the specific implementation method may be: if the priority information indicates that the priority of system power consumption is greater than the priority of system time consumption, that is, system power consumption is prioritized, then the electronic device determines that the processor to be used is the CPU; if the priority information indicates that the priority of system time consumption is greater than the priority of system power consumption, that is, system time consumption is prioritized, then the electronic device determines that the processor to be used is the GPU. Of course, this is just an example, and the electronic device may also determine to use the CPU and GPU for simultaneous processing based on other information such as system power consumption and system time consumption, or may determine other processors, which are not limited here. Specifically, the processing engine of the electronic device may use the conversion module to dynamically select the processor to be called based on the priority information.

[0210] It can be seen that based on Figure 5 In the described method, the electronic device first obtains the configuration information of the first image, the configuration information includes the first image data, data format information, algorithm association information and scene information, wherein the scene information includes multiple algorithm modules corresponding to the first image; then the electronic device generates the first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information; the first algorithm path here is the transmission path of the first image data between the multiple algorithm modules; finally, the electronic device can use the processor in the first algorithm path to process the first image data to obtain the second image data, thereby completing the processing of the image data. Since the electronic device combines the data format information and the algorithm association information when creating and configuring the first algorithm path, it can better reduce the number of conversions of the image data format, thereby helping to reduce system power consumption, improve system performance, and achieve adaptive optimization of the algorithm path.

[0211] See also Fig. 9 , Fig. 9 A schematic structural diagram of a data processing device 900 according to an embodiment of the present application is shown. Fig. 9 The data processing device shown may be an electronic device, or a device in an electronic device, or a device that can be used in conjunction with an electronic device. Fig. 9 The data processing device shown may include an acquisition unit 901 and a processing unit 902. Among them:

[0212] An acquisition unit 901 is used to acquire configuration information of a first image, where the configuration information includes first image data, data format information, algorithm association information, and scene information; the scene information includes a plurality of algorithm modules corresponding to the first image;

[0213] A processing unit 902 is configured to generate a first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information; the first algorithm path is a transmission path of the first image data between the multiple algorithm modules;

[0214] The processing unit 902 is used to process the first image data using a processor in the first algorithm path to obtain second image data.

[0215] In a possible implementation, the scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the method also includes: determining a processor based on the scene information.

[0216] In one possible implementation, the scene information also includes module information corresponding to the first image and / or the mode adopted by the first image; the configuration information also includes system status information; the processing unit 902 is also used to: determine the processor based on the system status information, or determine the processor based on the system status information and the scene information.

[0217] In one possible implementation, the processing unit 902, when determining the processor based on the system status information and the scene information, is specifically used to: determine the priority information corresponding to the first image based on the system status information and the scene information; the priority information is used to indicate that the priority of the system power consumption is greater than the priority of the system time consumption, or the priority information is used to indicate that the priority of the system time consumption is greater than the priority of the system power consumption; determine the processor based on the priority information.

[0218] In a possible implementation manner, the data format information includes a data format of the first image data, a data format of the second image data, and an image data format supported by each algorithm module in the multiple algorithm modules.

[0219] In one possible implementation, the processing unit 902, when determining the first algorithm path corresponding to the first image based on the multiple algorithm modules, the data format information and the algorithm association information, is specifically used to: sort the multiple algorithm modules based on the data format information to obtain a first sequence; or, sort the multiple algorithm modules based on the algorithm association information to obtain a second sequence; adjust the first sequence based on the algorithm association information to obtain a third sequence; or, adjust the second sequence based on the data format information to obtain a fourth sequence; and process the multiple algorithm modules according to the third sequence or the fourth sequence to generate the first algorithm path corresponding to the first image.

[0220] In one possible implementation, there are multiple third sequences, or there are multiple fourth sequences; the processing unit 902, before processing the multiple algorithm modules according to the third sequence or the fourth sequence to generate the first algorithm path corresponding to the first image, is further used to: determine a target sequence from the multiple third sequences or the multiple fourth sequences; the target sequence is a sequence with the lowest number of image format conversions between the multiple algorithm modules in the multiple third sequences or the multiple fourth sequences; the processing unit 902, when processing the multiple algorithm modules according to the third sequence or the fourth sequence to generate the first algorithm path corresponding to the first image, is specifically used to: process the multiple algorithm modules according to the target sequence to generate the first algorithm path corresponding to the first image.

[0221] In a possible implementation, the scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the configuration information also includes system status information; the processing unit 902, when determining a target sequence from multiple third sequences or multiple fourth sequences, is specifically used to: determine the target sequence from multiple third sequences or multiple fourth sequences based on the scene information and / or the system status information.

[0222] For the case where the data processing device may be a chip or a chip system, see Fig.10 Schematic diagram of the chip structure shown. Fig.10 The chip 1000 shown includes a processor 1001 and an interface 1002. Optionally, it may also include a memory 1003. The number of the processors 1001 may be one or more, and the number of the interfaces 1002 may be multiple.

[0223] For the case where the chip is used to implement the electronic device in the embodiment of the present application:

[0224] The interface 1002 is used to receive or output signals;

[0225] The processor 1001 is used to execute data processing operations of the electronic device.

[0226] It is understandable that some optional features in the embodiments of the present application may be implemented independently in certain scenarios without relying on other features, such as the solution on which they are currently based, to solve corresponding technical problems and achieve corresponding effects, or may be combined with other features according to needs in certain scenarios. Accordingly, the data processing device provided in the embodiments of the present application may also implement these features or functions accordingly, which will not be described in detail here.

[0227] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0228] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0229] The present application also provides a data processing system, which includes an electronic device; wherein the electronic device is used to execute the method executed by the electronic device in any of the above method embodiments.

[0230] The present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed on an electronic device, the functions of any of the above method embodiments are implemented.

[0231] The present application also provides a computer program product. When the computer program product is executed on a computer, the computer can implement the functions of any of the above method embodiments.

[0232] As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items. As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrase "when determining..." or "if (stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "when (stated condition or event) is detected" or "in response to detecting (stated condition or event)", depending on the context.

[0233] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. A person of ordinary skill in the art may understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program to instruct the relevant hardware to complete the process, and the program may be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0234] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A data processing method, characterized in that: The method comprises: Acquire configuration information of the first image, the configuration information including first image data, data format information, algorithm association information and scene information; the scene information includes a plurality of algorithm modules corresponding to the first image; Sorting the multiple algorithm modules based on the data format information to obtain a first sequence; or sorting the multiple algorithm modules based on the algorithm association information to obtain a second sequence; adjusting the first sequence based on the algorithm association information to obtain a third sequence; or adjusting the second sequence based on the data format information to obtain a fourth sequence; Processing the multiple algorithm modules according to the third sequence or the fourth sequence to generate a first algorithm path corresponding to the first image; In the first algorithm path, the first image data is processed by a processor to obtain second image data.

2. The method according to claim 1, characterized in that The scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the method also includes: The processor is determined based on the scene information.

3. The method according to claim 1, characterized in that The scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the configuration information also includes system status information; The method further comprises: The processor is determined based on the system status information, or the processor is determined based on the system status information and the scene information.

4. The method according to claim 3, characterized in that The determining the processor based on the system state information and the scene information includes: Determining priority information corresponding to the first image based on the system state information and the scene information; the priority information is used to indicate that the priority of system power consumption is greater than the priority of system time consumption, or the priority information is used to indicate that the priority of system time consumption is greater than the priority of system power consumption; The processor is determined based on the priority information.

5. The method according to any one of claims 1 to 4, characterized in that The data format information includes a data format of the first image data, a data format of the second image data, and an image data format supported by each algorithm module of the multiple algorithm modules.

6. The method according to claim 1, characterized in that The third sequence is multiple, or the fourth sequence is multiple; Before the plurality of algorithm modules are processed according to the third sequence or the fourth sequence to generate a first algorithm path corresponding to the first image, the method further includes: Determine a target sequence from the plurality of third sequences or the plurality of fourth sequences; the target sequence is a sequence having the lowest number of image format conversions between the plurality of algorithm modules among the plurality of third sequences or the plurality of fourth sequences; The processing of the plurality of algorithm modules according to the third sequence or the fourth sequence to generate a first algorithm path corresponding to the first image includes: The multiple algorithm modules are processed according to the target sequence to generate a first algorithm pathway corresponding to the first image.

7. The method according to claim 6, characterized in that The scene information also includes module information corresponding to the first image and / or a mode adopted by the first image; the configuration information also includes system status information; Determining the target sequence from the plurality of third sequences or the plurality of fourth sequences comprises: A target sequence is determined from a plurality of third sequences or a plurality of fourth sequences based on the scene information and / or the system state information.

8. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-7.

9. A data processing system, characterized in that: Comprising an electronic device; wherein, the electronic device is used to execute the method as described in any one of claims 1-7.

10. A chip, characterized in that: It comprises a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in any one of claims 1 to 7 is executed.

11. A computer storage medium, characterized in that: The computer storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

12. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images