Pipeline creation method and related equipment

Abstract

The present application discloses a pipeline creation method and related equipment, the method comprising: determining a first output format of an image capture, the first output format being the image format before dumping; determining a first placeholder configuration corresponding to the first output format, the first placeholder configuration being used to indicate the position and type of a placeholder; obtaining a first pipeline style based on a parsed static configuration file and the first placeholder configuration, the first pipeline style including identifiers of multiple first plug-ins, positions of multiple first plug-ins, positions of placeholders, and types of placeholders; determining an identifier of a second plug-in based on the type of placeholder in the first pipeline style and first decision information; creating a first pipeline based on the identifiers of multiple first plug-ins, positions of multiple first plug-ins, an identifier of a second plug-in, and the position of the second plug-in, wherein the position of the second plug-in is determined based on the position of the placeholder. This sampling method can configure and create pipelines more flexibly.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a pipeline creation method and related equipment. Background Art

[0002] Plugins enable software to enhance and expand its functionality through modularity. Since plugins are independent of the main program, they are easier to update and maintain. Pipelines, on the other hand, combine multiple processing steps to create complex workflows. Plugins can be used as part of a pipeline to help implement more complex and diverse automated processes.

[0003] The traditional pipeline creation process can be roughly divided into: parsing the statically configured pipeline link, determining the required pipeline, creating all plug-ins in the pipeline; connecting the created plug-ins; and installing the entire pipeline. For example, the following pipelines are configured in the static configuration file: Pipeline A (plug-in 1-plug-in 2-plug-in 3); Pipeline B (plug-in 2-plug-in 4); Pipeline C (plug-in 2-plug-in 3-plug-in 4). If it is determined that the required pipeline is pipeline B, plug-in 2 and plug-in 4 in pipeline B are created. Then plug-in 2 and plug-in 4 are connected to obtain the entire pipeline. However, this traditional pipeline creation method requires static configuration of multiple pipelines that may be used for scenarios in the field of image processing, and cannot flexibly configure and create pipelines. Summary of the Invention

[0004] The present application provides a pipeline creation method and related equipment, which can configure and create pipelines more flexibly.

[0005] In a first aspect, some embodiments of the present application provide a pipeline creation method. The pipeline creation method may include: determining a first output format of image capture, the first output format being the image format before dumping; determining a first placeholder configuration corresponding to the first output format, the first placeholder configuration being used to indicate the position of a placeholder and the type of the placeholder; obtaining a first pipeline style based on a parsed static configuration file and the first placeholder configuration, the static configuration file being used to configure the identifiers of multiple first plug-ins and the positions of multiple first plug-ins, the first pipeline style including the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the positions of placeholders, and the types of placeholders; determining the identifier of a second plug-in based on the type of the placeholder in the first pipeline style and the first decision information; creating a first pipeline based on the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the identifier of a second plug-in, and the position of the second plug-in, the position of the second plug-in being determined based on the position of the placeholder.

[0006] Through the above approach, when configuring the first pipeline style, a location is reserved in the pipeline for the corresponding plug-in based on the first output format. Based on the current shooting environment, the corresponding plug-in is inserted into the reserved location. The inserted plug-in can be flexibly changed for different shooting environments, thus achieving flexible configuration and creation of pipelines.

[0007] In one possible implementation, the identifier of the second plug-in is determined based on the type of the placeholder in the first pipeline style and the first decision information, specifically: based on the type of the placeholder in the first pipeline style, a first plug-in decision set is determined from a plurality of preset plug-in decision sets, the plug-in decision set includes at least one enable function, and the enable function is used to decide whether to enable the corresponding plug-in; based on the first decision information and the first plug-in decision set, the identifier of the second plug-in is determined.

[0008] In the above manner, a plug-in decision set of this type is determined based on the type of the placeholder, and then based on the current shooting environment, the identifier of the corresponding plug-in is accurately determined from the plug-in decision set of this type.

[0009] In a possible implementation, the second plug-in is a plug-in corresponding to the enabling function satisfied by the first decision information.

[0010] In the above manner, the identifier of the plug-in inserted into the reserved position (the position of the placeholder) can be accurately determined based on the current shooting environment.

[0011] In one possible implementation, a first pipeline is created based on the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the identifier of the second plug-in, and the position of the second plug-in. Specifically, based on the identifiers of the multiple first plug-ins and the identifiers of the second plug-ins, multiple first plug-ins and second plug-ins are created; the position of the placeholder is determined to be the position of the second plug-in; and based on the positions of the multiple first plug-ins and the positions of the second plug-in, the multiple first plug-ins and the second plug-in are connected to obtain the first pipeline.

[0012] In the above manner, based on the plug-in identifier, the plug-in indicated by the plug-in identifier is accurately created, and based on the position of the plug-in, the plug-ins in the pipeline are accurately connected in sequence to obtain the first pipeline.

[0013] In one possible implementation, based on the positions of the multiple first plug-ins and the positions of the second plug-ins, the multiple first plug-ins are connected to obtain a first pipeline. Specifically, if the number of second plug-ins is greater than one, the multiple second plug-ins are connected; based on the positions of the multiple first plug-ins and the positions of the second plug-ins, the connected multiple second plug-ins are connected to the multiple first plug-ins to obtain the first pipeline, and the positions of the connected multiple second plug-ins are the positions of the placeholders.

[0014] In this way, when the determined second plug-in is a plurality of second plug-ins, the plurality of second plug-ins are first connected, and then the connected plurality of second plug-ins are inserted into the reserved positions and connected with other first plug-ins, so that the order of the plug-ins in the obtained first pipeline is accurate.

[0015] In one possible implementation, a first pipeline style is obtained based on a parsed static configuration file and a first placeholder configuration, specifically: system data of the electronic device is determined, the system data including the system's memory usage; if the system data meets a dump condition, a first pipeline style is obtained based on the parsed static configuration file and the first placeholder configuration, and the dump condition includes that the system's memory usage is less than a preset threshold.

[0016] Through the above method, when the memory resources of the electronic device are sufficient, the first pipeline style is configured and created, so as to avoid the process of configuring and creating the first pipeline style affecting the operation of other programs.

[0017] In one possible implementation, determining the first output format of image shooting includes: determining the first output format of image shooting based on the shooting mode; determining the first placeholder configuration corresponding to the first output format includes: determining the first placeholder configuration corresponding to the first output format based on a mapping relationship between a preset output format and the placeholder configuration.

[0018] In the above manner, different shooting modes may correspond to different output formats, and based on the shooting mode currently used by the electronic device, the output format of the shooting mode is determined, thereby making it possible to accurately determine the placeholder configuration.

[0019] In one possible implementation, if a change in shooting mode is detected, a second output image format is determined based on the changed shooting mode; if the first output image format is different from the second output image format, a second placeholder configuration corresponding to the second output image format is determined; based on the parsed static configuration file and the second placeholder configuration, a second pipeline style is obtained, the second pipeline style includes the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the positions of placeholders, and the types of placeholders; based on the type of placeholders in the second pipeline style and the second shooting environment information, the identifier of the second plug-in is determined; based on the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the identifier of the second plug-in, and the position of the second plug-in, a second pipeline is created.

[0020] In the above manner, when the output format changes, the second pipeline style is re-determined, and based on the shooting environment information, the plug-in inserted into the reserved position is re-determined, so that the configured and created pipeline can be applicable to the output format.

[0021] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are configured to store computer program code, the computer program code comprising computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the pipeline creation method according to any possible implementation of the first aspect.

[0022] In a third aspect, the present application provides a pipeline creation device, which can be an electronic device, a device in an electronic device, or a device that can be used in conjunction with an electronic device; wherein, the pipeline creation device can also be a chip system, and the pipeline creation device can execute the method performed by the electronic device in the first aspect. The functions of the pipeline creation device can be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more units corresponding to the above functions. The unit can be software and / or hardware. The operations and beneficial effects performed by the pipeline creation device can refer to the methods and beneficial effects described in the first aspect above, and repeated parts will not be repeated.

[0023] In a fourth aspect, the present application provides a chip system comprising a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions to execute the pipeline creation method in any possible implementation of the first aspect above.

[0024] In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program / instruction. When the computer program product runs on a computer, the computer executes the pipeline creation method in any possible implementation of the first aspect.

[0025] In a sixth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the pipeline creation method in any possible implementation of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flow chart of a traditional pipeline creation method provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0029] Figure 4A flowchart of a pipeline creation method based on a software structure provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of a pipeline creation method provided in an embodiment of the present application;

[0031] Figure 6A A schematic diagram of a first pipeline style provided in an embodiment of the present application;

[0032] Figure 6B A schematic diagram of a first pipeline created according to an embodiment of the present application;

[0033] Figure 6C A schematic diagram of another first pipeline style provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of the structure of a pipeline creation device provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0037] It should be understood that the terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish between different objects, rather than to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0038] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0039] To facilitate understanding of the solutions provided by the embodiments of the present application, the following describes the relevant concepts involved in the embodiments of the present application:

[0040] Raw format: RAW format is an uncompressed image file format that retains all the data captured by the camera sensor. The advantage of RAW format is that it can record the details of the photo and retain more image information, including exposure, tone and style.

[0041] UnpackRaw format: UnpackRaw format generally refers to converting the image's RAW data from a tightly packed format to a more easily processed unpacked format. UnpackRaw is a non-tightly packed format where each pixel's data is stored independently, making it easier for the image processor to read and process it.

[0042] Yuv format: The YUV format is a pixel format that represents brightness and chrominance components separately. It is mainly used in applications such as videos, pictures, and cameras. The YUV format contains three components: a brightness component (Y) and two chrominance components (U and V). Specifically: Luminance component (Y): represents brightness or grayscale value, which is the information of a black and white image. If only the Y component is displayed, the image will look like a black and white photo. Chrominance components (U and V): describe the color and saturation of the image. The U component represents the information after the brightness is removed from the blue part, and the V component represents the information after the brightness is removed from the red part. These two components are used together to specify the color of the pixel.

[0043] Pipeline creation: Pipeline creation is mainly as follows Figure 1 As shown, the process is divided into the following steps: parsing static configuration files, decision pipeline, creating plug-ins, connecting plug-ins, and installing pipeline. The following describes the above steps respectively:

[0044] Parsing a static configuration file: A static configuration file includes multiple pre-configured pipeline styles. For example, the static configuration file may contain the following three pipeline styles: Pipeline A (plugin 1 - plugin 2 - plugin 3); Pipeline B (plugin 2 - plugin 4); and Pipeline C (plugin 2 - plugin 3 - plugin 4). By parsing the static configuration file, you can obtain the three pipeline styles.

[0045] Decision pipeline: Based on the current scenario, the corresponding feature identifier is determined. This feature identifier is the feature identifier of the pipeline. The features of different pipelines can be the same, so different pipelines can correspond to the same feature identifier. For example, the feature identifiers corresponding to pipelines A and C can be the same, while the feature identifiers corresponding to pipelines A and B are different. This feature identifier can be determined based on the scenario in which the current pipeline is applied. For example, if the current scenario is a search query, the feature identifier is used to indicate the pipeline that can perform the search query. If pipeline A is the pipeline for the search query, the pipeline obtained by the feature identifier indication is pipeline A.

[0046] Create each plug-in: Based on the pipeline pattern indicated by the feature identifier, create each plug-in in the pipeline. For example, if the pipeline indicated by the feature identifier is pipeline A (plug-in 1-plug-in 2-plug-in 3), then create plug-in 1, plug-in 2, and plug-in 3.

[0047] Connect the plug-ins: Connect the plug-ins in the order of the pipeline style. For example, plug-in 1 is connected to plug-in 2, and plug-in 2 is connected to plug-in 3.

[0048] Installing Pipelines: If the feature identifier indicates multiple pipeline types, create and connect plugins for each of these pipelines before installing them. For example, for Pipeline A and Pipeline B, first create and connect the three plugins in Pipeline A (Plugin 1 - Plugin 2 - Plugin 3), and the two plugins in Pipeline B (Plugin 2 - Plugin 4), then install Pipeline A and Pipeline B together to create the final pipeline.

[0049] The above-mentioned method of configuring and creating pipelines requires that all pipeline styles be pre-configured in the static configuration file. That is to say, for scenarios where there are multiple plug-in combinations and the positions of plug-ins need to be flexibly changed, all combinations and positions need to be pre-listed and configured in the static configuration file. This method of configuring and creating pipelines requires re-parsing all pipeline styles in the static configuration file every time the scene changes and the pipeline needs to be updated, and traversing all pipeline styles to determine the pipeline style suitable for the current scene. It is not possible to flexibly configure and create pipelines, which not only wastes computing resources but is also time-consuming. In order to solve this problem, the present application provides a pipeline creation method and related equipment that can configure and create pipelines more flexibly. The pipeline creation method is further introduced below.

[0050] The hardware structure of the electronic device 100 is introduced below. Figure 2 , Figure 2 Schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0051] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0052] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0054] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0055] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves system efficiency. The processor 110 calls the instructions or data stored in the memory, causing the electronic device 100 to execute the shooting method performed by the electronic device in the following method embodiment.

[0056] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0057] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0058] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110.

[0059] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0060] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0061] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0062] The modem processor includes a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is passed to the application processor.

[0063] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0064] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. Display screen 194 may be an outward-folding screen, i.e., a display screen that folds outward.

[0065] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor. The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture still images or videos. The camera 193 may include a front camera and a rear camera, the front camera is located in the display area of ​​the screen, and the rear camera is located in the back area of ​​the screen. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs.

[0066] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0067] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area can store data (such as audio data) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc.

[0068] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0069] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0070] Speaker 170A, also known as a "horn," is used to convert audio electrical signals into sound signals. Receiver 170B, also known as an "earpiece," is used to convert audio electrical signals into sound signals. Microphone 170C, also known as a "microphone" or "microphone," is used to convert sound signals into electrical signals. Headphone jack 170D is used to connect wired headphones. Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A may be provided on display screen 194. Gyroscope sensor 180B may be used to determine the motion posture of electronic device 100. Air pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes a Hall sensor. Acceleration sensor 180E may detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector. Ambient light sensor 180L is used to sense ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate vibration prompts. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0071] Figure 3 A schematic diagram of the software structure of an electronic device 100 provided in an embodiment of the present application.

[0072] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the hardware abstraction layer (HAL).

[0073] The application layer can include a series of application packages. Figure 3 As shown, the application layer may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0074] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, an operating system, and the like.

[0075] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0076] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0077] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0078] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0079] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the system's top status bar as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the user's terminal, and flashing indicator lights.

[0080] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0081] The operating system can provide memory information for the electronic device, including memory usage. It can also provide temperature information, sleep / wakeup information, and application data. The sleep / wakeup information includes whether the electronic device is currently awake or asleep, and the application data includes the applications currently running on the electronic device.

[0082] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0083] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0084] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0085] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0086] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0087] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0088] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0089] A 2D graphics engine is a drawing engine for 2D drawings.

[0090] The Hardware Abstraction Layer (HAL) is an interface layer between the operating system kernel and the hardware circuitry. Its purpose is to abstract the hardware. It hides the platform-specific hardware interface details and provides a virtual hardware platform for the operating system. The HAL includes the perception engine, dump decision, session module, and decision engine.

[0091] In order to better understand the pipeline creation method provided by this application, Figure 4 The pipeline creation method is further introduced. Figure 4 As shown, Figure 4The present invention provides a pipeline creation method based on software architecture. The pipeline creation method based on software architecture mainly involves the following five modules: operating system, perception engine, dump decision, session module (Session), and decision engine. The operating system is located in the application architecture layer, and the perception engine, dump decision, session module, and decision engine are located in the hardware abstraction layer. Among them:

[0092] 401. The operating system sends operating system data to the perception engine.

[0093] Among them, the operating system data includes memory information, application data, and sleep wake-up information, etc. The memory information is the memory usage of the electronic device, or the memory free amount of the electronic device. For example, the memory information is 65% (memory usage of the electronic device), or the memory information is 35% (memory free amount of the electronic device). It should be noted that the memory information can also be in other forms, such as memory information is 900MB (memory usage of the electronic device).

[0094] The application data may include the currently running application, or may also include the priority of the currently running application (e.g., call applications have the highest priority, game applications have the next highest priority), or may also include the memory usage of the currently running application. For example, the application data may be: Game Application 1-A-27%, where Game Application 1 is the name of the currently running application, A is the priority of Game Application 1, and 27% is the memory usage of Game Application 1.

[0095] The sleep-wake-up information indicates whether the current state of the electronic device is a sleep state or a wake-up state. In a possible embodiment, when the electronic device is in a screen-off state, the electronic device may be considered to be in a sleep state, otherwise the electronic device may be considered to be in a wake-up state.

[0096] It should be noted that the operating system may also include other data besides the above, such as power data of electronic devices, etc., and this application does not impose any restrictions on this.

[0097] 402. The perception engine sends temperature information, memory information, sleep and wake-up information, and application data to the dump decision.

[0098] The temperature information may be determined by a perception engine, which may include a temperature sensor. Alternatively, the temperature information may be obtained by an operating system through a sensor and then sent to the perception engine. The temperature information is the current temperature of the electronic device.

[0099] Optionally, the perception engine may also send other data to the dump decision, such as power data of the electronic device, etc., which is not limited in this application.

[0100] 403. The dump decision is based on temperature information, memory information, sleep and wake-up information, and application data to determine whether a dump module needs to be added.

[0101] Whether to add a dump module can be determined based on whether various information meets dump adding conditions, including temperature threshold, memory threshold, sleep / wake-up state condition, and application data condition.

[0102] In a possible embodiment, when the temperature information is lower than the temperature threshold, the memory information (memory usage of the electronic device) is lower than the memory threshold, or the memory information is higher than the memory threshold (memory free amount of the electronic device), the sleep wake-up information meets the sleep wake-up state condition, and the application data meets the application data condition, it is determined that a dump module needs to be added.

[0103] For example, the temperature threshold is 30 degrees Celsius, the memory threshold is 40%, the sleep-wake state condition is awake, and the application data condition is that no high-priority applications (such as gaming applications) are running. If the temperature information is 25 degrees Celsius (lower than 30 degrees Celsius), the memory information is 30% (lower than 40%), the sleep-wake information is awake, and the running applications do not include high-priority applications, then it is determined that the dump module needs to be added.

[0104] Optionally, in addition to the above conditions, a power condition may also be included, and the information may also include the power of the electronic device. For example, if the power condition is that the power of the electronic device needs to be higher than 30%, if the current power of the electronic device is 40%, then the power condition is met.

[0105] Optionally, in addition to the above conditions, more conditions may be included, which are not limited in this application.

[0106] 404. If a dump module needs to be added, the dump decision sets the device environment information to the session module.

[0107] The environmental information includes the above-mentioned temperature information, memory information, sleep and wake-up information, application data, etc. The above-mentioned environmental information is recorded to provide a log to query the environmental information of the electronic device in each time period.

[0108] 405. The conversation module sends a decision instruction to the decision engine.

[0109] Among them, the decision instruction is used to instruct the decision engine to determine the feature identifier. The feature identifier can be found in the introduction above and will not be described in detail in this application.

[0110] 406. The decision engine determines the feature information and, based on the feature information, determines a feature identifier.

[0111] The decision engine responds to the decision instructions sent by the conversation module and determines feature information. This feature information is determined based on the current scenario. Different scenarios have different requirements and corresponding feature information. For example, in a shopping scenario, this feature can be a search query feature.

[0112] In a possible embodiment, if it is a shooting scene, the feature information may be the current shooting mode. For example, if the current shooting mode is portrait mode, the feature information is determined based on the portrait mode; or if the current shooting mode is moon shooting mode, the feature information is determined based on the moon shooting mode.

[0113] The feature information and the feature identifier may correspond one-to-one, or multiple feature information may correspond to the same feature identifier, or one feature information may correspond to multiple feature identifiers. For example, the feature identifier corresponding to the portrait mode and the high-definition mode may be the same feature identifier. The moon shooting mode may correspond to multiple feature identifiers, thereby determining multiple pipeline styles. In one possible implementation, the electronic device pre-stores a mapping relationship between the shooting mode and the feature identifier, and the corresponding feature identifier can be directly determined based on the shooting mode through the mapping relationship.

[0114] 407. The decision engine sends the feature identifier to the conversation module.

[0115] Among them, the feature identification can be found in the introduction of the above step 405, and this application will not elaborate on it here.

[0116] 408. The session module configures and creates a pipeline based on the feature identifier.

[0117] The process of configuring and creating a pipeline can be found in the following Figure 5 As shown, this application will not be described in detail here.

[0118] The following is a further introduction to the pipeline creation method provided in the embodiment of the present application:

[0119] 501. The electronic device determines a first output format of an image captured, where the first output format is a format of the image before dumping.

[0120] The first output image format is the format of the image before dumping. The format of the image before dumping is the format of the image to be dumped. The format of the image before dumping can be the format of the image obtained after the camera completes shooting. Alternatively, the first output image format can also be the image format before the image format is converted, for example, converting an image in the YUV format to the JPG format, with the YUV format being the first output image format. Alternatively, the first output image format can be the format of the first pipeline input.

[0121] In one possible embodiment, the electronic device determines the first output format of an image captured by: the electronic device determines the first output format of the image captured based on a shooting mode. The shooting mode is the shooting mode currently used by the camera, and the electronic device has a preset mapping relationship between the camera mode and the first output format. The first output format can be determined based on the camera mode and the mapping relationship.

[0122] Optionally, the electronic device determines the first output image format based on user-selected shooting parameters. For certain shooting modes (e.g., professional mode), the user can select shooting parameters (e.g., exposure parameters, focal length, etc.), and the first output image format is determined based on the user-selected shooting parameters and a preset mapping relationship between the shooting parameters and the output image format.

[0123] 502. The electronic device determines a first placeholder configuration corresponding to a first output image format, where the first placeholder configuration is used to indicate a position and a type of a placeholder.

[0124] The first placeholder configuration includes the position of the placeholder in the pipeline and the type of the placeholder. The position in the pipeline can be indicated in the following two ways: indicating the plug-in before the placeholder and the plug-in after the placeholder; or indicating the position of the placeholder in the pipeline. For example, Figure 6A As shown, the position of the placeholder can be indicated by indicating that the plug-in before the placeholder is a resource plug-in and the plug-in after the placeholder is plug-in A; or the position of the placeholder can be indicated by indicating that the placeholder is located at the second position in the pipeline.

[0125] In one possible embodiment, the placeholder type is the first output image format, which includes the following three types: Unpack Raw format, Raw format, and Yuv format. The above three types are output image formats, which can be found in the previous description and will not be described in detail in this application.

[0126] In one possible embodiment, the electronic device determines the first placeholder configuration based on the shooting mode. Different shooting modes correspond to different output image formats, and different output image formats correspond to different placeholder configurations. Therefore, the position of the placeholder can be directly determined based on the shooting mode.

[0127] In one possible embodiment, the electronic device determines the type of the placeholder based on the shooting mode. Different shooting modes correspond to different output image formats, and the output image format is the type of the placeholder. Therefore, the type of the placeholder can be directly determined based on the shooting mode.

[0128] In a possible embodiment, the electronic device determines the first placeholder configuration corresponding to the first output image format, specifically: the electronic device determines the first placeholder configuration corresponding to the first output image format based on a preset mapping relationship between the output image format and the placeholder configuration.

[0129] 503. The electronic device obtains a first pipeline style based on the parsed static configuration file and the first placeholder configuration. The static configuration file is used to configure the identifiers of multiple first plug-ins and the positions of multiple first plug-ins. The first pipeline style includes the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the positions of placeholders, and the types of placeholders.

[0130] The static configuration file includes a plurality of preset pipeline styles, and the first pipeline style is different from the preset pipeline style. Compared with the preset pipeline style, the first pipeline style has a new placeholder.

[0131] In a possible embodiment, the electronic device determines a characteristic identifier; the electronic device determines a default pipe style based on the characteristic identifier; the electronic device obtains a first pipe style based on the default pipe style and the first placeholder configuration. The process of determining the characteristic identifier and determining the default pipe style by the electronic device can be referred to above. Figure 4 As shown, this application will not be described in detail here.

[0132] For example, Figure 6A As shown, the electronic device determines the default pipeline style as (resource plug-in-plug-in A-plug-in B-plug-in C-data pool plug-in) based on the feature identifier. After parsing the first placeholder configuration, the electronic device inserts the placeholder after the resource plug-in and before plug-in A. The first pipeline style is as follows Figure 6A shown.

[0133] In a possible embodiment, the electronic device obtains a first pipeline style based on a parsed static configuration file and a first placeholder configuration, specifically: the electronic device determines the system data of the electronic device, and the system data includes the memory usage of the system; if the system data of the electronic device meets the dump condition, the electronic device obtains the first pipeline style based on the parsed static configuration file and the first placeholder configuration, and the dump condition includes that the memory usage of the system is less than a preset threshold.

[0134] Among them, it can be to determine the pipeline style and subsequent processes when the electronic device has sufficient memory. The data system and memory usage can be seen above Figure 4 The system data may include the amount of free memory in the system. In addition, the system data may also include other data, such as temperature information of the electronic device, sleep and wake-up information, application data, etc., which are not described in detail in this application.

[0135] Through the above method, when the memory resources of the electronic device are sufficient, the first pipeline style is configured and created, so as to avoid the process of configuring and creating the first pipeline style affecting the operation of other programs.

[0136] 504. The electronic device determines an identifier of the second plug-in based on the type of the placeholder in the first pipeline pattern and first decision information, where the first decision information includes shooting environment information.

[0137] The second plug-in is a plug-in placed in the placeholder position, and there may be one or more second plug-ins. The first decision information may be current environmental information, such as the brightness of the environment, or the distance between the photographed object and the electronic device. For example, in the moon photography mode, the first decision information is the distance between the photographed object and the electronic device. The first decision information may also be information other than the shooting environment information, such as shooting parameters.

[0138] In a possible embodiment, the electronic device determines the identifier of the second plug-in based on the type of the placeholder in the first pipeline style and the first decision information. Specifically, the electronic device determines a first plug-in decision set from a plurality of preset plug-in decision sets based on the type of the placeholder in the first pipeline style, where the plug-in decision set includes at least one enable function, which is used to decide whether to enable the corresponding plug-in; and determines the identifier of the second plug-in based on the first decision information and the first plug-in decision set.

[0139] In the above manner, a plug-in decision set of this type is determined based on the type of the placeholder, and then based on the current shooting environment, the identifier of the corresponding plug-in is accurately determined from the plug-in decision set of this type.

[0140] In a possible embodiment, the second plug-in is a plug-in corresponding to the enabling function satisfied by the first decision information.

[0141] The plug-in decision set may include a mapping relationship between the enabling function and the plug-in identifier, and the representation of the plug-in corresponding to the enabling function satisfied by the first decision information is used as the identifier of the second plug-in. Figure 6B As shown, if the type of the placeholder in the first pipeline style is Raw type, the first plug-in decision set is the Raw plug-in decision set, and if the first decision information satisfies the enabling function 2 in the Raw plug-in decision set, the determined identifier of the second plug-in is plug-in Raw2.

[0142] In the above manner, the identifier of the plug-in inserted into the reserved position (the position of the placeholder) can be accurately determined based on the current shooting environment.

[0143] Optionally, if the first decision information satisfies multiple enabling functions, the determined second plug-in identifiers are also multiple. For example, Figure 6B As shown, if the type of the placeholder in the first pipeline style is the UnpackRaw type, the first plug-in decision set is the UnpackRaw plug-in decision set, and if the first decision information satisfies the enable function 2 and enable function 3 in the Raw plug-in decision set, the determined second plug-in identifier is plug-in UnpackRaw2 and plug-in UnpackRaw3.

[0144] in, Figure 6B The same is true for the Yuv type in , which will not be described in detail in this application. Figure 6B This is just an example diagram, and the first pipeline style usually only includes Figure 6B A placeholder in , but may also include multiple placeholders, which is not described in detail in this application.

[0145] In one possible embodiment, the plugin decision set does not include a mapping between enable functions and plugin identifiers. Instead, it includes only the enable functions, whose outputs are plugin identifiers. For example, if the first decision information satisfies enable function 1, and the output of enable function 1 is plugin Raw2, the second plugin identifier is plugin Raw2.

[0146] 505. The electronic device creates a first pipeline based on the identifiers and positions of the multiple first plug-ins, the identifier and position of the second plug-in, where the position of the second plug-in is determined based on the position of the placeholder.

[0147] The plug-ins in the first pipeline are plug-ins corresponding to the identifiers of the multiple first plug-ins in the first pipeline style, and plug-ins corresponding to the identifiers of the second plug-ins. The positions of the plug-ins in the first pipeline are the same as the positions of the plug-ins in the first pipeline style. The first pipeline can be used to dump images, specifically to store images in an album. The position of the second plug-in can be a placeholder position.

[0148] In a possible embodiment, the electronic device creates a first pipeline based on the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the identifier of the second plug-in, and the position of the second plug-in, including: the electronic device creates multiple first plug-ins and second plug-ins based on the identifiers of the multiple first plug-ins and the identifiers of the second plug-ins; the electronic device determines the position of the placeholder as the position of the second plug-in; the electronic device connects the multiple first plug-ins and the second plug-in based on the positions of the multiple first plug-ins and the positions of the second plug-in to obtain the first pipeline.

[0149] The connection is based on the position of each plug-in, and the position of each plug-in in the connected pipeline is the same as that in the first pipeline style.

[0150] For example, Figure 6A As shown, a resource plug-in, a second plug-in, plug-in A, plug-in B, plug-in C, and a data pool plug-in are created. The second plug-in is located at the position of the placeholder (after the resource plug-in and before plug-in A). According to the position of each plug-in, the resource plug-in is connected to the second plug-in, the second plug-in is connected to plug-in A, plug-in A is connected to plug-in B, plug-in B is connected to plug-in C, and plug-in C is connected to the data pool plug-in to obtain the first pipeline.

[0151] There may be multiple determined identifiers of the second plug-in. If there are multiple identifiers of the second plug-in, after creating multiple second plug-ins based on the identifiers of the multiple second plug-ins, the second plug-ins are first connected, and then the connected second plug-ins are connected to the remaining first plug-ins.

[0152] In the above manner, based on the plug-in identifier, the plug-in indicated by the plug-in identifier is accurately created, and based on the position of the plug-in, the plug-ins in the pipeline are accurately connected in sequence to obtain the first pipeline.

[0153] In a possible embodiment, the electronic device connects multiple first plug-ins and second plug-ins based on the positions of the multiple first plug-ins and the positions of the second plug-ins to obtain a first pipeline, including: if the number of second plug-ins is greater than one, the electronic device connects the multiple second plug-ins; based on the positions of the multiple first plug-ins and the positions of the second plug-ins, the electronic device connects the connected multiple second plug-ins with the multiple first plug-ins to obtain the first pipeline, and the positions of the connected multiple second plug-ins are the positions of the placeholders.

[0154] The positions of the plurality of second plug-ins are the positions of placeholders, for example Figure 6B As shown, if the placeholder type is Unpack Raw, two second plug-in identifiers are determined: plug-in Unpack Raw2 and plug-in Unpack Raw3. After creating plug-ins Unpack Raw2 and Unpack Raw3, they are first connected. Then, the connected plug-ins Unpack Raw2 and Unpack Raw3 are connected to the resource plug-in and plug-in A. The positions of the connected plug-ins Unpack Raw2 and Unpack Raw3 in the pipeline are the positions of the Unpack Raw placeholder in the pipeline.

[0155] In this way, when the determined second plug-in is a plurality of second plug-ins, the plurality of second plug-ins are first connected, and then the connected plurality of second plug-ins are inserted into the reserved positions and connected with other first plug-ins, so that the order of the plug-ins in the obtained first pipeline is accurate.

[0156] After creating the first pipeline, it is applied to the scene. When the scene changes (for example, when the shooting mode changes), the pipeline needs to be updated to adapt to the new scene. The following is a further introduction to the pipeline update method:

[0157] In a possible embodiment, if the electronic device detects a change in shooting mode, it determines a second output image format based on the changed shooting mode; if the first output image format of the electronic device is different from the second output image format, it determines a second placeholder configuration corresponding to the second output image format; the electronic device obtains a second pipeline style based on the parsed static configuration file and the second placeholder configuration, the second pipeline style including the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the positions of placeholders, and the types of placeholders; the electronic device determines the identifier of the second plug-in based on the type of placeholder in the second pipeline style and the second shooting environment information; and creates a second pipeline based on the identifiers of multiple first plug-ins, the positions of multiple first plug-ins, the identifier of the second plug-in, and the position of the second plug-in.

[0158] Among them, since different shooting modes may correspond to the same output image format, the second output image format and the second output image format may also be the same. The method for determining the second placeholder configuration corresponding to the second output image format can refer to the method for the first placeholder configuration corresponding to the first output image format mentioned above; similarly, the method for obtaining the second pipeline style based on the parsed static configuration file and the second placeholder configuration can also refer to the above-mentioned method for obtaining the first pipeline style based on the parsed static configuration file and the first placeholder configuration; similarly, the subsequent creation of the second pipeline can also refer to the above-mentioned introduction to creating the first pipeline.

[0159] In the above manner, when the output format changes, the second pipeline style is re-determined, and based on the shooting environment information, the plug-in inserted into the reserved position is re-determined, so that the configured and created pipeline can be applicable to the output format.

[0160] In a possible embodiment, the step of determining whether to update can be as follows: Figure 6C As shown, Figure 6C The steps in bold are new steps added to the existing pipeline creation process. Figure 6C In the Unpack Raw insertion point configuration, Raw insertion point configuration and Yuv insertion point configuration, only one of the three can be configured, which is determined by the parsed placeholder. Similarly, when creating the Unpack Raw plug-in, Raw plug-in and Yuv plug-in, only one of them can be created. This application does not limit this. The process of determining whether the pipeline needs to be updated can also be as follows Figure 6C As shown, after determining whether the plug-in is enabled.

[0161] See Figure 7, Figure 7 A schematic diagram of the structure of a pipeline creation device 700 provided in an embodiment of the present application. Figure 7 The pipeline creation device shown may be an electronic device, a device in an electronic device, or a device that can be used in conjunction with an electronic device. Figure 7 The pipeline creation and training device shown may include a processing unit 701 and a parsing unit 702.

[0162] The processing unit 701 is configured to determine a first output image format of the image captured, where the first output image format is the image format before dumping;

[0163] The processing unit 701 is further configured to determine a first placeholder configuration corresponding to the first output image format, where the first placeholder configuration is used to indicate a position and a type of the placeholder.

[0164] A parsing unit 702 is configured to obtain a first pipeline style based on the parsed static configuration file and the first placeholder configuration, wherein the static configuration file is used to configure identifiers of the plurality of first plug-ins and positions of the plurality of first plug-ins, and the first pipeline style includes the identifiers of the plurality of first plug-ins, the positions of the plurality of first plug-ins, the positions of the placeholders, and the types of the placeholders;

[0165] The processing unit 701 is further configured to determine an identifier of the second plug-in based on the type of the placeholder in the first pipeline pattern and the first decision information;

[0166] The processing unit 701 is further configured to create a first pipeline based on the identifiers and positions of the multiple first plug-ins, the identifier and position of the second plug-in, wherein the position of the second plug-in is determined based on the position of the placeholder.

[0167] In one possible implementation, the processing unit 701 is further used to determine a first plug-in decision set from a plurality of preset plug-in decision sets based on the type of the placeholder in the first pipeline style, where the plug-in decision set includes at least one enable function, and the enable function is used to decide whether to enable the corresponding plug-in; and determine an identifier of the second plug-in based on the first decision information and the first plug-in decision set.

[0168] In a possible implementation, the second plug-in is a plug-in corresponding to the enabling function satisfied by the first decision information.

[0169] In one possible implementation, the processing unit 701 is further configured to create multiple first plug-ins and second plug-ins based on the identifiers of the multiple first plug-ins and the identifiers of the second plug-ins; determine the position of the placeholder as the position of the second plug-in; and connect the multiple first plug-ins and second plug-ins based on the positions of the multiple first plug-ins and the positions of the second plug-ins to obtain a first pipeline.

[0170] In one possible implementation, the processing unit 701 is further configured to connect multiple second plug-ins if the number of the second plug-ins is greater than one; and based on the positions of the multiple first plug-ins and the positions of the second plug-ins, connect the connected multiple second plug-ins with the multiple first plug-ins to obtain a first pipeline, where the positions of the connected multiple second plug-ins are the positions of the placeholders.

[0171] In one possible implementation, the processing unit 701 is also used to determine the system data of the electronic device, where the system data includes the system's memory usage; if the system data meets the dump condition, a first pipeline style is obtained based on the parsed static configuration file and the first placeholder configuration, where the dump condition includes that the system's memory usage is less than a preset threshold.

[0172] In one possible implementation, the processing unit 701 is further used to determine a first output format of the image shooting based on the shooting mode; and determine a first placeholder configuration corresponding to the first output format based on a mapping relationship between a preset output format and a placeholder configuration.

[0173] In one possible implementation, the processing unit 701 is further configured to determine a second output image format based on the changed shooting mode if a change in shooting mode is detected; determine a second placeholder configuration corresponding to the second output image format if the first output image format is different from the second output image format; obtain a second pipeline style based on the parsed static configuration file and the second placeholder configuration, the second pipeline style including identifiers of multiple first plug-ins, positions of multiple first plug-ins, positions of placeholders, and types of placeholders; determine an identifier of the second plug-in based on the type of placeholder in the second pipeline style and the second shooting environment information; and create a second pipeline based on the identifiers of multiple first plug-ins, positions of multiple first plug-ins, identifier of the second plug-in, and position of the second plug-in.

[0174] For the case where the pipeline creation device can be a chip or a chip system, see Figure 8 Schematic diagram of the chip structure shown. Figure 8 The chip 800 shown includes a processor 801 and an interface 802. Optionally, it may also include a memory 803. The number of the processors 801 may be one or more, and the number of the interfaces 802 may be multiple.

[0175] For the case where the chip is used to implement the electronic device in the embodiment of the present application:

[0176] The interface 802 is used to receive or output signals;

[0177] The processor 801 is configured to execute data processing operations of the electronic device.

[0178] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0179] It is understood that some optional features in the embodiments of the present application may, in certain scenarios, be implemented independently of other features, such as the solution on which they are currently based, to solve corresponding technical problems and achieve corresponding effects. Alternatively, they may be combined with other features in certain scenarios as needed. Accordingly, the pipeline creation device provided in the embodiments of the present application may also implement these features or functions accordingly, which will not be described in detail here.

[0180] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

[0181] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0182] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on an electronic device, the functions of any of the above method embodiments are implemented.

[0183] The present application also provides a computer program product, which, when executed on a computer, enables the computer to implement the functions of any of the above method embodiments.

[0184] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0185] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A pipeline creation method, characterized in that: The method comprises: Determining a first output image format of the image shooting, where the first output image format is the image format before dumping; Determine a first placeholder configuration corresponding to the first output image format, where the first placeholder configuration is used to indicate a position of a placeholder and a type of the placeholder, and the type of the placeholder is the first output image format; Obtaining a first pipeline style based on the parsed static configuration file and the first placeholder configuration, the static configuration file being used to configure identifiers of a plurality of first plug-ins and positions of the plurality of first plug-ins, the first pipeline style including the identifiers of the plurality of first plug-ins, the positions of the plurality of first plug-ins, the positions of the placeholders, and the types of the placeholders; determining an identifier of a second plug-in based on a type of the placeholder in the first pipeline style and first decision information, wherein the first decision information includes shooting environment information; A first pipeline is created based on the identifiers of the first plug-ins, the positions of the first plug-ins, the identifier of the second plug-in, and the position of the second plug-in, wherein the position of the second plug-in is determined based on the position of the placeholder.

2. The method according to claim 1, characterized in that The determining, based on the type of the placeholder in the first pipeline style and the first decision information, an identifier of the second plug-in includes: Determining, based on the type of the placeholder in the first pipeline style, a first plug-in decision set from a plurality of preset plug-in decision sets, wherein the plug-in decision set includes at least one enable function, and the enable function is used to decide whether to enable a corresponding plug-in; An identifier of a second plug-in is determined based on the first decision information and the first plug-in decision set.

3. The method according to claim 2, characterized in that The second plug-in is a plug-in corresponding to the enabling function satisfied by the first decision information.

4. The method according to any one of claims 1 to 3, characterized in that The creating a first pipeline based on the identifiers of the plurality of first plug-ins, the positions of the plurality of first plug-ins, the identifier of the second plug-in, and the position of the second plug-in includes: Creating the plurality of first plug-ins and the second plug-in based on the identifiers of the plurality of first plug-ins and the identifier of the second plug-in; Determine the position of the placeholder as the position of the second plug-in; Based on the positions of the plurality of first plug-ins and the position of the second plug-in, the plurality of first plug-ins and the second plug-in are connected to obtain a first pipeline.

5. The method according to claim 4, characterized in that The step of connecting the plurality of first plug-ins and the second plug-in based on the positions of the plurality of first plug-ins and the position of the second plug-in to obtain a first pipeline includes: If the number of the second plug-ins is greater than one, connecting multiple second plug-ins; Based on the positions of the multiple first plug-ins and the positions of the second plug-ins, the multiple second plug-ins are connected to the multiple first plug-ins to obtain a first pipeline, and the positions of the multiple second plug-ins are the positions of the placeholders.

6. The method according to claim 5, characterized in that The obtaining of a first pipeline style based on the parsed static configuration file and the first placeholder configuration includes: determining system data of the electronic device, the system data including memory usage of the system; If the system data meets the dump condition, a first pipeline style is obtained based on the parsed static configuration file and the first placeholder configuration, and the dump condition includes that the system memory usage is less than a preset threshold.

7. The method according to claim 6, characterized in that The step of determining a first output image format of the captured image includes: Determining a first output format of the image captured based on the shooting mode; Determining a first placeholder configuration corresponding to the first output image format includes: Based on a preset mapping relationship between an image output format and a placeholder configuration, a first placeholder configuration corresponding to the first image output format is determined.

8. The method according to claim 7, characterized in that The method further comprises: If a change in the shooting mode is detected, determining a second output image format based on the changed shooting mode; If the first output image format is different from the second output image format, determining a second placeholder configuration corresponding to the second output image format; Obtaining a second pipeline style based on the parsed static configuration file and the second placeholder configuration, the second pipeline style including identifiers of the plurality of first plug-ins, positions of the plurality of first plug-ins, positions of the placeholders, and types of the placeholders; determining an identifier of a second plug-in based on the type of the placeholder in the second pipeline pattern and the second shooting environment information; A second pipeline is created based on the identifiers of the first plug-ins, the positions of the first plug-ins, the identifier of the second plug-in, and the position of the second plug-in.

9. An electronic device comprising one or more memories and one or more processors, characterized in that: The memory is used to store a computer program; the processor is used to call the computer program, so that the electronic device executes the method according to any one of claims 1 to 8.

10. A chip system, applied to electronic equipment, characterized in that: The chip system includes at least one processor and an interface, wherein the interface is used to receive instructions and transmit them to the at least one processor; the at least one processor executes the instructions so that the electronic device executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images