Image filter application method and apparatus, device, medium, product

Abstract

The application relates to an image filter application method and device, equipment, medium and product, the method comprising: starting a camera unit to collect live image data, loading a filter generator to generate filter description data of a target filter; applying the filter description data to generate a preview image of the live image data by a preview filter thread to output and display, the preview image applying a first filter texture generated according to the filter description data; in response to an image export instruction, applying the filter description data to generate a static image of the live image data by an off-screen filter thread to export as a live image file, the static image applying a second filter texture generated according to the filter description data. The application optimizes the implementation architecture of filter application, realizes on-screen preview and image export of live image data collected by the camera unit respectively, can improve the filter application efficiency and effect, and has a wide application prospect.

Description

Technical Field

[0001] This application relates to image processing technology, and more particularly to an image filter application method, apparatus, device, medium, and product thereof. Background Technology

[0002] Various image processing applications often offer shooting filter functions, allowing users to apply corresponding filters to their captured images. Filters can change parameters such as color, brightness, contrast, and saturation of the images captured by the camera unit, thereby achieving different shooting effects.

[0003] Traditional mobile camera functions suffer from several problems with their filter management technology, such as difficulty in managing filters, inconvenience in use, and complex and intertwined real-time preview and off-screen rendering imaging status data. This leads to low efficiency, poor flexibility, and compatibility issues across various platforms when iterating on existing technical solutions or solving problems.

[0004] In detail, the problems with traditional filter management technology are mainly manifested in the following aspects: 1. Slow shooting speed: The traditional camera calling method involves multiple copies of data and memory during the photo and filter processing, and the multiple interactions between image and filter content result in long imaging time and high memory consumption; 2. Shooting compatibility issues: Due to the camera's state management and excessive memory consumption, the traditional shooting process is prone to a series of problems such as shooting memory overflow, shooting black screen, and shooting blur; 3. Poor scalability: The traditional camera's filter calling method has poor scalability and does not have the dynamic hot switching and shooting capability enhancement capabilities of third-party camera SDKs; 4. Repetitive code assembly: The code assembly is loose, seriously piled up, chaotic state management, and has poor scalability.

[0005] It is evident that traditional filter technology has many shortcomings and needs to be improved through technological advancements to perfect filter technology and enhance the efficiency of filter use. Summary of the Invention

[0006] The purpose of this application is to provide an image filter application method and corresponding apparatus, device, non-volatile readable storage medium, and computer program product.

[0007] According to one aspect of this application, an image filter application method is provided, comprising the following steps:

[0008] The camera unit is activated to acquire on-site image data, and the filter generator is loaded to generate filter description data for the target filter.

[0009] A preview image of the live image data is generated by the preview filter thread using the filter description data and displayed as an output. The preview image applies a first filter texture generated based on the filter description data.

[0010] In response to an image export command, the off-screen filter thread applies the filter description data to generate a static image of the scene image data for export as a scene image file. The static image applies a second filter texture generated based on the filter description data.

[0011] In an optional embodiment, the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, including:

[0012] The preview filter thread generates the first image texture of the live image data in the first texture space;

[0013] The preview filter thread generates the first filter texture of the target filter by overlaying it onto the first image texture based on the filter description data;

[0014] The preview filter thread displays the first image texture, overlaid with the first filter texture, as the preview image in the graphical user interface.

[0015] In an optional embodiment, before the preview filter thread displays the first image texture overlaid with the first filter texture as the preview image in the graphical user interface, the following steps are included:

[0016] The preview filter thread calls a preset artificial intelligence analysis interface to analyze the result data corresponding to the texture of the first image in each frame;

[0017] The preview filter thread renders the result data as a result texture and overlays it onto the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data.

[0018] In an optional embodiment, the preview filter thread renders the result data as a result texture overlaid on the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data, and then includes:

[0019] The preview filter thread transmits the result data to the filter generator;

[0020] The filter generator merges the result data into the filter description data so that the off-screen filter thread can call the filter description data.

[0021] In an optional embodiment, in response to an image export command, an off-screen filter thread applies the filter description data to generate a static image of the scene image data for export as a scene image file, including:

[0022] In response to the image export command, the off-screen filter thread generates a second image texture of the live image data in the second texture space;

[0023] The filter description data applied to the first image texture is copied by the off-screen filter thread, and the second filter texture generated according to the filter description data is superimposed on the second image texture in the second texture space.

[0024] The second image texture, overlaid with the second filter texture, is output as a live image file in the target format, which is used as the live image file.

[0025] In an optional embodiment, after the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, the process includes:

[0026] In response to video recording commands, acquire multiple consecutive preview images from the first texture space;

[0027] The multi-frame preview images of the connection are encoded into video data using a preset encoding protocol;

[0028] In response to the video save command, the video data is output as a live video file.

[0029] In an optional embodiment, after the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, the process includes:

[0030] In response to filter switching commands, a filter list is displayed in the graphical user interface where the preview image is located, and multiple candidate filters are displayed in the filter list;

[0031] The filter generator obtains the candidate filter selected by the user as the currently active target filter, and loads the resource package of the target filter to generate the corresponding filter description data;

[0032] The filter generator notifies the preview filter thread and the off-screen filter thread to update the corresponding filter textures based on the filter description data of the currently active target filter.

[0033] According to another aspect of this application, an image filter application apparatus is provided, comprising:

[0034] The image acquisition module is configured to start the camera unit to acquire on-site image data and load the filter generator to generate filter description data for the target filter;

[0035] The on-screen preview module is configured to generate a preview image of the live image data by applying the filter description data to the preview filter thread for output display. The preview image applies a first filter texture generated according to the filter description data.

[0036] The off-screen export module is configured to respond to image export commands by having the off-screen filter thread apply the filter description data to generate a static image of the on-site image data for export as an on-site image file. The static image applies a second filter texture generated based on the filter description data.

[0037] According to another aspect of this application, an image filter application device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the image filter application method described in this application.

[0038] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the image filter application method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, performs the steps included in the method.

[0039] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0040] Compared with existing technologies, this application has several technological advantages, including but not limited to the following:

[0041] First, this application generates preview images and on-site image files based on the on-site image data collected by the camera unit through a preview filter thread and an off-screen filter thread. The preview images and on-site image files share filter description data through a filter generator. The filter generator centrally maintains the filter description data of the target filter and keeps it synchronized between the preview images and on-site image files. This ensures that the real-time preview and image export of the image content captured by the camera unit can proceed in parallel. In particular, the exported image is not limited by the memory size occupied by the texture space used by the preview image, and can obtain high-definition image quality at a faster imaging speed, avoiding a series of problems such as memory overflow, blurry images, and black screens.

[0042] Secondly, by providing an independent filter generator to centrally maintain filter description data, the two workflows of previewing real-time rendering on-screen and off-screen rendering image generation can be effectively decoupled. This is equivalent to constructing two filter chains. Between the two filter chains, data management and state interaction can be achieved through data sharing based on optional synchronization methods. This results in orderly management and a reasonable architecture, thereby creating more flexible and efficient scalability. It also makes it convenient to connect target filters from various sources for application by adhering to the specifications constrained by the filter generator.

[0043] Furthermore, since the filter generator of this application can achieve centralized management and synchronous application of filters, and provides filter description data at the rendering thread level of the camera unit, as long as the filter generator is implemented as a functional component with parsing and execution capabilities, the filter can be implemented as a resource package described in a standardized language. By parsing the resource package into filter description data through the filter generator, it can be connected to terminal devices of different systems for application, so that the filter resource package can achieve cross-platform application effects independent of the platform system's native language. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the network architecture of an exemplary application environment for this application;

[0046] Figure 2 A schematic diagram of the technical architecture of the filter generation service of this application;

[0047] Figure 3 , Figure 4 , Figure 5 , Figure 6 These are exemplary graphical user interfaces that display preview images with certain filters applied. The watermark filter, color filter, analysis filter, and framing filter are applied to each image respectively.

[0048] Figure 7 This is a flowchart illustrating one embodiment of the image filter application method of this application.

[0049] Figure 8 This is a flowchart illustrating the preview filter thread in an embodiment of this application;

[0050] Figure 9 This is a schematic diagram illustrating the process of obtaining the result data from the artificial intelligence analysis interface and drawing the corresponding result texture in the preview filter process of this application.

[0051] Figure 10 This is a schematic diagram illustrating the process of the preview filter thread transmitting the result data of the artificial intelligence analysis interface to the off-screen filter thread through the filter generator in an embodiment of this application.

[0052] Figure 11 This is a schematic diagram illustrating the process of exporting on-site image files using a second image texture in an embodiment of this application;

[0053] Figure 12 This is a schematic diagram of the process of exporting a live video file using the first image texture in an embodiment of this application;

[0054] Figure 13 This is a schematic diagram of the filter switching process in an embodiment of this application;

[0055] Figure 14 This is a schematic block diagram of the image filter application device of this application;

[0056] Figure 15 This is a schematic diagram of the structure of an image filter application device used in this application. Detailed Implementation

[0057] Please see Figure 1 The network architecture used in an exemplary application scenario of this application includes a terminal device 80 and a server 82, and the terminal device 80 and the server 82 can communicate data via the Internet or a local area network.

[0058] The terminal device can be a mobile terminal such as a smartphone, or other forms of terminal such as a personal computer or tablet computer. These terminal devices include a camera unit for capturing images of the surrounding environment to obtain on-site image data and generate corresponding on-site images. After activation, the camera unit acquires on-site image data through its camera, then creates a texture space corresponding to a preview image of the on-site image data according to its underlying driver's business logic. An image texture for the preview image is generated in this texture space, and then rendered onto a graphical user interface to display the preview image. The terminal device can install image application software that applies the method of this application to apply filters to the captured on-site images. These filters correct, enrich, or optimize the image content of the on-site images, ultimately obtaining the corresponding preview image. The preview image can be displayed in the terminal device's graphical user interface or further exported as an on-site image file, such as an on-site photo file or an on-site video file. The operating system of the terminal device is not limited and can be any released or unreleased operating system, including but not limited to iOS, Android, HomonyOS, Linux, Windows, etc.

[0059] The server 82 is used to provide filters for image application software in the terminal device, including providing a candidate filter list consisting of multiple candidate filters and providing resource packages for various filters. The server 82 can also store and provide the installation files of the image application software, and can also store and provide a component of the image application software, such as an SDK file encapsulating a computer program product implemented according to the method of this application. It can also be used to store and publish preview images generated by the terminal device. Of course, these functions of the server 82 can also be distributed among multiple servers independently responsible for individual functions, depending on the actual situation. The server 82 can refer to a front-end server in a server cluster, and the servers in the server cluster can also be deployed in a distributed manner.

[0060] The image application software of this application can take the form of different functions depending on the application scenario it serves. For example, it can be camera software, image optimization software, image evidence storage software, project management software for scenarios such as property factories or construction sites. It is easy to understand that different functions of software can constitute the image application software referred to in this application by using computer program products programmed according to the method of this application.

[0061] The image application software of this application, after running on the terminal device, can achieve... Figure 2 The technical architecture shown consists of multiple protocol layers, including an application layer, an embedding layer, an engine layer, and an execution layer. The application layer is primarily responsible for providing upper-layer services to users, such as camera filter generation and data communication with the server. The embedding layer is mainly responsible for operating system adaptation, executing relevant business logic on the CPU, and providing services such as class encapsulation, image library environment creation, error collection and reporting, event notification, filter data source acquisition, and filter download. The engine layer is mainly responsible for implementing the pass-through service between the embedding and execution layers, creating a rendering virtual machine to establish a data channel between them, allowing the rendering virtual machine to call the execution layer's rendering interface and transmit various data required for filter generation, as well as handle various events generated as a result. The execution layer mainly operates at the GPU level, providing various rendering interfaces for the engine layer's rendering virtual machine to call. When called, it executes corresponding drawing and rendering tasks, generating and displaying preview images with applied filters. The rendering interface of the execution layer belongs to the rendering interface provided by the Open Graphics Library (OpenGL). Depending on the content to be rendered, it may include layout rendering interface, text rendering interface, pattern rendering interface, etc.

[0062] After the computer program product of this application is run on the terminal device, it can implement the rendering virtual machine, so that the rendering virtual machine can execute the various related steps of the method of this application, construct a filter generator, a preview filter thread, and an off-screen filter thread, etc., to realize the application of filters, so that the preview image and on-site image file and other products can be output in the end.

[0063] The filters in this application can be assigned corresponding identifiers for indexing and retrieval. They can be implemented to perform different specific functions; for example, they can be implemented to change any one or more of the color, brightness, contrast, and saturation of the original image; they can be implemented to overlay a certain style of watermark, special effects, or beautification effect on the original image, and so on. All of these can be pre-customized to create corresponding filters. Different filters have different image effects, and all can be pre-customized, for example... Figure 3 The image shows the effect of a watermark filter applied to a preview image, where an outer frame is added to the preview image, along with the shooting time and location information. Figure 4 This demonstrates the effect of a color filter on a preview image, showing that the preview image can be enhanced with different color styles. Figure 5 The image shows the effect of an analysis filter on a preview image. The analysis filter analyzes the preview image by calling an artificial intelligence analysis interface, obtains the corresponding result data, and overlays the result data onto the preview image to achieve the effect of counting pipes in the preview image. Figure 6 This demonstrates the effect of a viewfinder filter applied to a preview image, where a viewfinder has been added to capture the preview image. The framing scale of this viewfinder can be flexibly defined by the user to achieve the desired effect.

[0064] Filters can be compiled using cross-platform languages, and these files are then packaged into a resource package containing watermark templates. The resource package can be encrypted and stored on a server. When image application software requests a filter's resource package from the server, it decrypts the package to obtain the files. The files in the resource package mainly include executable scripts encapsulating the filter's generation business logic, data configuration information for each data item in the filter, and layout configuration information for formatting the filter's display style. The executable scripts can be written in any scripting language, such as Lua or Java. Naturally, the computer program product of this application included in the image application software will also support the corresponding programming language to parse and execute the executable scripts in the resource package. The executable scripts then generate filter description data based on the data configuration and layout configuration information in the resource package. In some embodiments, the resource package may also include files containing patterns to be displayed in the image. The data configuration information and style configuration information in these files can also be compiled into the same file, such as an XML file, and the executable script can be a separate file for easy access. As can be seen, the file structure in the resource package is quite flexible. As long as the executable scripts, data configuration information, and layout configuration information required to describe the filter can be provided through various files, it is sufficient to generate the corresponding filter.

[0065] The above technical architecture enables efficient filter applications, improves filter application efficiency and image rendering effects, optimizes the development efficiency of image application software, and enhances filter update efficiency, offering numerous advantages.

[0066] Please see Figure 7 According to an image filter application method provided in this application, in one embodiment, it includes the following steps:

[0067] Step S2100: Start the camera unit to acquire on-site image data, and load the filter generator to generate filter description data of the target filter;

[0068] When the camera function in the image application software is enabled by the user, the camera unit will start and enter the shooting preview mode. In the shooting preview mode, the camera unit will drive the camera to start collecting on-site image data in order to run the preview filter thread and execute the rendering interface of the open graphics library in the GPU to generate a preview image corresponding to the on-site image data.

[0069] After entering the shooting preview mode, the filter generator implemented in this application can be loaded. The filter generator can generate filter description data of the target filter based on the executable script, data configuration information and layout configuration information carried in the resource package of the target filter specified by the user. During the execution of the executable script, the filter description data of the target filter can be generated based on the data configuration information and layout configuration information. This filter description data can be directly transmitted to the rendering interface of the open graphics library for calling, so that the corresponding filter texture can be generated based on the filter description data during the execution of the rendering interface instructions by the GPU.

[0070] In some embodiments, the filter generator can also be responsible for obtaining the resource package corresponding to the filter from the server, then parsing the resource package, extracting the executable script, data configuration information and layout configuration information, and storing the unique identifier associated with the filter in the local space for easy subsequent use.

[0071] In some embodiments, the image application software can obtain a list of candidate filters from the server, and then display each candidate filter in the graphical user interface for the user to select. When the user selects a candidate filter, the candidate filter is determined as the target filter, and then the image application software or filter generator downloads the resource package of the target filter from the server for parsing and application.

[0072] Step S2200: The preview filter thread applies the filter description data to generate a preview image of the scene image data for output display. The preview image applies a first filter texture generated according to the filter description data.

[0073] In order to display the image effect of the target filter applied in the preview image generated by the camera unit, the inherent preview image generation thread of the camera unit is transformed into a preview filter thread. By running the preview filter thread in the GPU, the first filter texture corresponding to the target filter is superimposed on the image texture corresponding to the preview image.

[0074] Specifically, after the preview filter thread obtains the filter description data generated by the filter generator, it draws the first filter texture in the first texture space used to generate the live image data captured by the camera unit according to the filter description data. The first filter texture is then superimposed on the first image texture corresponding to the preview image. Finally, the first image texture superimposed with the first filter texture is output to the graphical user interface of the terminal device for display, so that the preview image obtains the image effect of the corresponding filter.

[0075] The preview image achieves different image effects depending on the function of the target filter applied to it. For example, when the target filter is a watermark filter, the preview image contains a watermark image, which typically contains watermark information. In one embodiment, the watermark information may include the shooting time and / or shooting location information of the scene image data, so as to present scene information indicating the shooting scene in the preview image. When the target filter is an analysis filter, the preview image contains the result data obtained after the artificial intelligence analysis interface analyzes the preview image. And so on, to name a few.

[0076] The process of generating preview images in the preview filter thread can be controlled by external threads, specifically threads running on the CPU, to implement the entire process. To this end, image application software can schedule various interfaces of the Open Graphics Library through a rendering virtual machine to enable the execution of the preview filter thread.

[0077] In some embodiments, when it is necessary to export the preview image as a live video file, multiple frames of the preview image can be acquired, encoded into corresponding video images using a predetermined encoding protocol, and stored as a live video file. The live video file can be a streaming media file forming a video stream. In the live streaming application scenario of a network live broadcast room, this video stream can be pushed to the live broadcast room for playback and display, allowing viewers to see the effect of a live video stream with filters applied.

[0078] Step S2300: In response to the image export command, the off-screen filter thread applies the filter description data to generate a static image of the scene image data to export it as a scene image file. The static image applies a second filter texture generated according to the filter description data.

[0079] Since the preview filter thread is modified based on the inherent rendering thread of the camera unit, the capacity of the first texture space it uses is relatively small and insufficient to generate high-definition images. If the preview image is directly exported as a live image file, its imaging quality is difficult to achieve the expected high quality. Therefore, when the user needs to export the preview image as a live image file, an off-screen filter thread is pre-implemented to generate a higher quality live image file for the user.

[0080] Users can trigger an image export command by operating specific controls or in any other feasible way, such as the shooting button on the camera software interface. In response to the image export command, under the control of the rendering virtual machine, the off-screen filter thread creates a second texture space with a larger capacity than the first texture space. It synchronously acquires the scene image data of each frame collected by the camera unit and renders the corresponding image textures of each frame in the second texture space, which are called the second image textures. Further, the copy filter generator generates filter description data corresponding to the first filter texture of the corresponding frame in the first texture space. In the same way as the preview filter thread, the second filter texture corresponding to the first image texture is generated based on the filter description data and superimposed on the second image texture. Then, the static image corresponding to each frame of scene image data is obtained.

[0081] Based on the static images generated by the off-screen filter thread, according to the file format corresponding to the image export instruction, such as video or image files, the appropriate encoding protocol is applied to encode multiple frames or a single frame of static images into corresponding video or image files, thus obtaining on-site video files or on-site image files, both of which are on-site image files. It's easy to understand that the on-site image files generated by the off-screen filter thread have higher imaging size, resolution, and image quality than the preview image itself. Because the second texture space is larger than the first texture space and has higher memory capacity, memory overflow and imaging anomalies are less likely to occur when generating the corresponding static images.

[0082] In a vivid example, a user triggers an image export command by touching the camera preview button on the camera preview interface. This drives the off-screen filter thread to obtain the static image of the corresponding frame of the preview image displayed on the graphical user interface at the time of the trigger, and uses this static image as a live image file for the user to process further.

[0083] As can be seen from the above embodiments, this application has many technical advantages, including but not limited to the following aspects:

[0084] First, this application generates preview images and on-site image files based on the on-site image data collected by the camera unit through a preview filter thread and an off-screen filter thread. The preview images and on-site image files share filter description data through a filter generator. The filter generator centrally maintains the filter description data of the target filter and keeps it synchronized between the preview images and on-site image files. This ensures that the real-time preview and image export of the image content captured by the camera unit can proceed in parallel. In particular, the exported image is not limited by the memory size occupied by the texture space used by the preview image, and can obtain high-definition image quality at a faster imaging speed, avoiding a series of problems such as memory overflow, blurry images, and black screens.

[0085] Secondly, by providing an independent filter generator to centrally maintain filter description data, the two workflows of previewing real-time rendering on-screen and off-screen rendering image generation can be effectively decoupled. This is equivalent to constructing two filter chains. Between the two filter chains, data management and state interaction can be achieved through data sharing based on optional synchronization methods. This results in orderly management and a reasonable architecture, thereby creating more flexible and efficient scalability. It also makes it convenient to connect target filters from various sources for application by adhering to the specifications constrained by the filter generator.

[0086] Furthermore, since the filter generator of this application can achieve centralized management and synchronous application of filters, and provides filter description data at the rendering thread level of the camera unit, as long as the filter generator is implemented as a functional component with parsing and execution capabilities, the filter can be implemented as a resource package described in a standardized language. By parsing the resource package into filter description data through the filter generator, it can be connected to terminal devices of different systems for application, so that the filter resource package can achieve cross-platform application effects independent of the platform system's native language.

[0087] Based on any embodiment of this application, please refer to Figure 8 The preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, including:

[0088] Step S2210: The preview filter thread generates the first image texture of the live image data in the first texture space;

[0089] As mentioned earlier, the preview filter thread is modified based on the inherent rendering business logic of the camera unit. Therefore, when the camera unit starts up and enters the shooting preview mode, the preview filter thread will draw the first image texture of each frame in the first texture space according to the on-site image data captured by the camera unit. It is easy to understand that when no target filter is applied, the first image texture of each frame will be directly rendered and displayed on the screen as a preview image. This preview image is the original image generated by the camera unit, and no filter has been applied yet.

[0090] Step S2230: The preview filter thread generates the first filter texture of the target filter by superimposing it on the first image texture according to the filter description data;

[0091] When the user specifies a target filter for the preview image, the preview filter thread obtains the filter description data generated and passed through by the filter generator. The filter description data describes the various information required for the filter texture. Based on the filter description data, the preview filter thread can overlay and draw the first filter texture in the first texture space on the basis of the first image texture, so that the first image texture contains the first filter texture and achieves the effect of applying the target filter.

[0092] Step S2250: The preview filter thread displays the first image texture, overlaid with the first filter texture, as the preview image in the graphical user interface.

[0093] Finally, the preview filter thread, according to its inherent rendering logic, renders the first image texture of each frame overlaid with the first filter texture and displays it on the screen, corresponding to the preview images of each frame in the graphical user interface. Thus, the preview filter thread has implemented the application of the target filter, intuitively presenting the filter application effect to the user.

[0094] As can be seen from the above embodiments, the application of the target filter in the preview image is implemented by modifying the rendering business logic of the camera unit's inherent preview function. Therefore, the process of applying the target filter is executed in the GPU, that is, the application of the target filter is implemented at the system bottom layer. The drawing of the target filter does not need to rely on the view layer controls provided by the native development language of the image application software, but directly reaches the system bottom layer to draw and implement in the GPU. Since the filter generator has the ability to independently parse the resource package of the target filter, combining these two advantages, the parsing and application of the filter can be decoupled from the native development language environment of the image application software, thereby obtaining the scalability of the filter application and ensuring that the filter can be applied across platforms.

[0095] Based on any embodiment of this application, please refer to Figure 9 Before the preview filter thread displays the first image texture, overlaid with the first filter texture, as the preview image in the graphical user interface, it includes:

[0096] Step S2241: The preview filter thread calls the preset artificial intelligence analysis interface to analyze the result data corresponding to the texture of the first image in each frame;

[0097] In some filters, it is desirable to perform image analysis on the scene images obtained by the camera unit, obtain corresponding result data, and present it accordingly in the scene images. For example, Figure 5 The on-site image shown applies an analysis filter. This filter sequentially counts the pipe components in the image and generates corresponding count numbers for each component. For this analysis filter, a corresponding artificial intelligence analysis interface can be called to analyze the preview image of the relevant frame. After the AI ​​analysis interface completes its analysis of the preview image, the returned result data can be obtained.

[0098] Step S2242: The preview filter thread renders the result data as a result texture and overlays it onto the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data.

[0099] After the preview filter thread obtains the result data returned by the artificial intelligence analysis interface, it further draws the result texture corresponding to the result data based on the first image texture. Subsequently, as the first image texture is rendered and displayed on the screen, the result data can be viewed in the preview image of the graphical user interface.

[0100] The above embodiments further leverage the analytical capabilities of the artificial intelligence analysis interface to obtain the desired result data for the analysis filter, and synthesize the result data into the preview image to achieve richer functions, enabling the preview filter thread to have richer filter application capabilities when applying the target filter to the user.

[0101] Based on any embodiment of this application, please refer to Figure 10 The preview filter thread renders the result data as a result texture and overlays it onto the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data, including:

[0102] Step S2243: The preview filter thread transmits the result data to the filter generator;

[0103] While the result data generated through the artificial intelligence analysis interface can be rendered into the preview image, sometimes users also want the corresponding result data image to be included in the live image in their exported live image file. To this end, in one embodiment, the result data can be obtained by calling the artificial intelligence analysis interface based on the live image generated by the second image texture in the off-screen filter thread, just like in the preview filter thread, and then drawn onto the second image texture. However, this embodiment can provide a more resource-efficient solution to achieve shared utilization of the result data obtained by the preview filter thread.

[0104] Therefore, the preview filter thread can transmit the obtained result data to the filter generator, so that the filter generator can pass the result data through to the off-screen filter thread for rendering.

[0105] Step S2244: The filter generator merges the result data into the filter description data so that the off-screen filter thread can call the filter description data.

[0106] After receiving the result data from the preview filter thread, the filter generator can convert the result data into part of the filter description data and then pass it through to the off-screen filter thread. Therefore, when the off-screen filter thread generates the second filter texture based on the filter description data, in addition to generating the second filter texture of the filter itself, the result texture of the result data will also be generated accordingly. The final exported live image file will then contain the corresponding result data.

[0107] As can be seen from the above embodiments, with the help of the filter generator, all kinds of result data obtained by analyzing the preview image in the preview filter thread can be passed through to the off-screen filter thread. The off-screen filter thread can share the analysis results without calling the artificial intelligence analysis interface separately, realize the copying of the result data, ensure that the generated static image is consistent with the corresponding preview image in terms of image content, and achieve the effect of what you see is what you get.

[0108] Based on any embodiment of this application, please refer to Figure 11 In response to an image export command, the off-screen filter thread applies the filter description data to generate a static image of the scene image data for export as a scene image file, including:

[0109] Step S2310: In response to the image export command, the off-screen filter thread generates the second image texture of the live image data in the second texture space;

[0110] When a user triggers an image export command, the off-screen filter thread, in response to this command, is responsible for generating the static image to be exported. If the off-screen filter thread has already created a second texture space, it will draw the corresponding image textures (i.e., the second image textures) frame by frame from the live image data captured by the camera unit within that second texture space. The drawing method of the second image texture by the off-screen filter thread can be the same as that of the preview filter thread, that is, it also calls the interface of the system's underlying open graphics library to achieve the drawing and generation of the second image texture.

[0111] Step S2320: The off-screen filter thread copies the filter description data applied to the first image texture, and superimposes the second filter texture generated according to the filter description data onto the second image texture in the second texture space;

[0112] In order to draw the image texture of the target filter, i.e., the second filter texture, the off-screen filter thread needs to obtain the filter description data of the target filter. Since the filter description data has already been generated centrally by the filter generator, the filter generator can simply transfer the filter description data to the off-screen filter thread, thereby enabling the off-screen filter thread to copy the filter description data used by the preview filter thread.

[0113] Furthermore, the off-screen filter thread uses the obtained filter description data to overlay the second filter texture of the target filter onto the second image texture in the second texture space. After the second image texture with the overlaid second filter texture is converted into image data, the corresponding static image can be obtained.

[0114] Step S2330: Output the second image texture with the second filter texture superimposed as a target format on-site image file, as the on-site image file.

[0115] To obtain a live image file, the second image texture, overlaid with a second filter texture, is converted into a live image file according to a target format such as JPG or PNG. In addition to containing the corresponding still image, other metadata information can be added to this live image file. This live image file is also a type of live image file.

[0116] As can be seen from the process of generating static images by the off-screen filter thread as shown in the above embodiments, the off-screen filter thread shares the filter description data of the target filter through the filter generator, which can be consistent with the preview image and maintain the same filter to obtain the corresponding image effect. Since the second texture space used by the off-screen filter thread is larger than the first texture space used by the preview filter thread, the static image generated by the off-screen filter thread is expected to be superior to the preview image in terms of scale, resolution, and image quality. In this process, the filter generator plays the role of passing through various data related to the filter, decoupling the technical logic of rendering to the screen and compositing output for two different purposes, so that the export of the image is not affected by the preview, ensuring that the export is more efficient and smooth.

[0117] Based on any embodiment of this application, please refer to Figure 12 After the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, the process includes:

[0118] Step S3100: Respond to the video recording command and obtain multiple consecutive preview images from the first texture space;

[0119] In many scenarios, it is necessary to generate videos with target filters applied, where the image quality requirements for the videos are relatively low, that is, the video resolution is lower than a preset threshold, such as 1080P, 720P, 480P, etc. For these scenarios, the preview image can be used directly for generation.

[0120] To this end, users can trigger a video recording command by pressing the recording start button on the camera software interface. In response to the video recording command, the system begins to acquire multiple preview images continuously generated from the first texture space until the user touches the recording end button, thereby obtaining multiple consecutive preview images.

[0121] Step S3200: Encode the multi-frame preview image of the connection into video data using a preset encoding protocol;

[0122] To generate the corresponding live video file, during the recording process, continuously acquired multiple frames of preview images are encoded according to a preset encoding protocol, transforming them into corresponding video data. As the recording continues, the encoding process also continues, thus continuously generating the corresponding video data. The encoding protocol used to obtain the video data can be any feasible protocol, such as H.264, AVI, MPEG, etc.

[0123] Step S3300: Respond to the video save command and output the video data as a live video file.

[0124] When the user touches the record end button, a video save command is triggered. In response to this video save command, the video data can be output as a live video file, enabling the preview image to be quickly exported as a video.

[0125] As can be seen from the above embodiments, using preview images to generate live video files balances image quality requirements and operating efficiency, enabling the goal to be achieved quickly. It also achieves the division of labor between the off-screen filter thread and the preview filter thread, allowing users to generate video files of different qualities using different image outputs according to different needs.

[0126] Based on any embodiment of this application, please refer to Figure 13 After the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, the process includes:

[0127] Step S4100: In response to the filter switching command, display a filter list in the graphical user interface where the preview image is located, and display multiple candidate filters in the filter list;

[0128] When a user is dissatisfied with the current application's filters and wants to switch them, a filter reset function can be provided. To achieve this, a filter switching entry can be provided in the graphical user interface; when the user interacts with this entry, the corresponding filter switching command is triggered.

[0129] In response to the filter switching command, the image application software queries the server to obtain a filter list containing multiple candidate filters. Then, the identifiers of each candidate filter in the filter list are displayed in the graphical user interface where the preview image is located for the user to select.

[0130] Step S4200: The filter generator obtains the candidate filter selected by the user as the currently effective target filter, and loads the resource package of the target filter to generate the corresponding filter description data;

[0131] When a user selects a candidate filter from the displayed filter list, the selected candidate filter becomes the currently active target filter. The filter generator can download the resource package of the currently active target filter, or if the resource package is already stored in the local space, it can be directly called. Specifically, it calls various files in the resource package so that the filter generator can generate the filter description data of the currently active target filter based on the resource package, thereby replacing the filter description data of the currently applied filter.

[0132] Step S4300: The filter generator notifies the preview filter thread and the off-screen filter thread to update the corresponding filter textures according to the filter description data of the currently active target filter.

[0133] After the filter generator generates the filter description data corresponding to the currently effective target filter, it can synchronously notify the preview filter thread and the off-screen filter thread based on a message notification mechanism or through an interface call. This allows both the preview filter thread and the off-screen filter thread to obtain the filter description data of the currently effective target filter and update the corresponding first and second filter textures based on the filter description data. This ensures that the subsequently generated preview images and static images are updated to reflect the filter effect of the new target filter selected by the user, thus completing the filter switching.

[0134] As can be seen from the above embodiments, when a user switches filters, the filter generator plays a central and unified role in switching new filters. Based on the new filter selected by the user, it generates the corresponding filter description data and synchronously controls the preview filter thread and the off-screen filter thread to update the filter. It can be seen that the filter generator plays a central role in the two filter workflows, unifies the management of filter-related data, and facilitates the realization of a platform-based architecture.

[0135] Please see Figure 14 According to one aspect of this application, an image filter application device includes an image acquisition module 2100, an on-screen preview module 2200, and an off-screen export module 2300. The image acquisition module 2100 is configured to initiate a camera unit to acquire on-site image data and load a filter generator to generate filter description data for a target filter. The on-screen preview module 2200 is configured to use a preview filter thread to apply the filter description data to generate a preview image of the on-site image data for output display, wherein the preview image applies a first filter texture generated based on the filter description data. The off-screen export module 2300 is configured to respond to an image export command by using an off-screen filter thread to apply the filter description data to generate a static image of the on-site image data for export as an on-site image file, wherein the static image applies a second filter texture generated based on the filter description data.

[0136] Based on any embodiment of this application, the on-screen preview module 2200 includes: a preview original image generation module, configured to generate a first image texture of the on-site image data in a first texture space by a preview filter thread; a preview filter generation module, configured to generate a first filter texture of the target filter by superimposing the first image texture on the first image texture according to the filter description data by the preview filter thread; and a preview image display module, configured to display the first image texture superimposed with the first filter texture as the preview image in the graphical user interface by the preview filter thread.

[0137] Based on any embodiment of this application, the on-screen preview module 2200 of this application further includes: an artificial intelligence analysis module, configured to call a preset artificial intelligence analysis interface by the preview filter thread to analyze the result data corresponding to the first image texture of each frame; and an analysis result application module, configured to draw the result data as a result texture by the preview filter thread and superimpose it on the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data.

[0138] Based on any embodiment of this application, the on-screen preview module 2200 of this application further includes: an analysis result pass-through module, configured to transmit the result data to the filter generator by the preview filter thread; and a description data update module, configured by the filter generator to merge the result data into the filter description data, so that the off-screen filter thread can call the filter description data.

[0139] Based on any embodiment of this application, the off-screen export module 2300 includes: an off-screen original image generation module, configured to generate a second image texture of the scene image data in a second texture space by an off-screen filter thread in response to an image export command; an off-screen filter generation module, configured to copy the filter description data applied to the first image texture by the off-screen filter thread, and superimpose a second filter texture generated according to the filter description data on the second image texture in the second texture space; and an off-screen image output module, configured to output the second image texture superimposed with the second filter texture as a scene image file in a target format, as the scene image file.

[0140] Based on any embodiment of this application, the image filter application device of this application includes: a recording start module, configured to respond to a video recording command and acquire a series of consecutive preview images from a first texture space; a video encoding module, configured to encode the connected series of preview images into video data using a preset encoding protocol; and a video output module, configured to respond to a video save command and output the video data as a live video file.

[0141] Based on any embodiment of this application, the image filter application device of this application includes: a switching response module, configured to respond to a filter switching command, display a filter list in the graphical user interface where the preview image is located, and display multiple candidate filters in the filter list; a resource loading module, configured to have the filter generator obtain the candidate filter selected by the user as the currently effective target filter, load the resource package of the target filter to generate the corresponding filter description data; and a filter updating module, configured to have the filter generator notify the preview filter thread and the off-screen filter thread to update the corresponding filter texture according to the filter description data of the currently effective target filter.

[0142] Another embodiment of this application also provides an image filter application device. For example... Figure 15 The diagram shows the internal structure of an image filter application device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement an image filter application method.

[0143] The processor of this image filter application device provides computing and control capabilities to support the operation of the entire device. The memory of the device can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the image filter application method of this application. The network interface of the device is used for communication with a terminal.

[0144] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the image filter application device to which the solution of this application is applied. A specific image filter application device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0145] In this embodiment, the processor is used to execute... Figure 14 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules or sub-modules. The network interface is used to realize data transmission between user terminals or servers. In this embodiment, the non-volatile readable storage medium stores the program code and data required to execute all modules in the image filter application device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0146] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the image filter application method of any embodiment of this application.

[0147] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0148] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0149] In summary, this application optimizes the implementation architecture of filter applications by centrally providing filter description data for the preview filter thread and the off-screen filter thread through a filter generator. It enables on-screen preview and image export of the on-site image data collected by the camera unit, respectively, which can improve the efficiency and effect of filter applications and has broad application prospects.

Claims

1. A method for applying an image filter, characterized in that, include: The camera unit is activated to acquire on-site image data, and the filter generator is loaded to generate filter description data for the target filter. A preview image of the live image data is generated by the preview filter thread using the filter description data and displayed as an output. The preview image applies a first filter texture generated based on the filter description data. In response to an image export command, the off-screen filter thread applies the filter description data to generate a static image of the scene image data for export as a scene image file. The static image applies a second filter texture generated based on the filter description data.

2. The image filter application method according to claim 1, characterized in that, The preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, including: The preview filter thread generates the first image texture of the live image data in the first texture space; The preview filter thread generates the first filter texture of the target filter by overlaying it onto the first image texture based on the filter description data; The preview filter thread displays the first image texture, overlaid with the first filter texture, as the preview image in the graphical user interface.

3. The image filter application method according to claim 2, characterized in that, Before the preview filter thread displays the first image texture, overlaid with the first filter texture, as the preview image in the graphical user interface, it includes: The preview filter thread calls a preset artificial intelligence analysis interface to analyze the result data corresponding to the texture of the first image in each frame; The preview filter thread renders the result data as a result texture and overlays it onto the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data.

4. The image filter application method according to claim 3, characterized in that, The preview filter thread renders the result data as a result texture and overlays it onto the first image texture of the corresponding frame, so that the output preview image contains the corresponding result data, including: The preview filter thread transmits the result data to the filter generator; The filter generator merges the result data into the filter description data so that the off-screen filter thread can call the filter description data.

5. The image filter application method according to claim 2, characterized in that, In response to an image export command, the off-screen filter thread applies the filter description data to generate a static image of the scene image data for export as a scene image file, including: In response to the image export command, the off-screen filter thread generates a second image texture of the live image data in the second texture space; The filter description data applied to the first image texture is copied by the off-screen filter thread, and the second filter texture generated according to the filter description data is superimposed on the second image texture in the second texture space. The second image texture, overlaid with the second filter texture, is output as a live image file in the target format, which is used as the live image file.

6. The image filter application method according to any one of claims 1 to 5, characterized in that, After the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, the process includes: In response to video recording commands, acquire multiple consecutive preview images from the first texture space; The consecutive preview images are encoded into video data using a preset encoding protocol; In response to the video save command, the video data is output as a live video file.

7. The image filter application method according to any one of claims 1 to 5, characterized in that, After the preview filter thread applies the filter description data to generate a preview image of the scene image data for output display, the process includes: In response to filter switching commands, a filter list is displayed in the graphical user interface where the preview image is located, and multiple candidate filters are displayed in the filter list; The filter generator obtains the candidate filter selected by the user as the currently active target filter, and loads the resource package of the target filter to generate the corresponding filter description data; The filter generator notifies the preview filter thread and the off-screen filter thread to update the corresponding filter textures based on the filter description data of the currently active target filter.

8. An image filter application device, characterized in that, include: The image acquisition module is configured to start the camera unit to acquire on-site image data and load the filter generator to generate filter description data for the target filter; The on-screen preview module is configured to generate a preview image of the live image data by applying the filter description data to the preview filter thread for output display. The preview image applies a first filter texture generated according to the filter description data. The off-screen export module is configured to respond to image export commands by having the off-screen filter thread apply the filter description data to generate a static image of the scene image data for export as a scene image file. The static image applies a second filter texture generated based on the filter description data.

9. An image filter application device, comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images