Multimedia data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a multimedia data processing method and device, electronic equipment, computer readable storage medium and product, and relates to the field of data processing. The method comprises: determining multimedia data to be processed in a target application; in the case that a first plug-in is called by the target application, in response to receiving a selection operation of a target function control of a first web component through an operation panel of the first plug-in, sending target related data corresponding to the target function control to the first web component through the first plug-in, so that the first web component performs processing corresponding to the target function control on the multimedia data based on the target related data, and obtains processed multimedia data; and displaying the processed multimedia data. The present application provides a wider development space for the display style of the plug-in, can meet the diversified display demand of the user for the plug-in panel, and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a multimedia data processing method, apparatus, electronic device, computer-readable storage medium, and product. Background Technology

[0002] The Unified Extension Platform, as a plugin development framework, provides a secure, stable, and lightweight way to develop plugins for applications, allowing developers to create plugins for their applications. Existing technologies based on the Unified Extension Platform are used to develop plugins for image processing applications, which can improve the efficiency of applications in image editing and processing based on the functionality of the plugins.

[0003] However, existing plugins developed using the unified extension platform have limitations in supporting web page content element types. They only support a small number of web page content elements and cannot support all types of web page content elements. This makes it impossible for technicians to achieve the complex styles they want when using HTML, CSS and other technologies to build plugin interfaces, such as some advanced animation effects and custom font styles. This seriously affects the user experience and functionality of the plugins and cannot meet users' needs for high-quality and diverse plugins. Summary of the Invention

[0004] This application provides a multimedia data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can solve the problem that existing plugins developed through a unified extension platform support limited webpage content elements, resulting in the inability to achieve the desired style on the plugin interface. The technical solution is as follows:

[0005] According to a first aspect of the embodiments of this application, a multimedia data processing method is provided, the method comprising:

[0006] The target application is identified as containing multimedia data to be processed. The target application has a first plugin with an operation panel, the operation panel of which displays at least one functional control based on a first web page component.

[0007] When the first plugin is invoked by the target application, in response to receiving a selection operation for the target function control of the first web page component through the operation panel of the first plugin, the target-related data corresponding to the target function control is sent to the first web page component through the first plugin, so that the first web page component performs processing on the multimedia data based on the target-related data, corresponding to the target function control, to obtain processed multimedia data; the target-related data is the related data of the multimedia data to be processed.

[0008] The processed multimedia data is displayed.

[0009] As an optional embodiment, the operation panel of the first plugin is implemented based on the first web page component;

[0010] The step of responding to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, and sending the target-related data corresponding to the target function control to the first web page component through the first plugin, includes:

[0011] In response to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, the following operation is triggered:

[0012] Based on the first web page component, the data acquisition request corresponding to the target function control is sent to the first plugin;

[0013] When the first plugin receives the data acquisition request, the first plugin accesses the target application programming interface of the target application to obtain the target-related data corresponding to the data acquisition request, and provides the target-related data to the first web page component.

[0014] As an optional embodiment, the first webpage component sends a data retrieval request corresponding to the target functional control to the first plugin, including:

[0015] The data acquisition request is transmitted to the first communicator of the web page component based on the first web page component. The first communicator then sends the data acquisition request to the second communicator of the first plugin based on the message passing mechanism. The data acquisition request is then transmitted to the first plugin through the second communicator.

[0016] Providing the target-related data to the first webpage component includes:

[0017] The first plugin transmits the target-related data to the second communicator, which then sends the target-related data to the first communicator based on a message passing mechanism. The first communicator then sends the target-related data to the first web page component.

[0018] As an optional embodiment, the first webpage component registers a first proxy module of the first plugin, the first plugin registers a second proxy module of the first plugin, and the first webpage component transmits the data acquisition request to the first communicator, including:

[0019] When the first webpage component initiates the data retrieval request, the first proxy module intercepts the data retrieval request sent by the first webpage component and performs the following operations:

[0020] The data request method of the data acquisition request is determined, the data acquisition request is encapsulated according to the agreed protocol corresponding to the data acquisition request based on the data request method, and the encapsulated data acquisition request is transmitted to the first communicator, wherein the agreed protocol is an agreed data format between the first proxy module and the second proxy module;

[0021] Sending the data acquisition request to the first plugin via the second communicator includes:

[0022] The second communicator transmits the encapsulated data acquisition request to the second proxy module, so that the second proxy module can perform the following operations:

[0023] The data request method is determined based on the encapsulated data acquisition request, and the target-related data corresponding to the data acquisition request is obtained based on the data request method.

[0024] As an optional embodiment, the data request method includes a first request method or a second request method, wherein the first request method is to request data by calling a system interface, and the second request method is to request data by accessing system module attributes;

[0025] The determination of the data request method for the data acquisition request includes:

[0026] If the data acquisition request carries a first identifier, the data request method is determined to be a second request method; if the data acquisition request does not carry the first identifier, the data request method is determined to be a first request method. The first identifier is related to the attributes of the system module to be accessed.

[0027] The step of encapsulating the data acquisition request according to the agreed protocol corresponding to the data request method includes:

[0028] If the data request method is the first request method, the interface path and corresponding condition parameters of the called interface in the data acquisition request are encapsulated according to the agreed protocol corresponding to the first request method to obtain the encapsulated data acquisition request;

[0029] If the data request method is determined to be the second request method, the attribute path of the accessed attribute is obtained from the data acquisition request, and the attribute path is encapsulated to obtain the encapsulated data acquisition request.

[0030] As an optional embodiment, the method further includes:

[0031] The method further includes:

[0032] When an update prompt pop-up is displayed on the operation panel of the first plugin, in response to receiving an update confirmation operation, the update data of the configuration information is obtained through the first web page component, and the configuration information of the first web page component is updated based on the update data;

[0033] The update notification pop-up is displayed when the first webpage component determines that the corresponding configuration information has changed.

[0034] As an optional embodiment, when the updated data includes data related to the display style of the functional controls of the operation panel, updating the configuration information of the first webpage component based on the updated data includes:

[0035] Based on the data related to the display style of the function controls in the operation panel, the style of the relevant function controls in the operation panel is updated.

[0036] As an optional embodiment, providing the target-related data to the first webpage component includes:

[0037] The first plugin determines the data identifier of the data acquisition request; it encapsulates the target-related data and the data identifier to obtain cached data;

[0038] The target-related data and the data identifier are provided to the first web page component. If the first web page component determines that the target-related information has been garbage collected, the data identifier is sent to the first plugin so that the cached data can be deleted through the first plugin.

[0039] As an optional embodiment, the processed multimedia data includes at least one of the following:

[0040] The processed multimedia data is obtained through the first web page component;

[0041] If the first plugin also includes a display panel, the processed multimedia data is obtained through the first web page component or through the second web page component, wherein the display panel is implemented based on the second web page component;

[0042] The processed multimedia data being displayed includes at least one of the following:

[0043] When the processed multimedia data is obtained through the first web page component, the processed multimedia data is displayed in the operation panel through the first web page component, or the processed multimedia data is sent to the first plugin through the first web page component, so that the processed multimedia data is displayed in the display interface of the target application through the first plugin.

[0044] When the processed multimedia data is obtained through the second web page component, the processed multimedia data is displayed in the display panel through the second web page component.

[0045] As an optional embodiment, the step of sending the data acquisition request to the second communicator based on the first communicator using a message passing mechanism, and then sending the data acquisition request to the first plug-in based on the second communicator, includes:

[0046] The first communicator, based on a message passing mechanism, sends the data acquisition request to the first plug-in via the second communicator through the first communication link between the first and second communicators;

[0047] The step of sending the target-related data to the first communicator based on the second communicator using a message passing mechanism, and sending the data acquisition request to the first webpage component based on the first communicator, includes:

[0048] The second communicator, based on a message passing mechanism, sends the data acquisition request to the first web page component through the first communicator via the second communication link between the first and second communicators.

[0049] As an optional embodiment, the first communicator sends the data acquisition request to the second communicator of the first plug-in based on a message passing mechanism, including:

[0050] When the data volume of the data acquisition request exceeds a preset threshold, the data acquisition request is compressed based on the first communicator, and the compressed data acquisition request is sent to the second communicator based on the message passing mechanism of the first communication module. The compressed data acquisition request is then decompressed by the second communicator to obtain the decompressed data acquisition request.

[0051] The first plugin transmits the target-related data to the second communicator, which then sends the target-related data back to the first communicator based on a message passing mechanism, including:

[0052] When the amount of target-related data exceeds a preset threshold, the target-related data is compressed using the second communicator. The compressed target-related data is then sent to the first communicator via the second communicator using a message passing mechanism. The first communicator decompresses the compressed target-related data to obtain a decompressed data acquisition request.

[0053] According to a second aspect of the embodiments of this application, a multimedia data processing apparatus is provided, the apparatus comprising:

[0054] A determination module is used to determine the multimedia data to be processed in a target application, wherein the target application has a first plug-in with an operation panel, and the operation panel of the first plug-in displays at least one functional control based on a first web page component.

[0055] The acquisition module is configured to, when the first plugin is invoked by the target application, respond to receiving a selection operation for a target function control of the first webpage component through the operation panel of the first plugin, send the target-related data corresponding to the target function control to the first webpage component through the first plugin, so that the first webpage component performs processing on the multimedia data based on the related data, corresponding to the target function control, to obtain processed multimedia data; the target-related data is the related data of the multimedia data to be processed.

[0056] The display module is used to display the processed multimedia data.

[0057] According to a third aspect of the present application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described data processing method.

[0058] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the above-described data processing method.

[0059] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described data processing method.

[0060] The beneficial effects of the technical solutions provided in this application are:

[0061] After embedding the first webpage component into the plugin's operation panel, web technologies (such as HTML, CSS, and JavaScript) can be used to build a rich plugin panel. This allows developers to add more complex and diverse functional controls to the plugin panel, solving the problem of limited display styles in the original plugin panel and enhancing the plugin's functional extensibility. By responding to the selection operation of the target functional control, the system can determine the user's desired data processing based on the target function. The first plugin then sends the target-related data to the first webpage component, enabling the first webpage component to obtain the target-related data and perform processing on the multimedia data corresponding to the target function. Displaying this processed multimedia data provides users with an intuitive view of the multimedia data corresponding to the target function, improving the user experience. This embodiment of the application provides a wider development space for the plugin's display style, meeting users' diverse display needs for the plugin panel and improving the user experience. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0063] Figure 1 This is a schematic diagram of the system architecture for implementing the multimedia data processing method provided in the embodiments of this application;

[0064] Figure 2 A flowchart illustrating a multimedia data processing method provided in an embodiment of this application;

[0065] Figure 3 A schematic diagram of the operation panel of a first plug-in provided in an embodiment of this application;

[0066] Figure 4 A schematic diagram illustrating the loading state of a display panel as provided in an embodiment of this application;

[0067] Figure 5 A schematic diagram illustrating the display state of a display panel provided in an embodiment of this application;

[0068] Figure 6 A schematic diagram of an application interface provided in an embodiment of this application;

[0069] Figure 7 A schematic diagram illustrating interaction between communicators, provided as an embodiment of this application;

[0070] Figure 8 A schematic diagram of a connectivity bridge on the webpage component side provided in an embodiment of this application;

[0071] Figure 9 A schematic diagram of a first plug-in side connectivity bridge provided in an embodiment of this application;

[0072] Figure 10 A flowchart illustrating a caching mechanism provided in an embodiment of this application;

[0073] Figure 11 A flowchart illustrating a multimedia data processing method provided in an embodiment of this application;

[0074] Figure 12 This is a schematic diagram illustrating an application scenario for implementing the multimedia data processing method provided in an embodiment of this application.

[0075] Figure 13 This is a schematic diagram of the structure of a multimedia data processing device provided in an embodiment of this application;

[0076] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0077] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0078] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0079] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0080] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0081] First, let's introduce and explain several terms used in this application:

[0082] The Unified Extensibility Platform (UXP) is a plugin development framework that allows developers to create plugins for applications, such as adding custom functionality or automating image processing tasks. Developers can use the APIs and tools provided by UXP to build and deploy plugins that can be used across multiple applications, thereby improving development efficiency and plugin versatility.

[0083] The existing technical solutions for developing UXP plugins are as follows:

[0084] S1. Development environment preparation; S2. Install the target application client; S3. Install the UXP development tools; S4. Install Node.js and npm (Node Package Manager) for dependency management and build tool management; S5. Create a plugin project; S6. Create a new plugin project using Adobe's command-line tools (such as @adobe / uxp-cli) or the visual tools of the UXP development tools. The plugin project includes the necessary files and directories required by UXP; S7. Develop plugin functionality; build the user interface and interaction logic using UXP-supported JavaScript, HTML, and CSS features, and interact with the target application using the system APIs provided by UXP.

[0085] The debugging and testing process during development involves using the UXP developer tools to check for errors and performance issues in the code. The UXP development tools help users load plugins into the target application for testing, ensuring functionality and meeting expectations, while also providing a developer debugging panel.

[0086] Once development is complete, the project is packaged and released by using a command-line tool to package the plugin into a distributable format (.ccx file) and then publishing the plugin to a plugin application market for users to download and install.

[0087] The plugin is maintained and updated regularly based on user feedback and technical updates. This includes fixing bugs, adding new features, and re-releasing the plugin as a new package.

[0088] Because UXP supports limited web features—for example, only a few JavaScript, CSS, and HTML features are supported, and it doesn't support some new JavaScript features, a large number of CSS features, or some new HTML features—the lack of support for many features makes it difficult to implement the UI / UX of plugins. Furthermore, UXP's poor web feature support and inability to use mature web component libraries, coupled with the lack of a complete component library, mean that complex interactive functions can only be implemented using standard HTML tags and attributes, combined with a limited number of supported CSS and JavaScript features, requiring manual creation of custom user interface components. This results in low efficiency in UXP plugin development, requiring developers to spend more time and effort to implement the same functionality, which to some extent limits the speed and quality of plugin development.

[0089] WebView is a component used to display web content in an application. It can be embedded into the application's user interface to display web pages, HTML content, web forms, etc. WebView provides a browser engine that enables the application to load and display web pages and interact with them. However, WebView and the application run in different processes, making information transfer between WebView and the application difficult in current technologies.

[0090] The postMessage messaging mechanism is an API for cross-document communication, allowing secure message passing between windows or frames from different origins. In web development, due to the same-origin policy, documents from different origins typically cannot communicate directly. postMessage provides a mechanism that allows developers to send and receive messages between different windows without violating the same-origin policy. However, postMessage has some limitations when transmitting messages. Because it cannot share the function execution context when transmitting data, it cannot support function transmission, and it may alter the data structure of some data during transmission.

[0091] Hot Module Replacement (HMR) is a technique that dynamically replaces, adds, or removes modules during application runtime without reloading the entire page. This technique greatly improves development efficiency and makes the development process smoother. Existing UXP plugins do not support hot module replacement.

[0092] The multimedia data processing methods, apparatuses, electronic devices, computer-readable storage media, and computer program products provided in this application are intended to solve the above-mentioned technical problems of the prior art.

[0093] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0094] Figure 1 This application provides a schematic diagram of a system architecture for implementing a multimedia data processing method, including a terminal 10, an application server 20, and a function server 30. The terminal includes a client application that can process multimedia content, specifically images, and the application can be image processing software. The terminal 10 also includes a plugin deployed on the application. This plugin can be developed based on UXP, and its operation panel embeds a web page component, which can be a WebView component. The web page component displays a user interface built using Web technologies (such as HTML, CSS, and JavaScript), showcasing rich web page content styles in the plugin panel and providing functional controls for user operation.

[0095] Application server 20 is the server for applications. The server is used to maintain the version information of applications. Application clients can be deployed locally for use. When the configuration of the application is updated, the server packages the new version of the application and uploads it to the application market. Terminal 10 downloads the new version of the application and deploys it locally for use.

[0096] The function server 30 can be a server for multimedia data processing tools. The function server 30 provides its own access interface, through which web page components can access and use the function server to process multimedia data. For example, when the multimedia data is an image, the function server can be a server with an image processing model deployed. The web page components in terminal 10 can have multiple function controls, each corresponding to a function server. Terminal 10 can also include multiple web page components, each corresponding to multiple function servers. Developers can develop based on user needs. The web page components can be WebViews. Developing plugins and operation panel display styles based on WebViews allows for hot module replacement during debugging, greatly improving debugging efficiency.

[0097] In this system architecture, the plugin's operation panel can be understood as a container displaying the content of the web component. When a user selects multimedia data and clicks a function control on the plugin's operation panel, the terminal confirms the selected multimedia data. The plugin then interacts with the application, obtaining the target-related data and passing it to the web component. The web component can then determine the corresponding interface of the function server based on the function control. The function server processes the multimedia data to obtain the processed multimedia data, which can be displayed on the plugin's operation panel or on the application's interface. For example, if developers want to implement a text-to-image function based on the web component, they can display an "Input Text" input box on the web component, along with multiple style effects provided by the text-to-image function server. Users can input text, click on their preferred effect, and then click the "Generate" control on the web component. The web component then obtains the target-related data and calls the text-to-image function server to generate the corresponding image. Currently, the operation panel of existing plugins cannot display effect styles. This embodiment utilizes web technologies (such as HTML, CSS, and JavaScript) corresponding to the web component to construct a rich user interface, thereby enabling the display of more complex effect styles.

[0098] The aforementioned servers (application server 20 and function server 30) can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. In some embodiments, the terminal 10 can install a client for an image processing application, which can be a smartphone, tablet, laptop, desktop computer, smartwatch, smart voice interaction device, smart home appliance, in-vehicle terminal, etc., but is not limited to these. Communication between the application server 20 and the terminal 10, and between the function server 30 and the terminal 10, can be achieved through wired or wireless networks.

[0099] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can be any network, including but not limited to any combination of local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), mobile, wired or wireless networks, private networks, or virtual private networks.

[0100] This application provides a multimedia data processing method, which can be executed by terminal 10, such as... Figure 2 As shown, the method includes:

[0101] S201. Identify the multimedia data to be processed in the target application.

[0102] The target application has a first plugin with an operation panel installed, and the operation panel of the first plugin displays at least one functional control based on the first web page component.

[0103] In this embodiment of the application, a client of the target application is running on the terminal. The target application can be a multimedia data processing application. The following description uses multimedia data as images and the target application as a drawing application as an example.

[0104] Users can launch the target application in the terminal by selecting its icon. Once launched, the application interface is displayed on the terminal. Users can interact with the application through buttons, menus, icons, and other elements on the interface. They can open the image to be processed or generate a blank image based on the application. They can then create artwork on the blank image using the application's drawing tools and process the image using image processing tools. For example, the application includes cropping, adjustment, and filter tools. Users can use the cropping tool to remove unwanted parts of the image, adjust the composition to better suit design requirements, change the image's tone and brightness using the adjustment tool, and add corresponding filter effects using the filter tool.

[0105] In this embodiment, a first plugin is installed in the target application. The first plugin includes an operation panel displaying at least one functional control based on a first webpage component. The webpage component can be a WebView, a component used to embed webpage content into an application. It allows developers to directly load webpage content into the application interface. In this embodiment, the webpage component is embedded in the operation panel of the first plugin. Specifically, the first plugin is developed based on a unified extension platform. When developing the first plugin, the unified extension platform only develops the plugin project and the operation panel. After embedding the WebView in the operation panel, a UI interface for user interaction is developed based on the WebView. The operation panel can then be used as a container for displaying webpage content.

[0106] In this embodiment of the application, when the target application starts, all plugins installed on the target application can be loaded. That is, after the target application starts, the plugins of the target application are automatically started. Optionally, the application interface of the target application can display a plugin launch control. After the user clicks the plugin launch control, the corresponding plugin is launched.

[0107] In this embodiment, the unified extension platform provides a system API, which includes the API of the target application and a general API provided by the platform. The API of the target application is the interface provided by the target application, which is exposed to the same extension platform. Therefore, plugins developed based on the same extension platform can call the API.

[0108] For example, within the target application, the unified extension platform provides APIs for that application, allowing developers to access and manipulate its documents, layers, tools, and other functionalities. These APIs are specifically designed for interacting with a particular application, giving developers deep control over the application's behavior and data. In addition to the target application's APIs, several general-purpose APIs are also provided. These APIs are application-independent, providing common functionalities and tools for plugin development. For example, APIs for file I / O operations allow plugins to read and write files; there are also APIs for network requests for communicating with external services. These general-purpose APIs enable plugins to implement cross-application functionalities such as data synchronization and remote calls. Firstly, plugins can interact with applications by accessing system APIs.

[0109] In this embodiment, the multimedia data to be processed can be an image that the user wants to process based on a webpage component. That is, the multimedia data to be processed is an image to be processed. The image to be processed can be a complete image displayed in the target application, or a part of that complete image. Specifically, the user can use a drawing tool to select the image they want to process from the displayed complete image. After the user selects the image to be processed, the terminal can determine the image to be processed based on the target application client.

[0110] S202, when the first plugin is invoked by the target application, in response to receiving a selection operation for the target function control of the first web page component through the operation panel of the first plugin, the target-related data corresponding to the target function control is sent to the first web page component through the first plugin, so that the first web page component performs processing on the multimedia data based on the target-related data, corresponding to the target function control, and obtains the processed multimedia data.

[0111] In this embodiment, when the first plugin is invoked by the target application, the first plugin is in a running state within the target application. As described above, the first plugin can be started by default when the target application starts, or it can be started by the user clicking the startup control of the first plugin on the application interface of the target application. After the first plugin is started, its operation panel can be displayed on the application interface of the target application. This operation panel can be an active panel, which the user can drag to any position on the application interface.

[0112] In this embodiment of the application, the operation panel displays one or more functional controls. After selecting an image, the user can click on the target functional control of the first webpage component. At this time, the first webpage component can determine the target function that the user wants to use based on the target functional control. Figure 3 The above, Figure 3This is a schematic diagram of an operation panel for a first plug-in provided in an embodiment of this application. The content displayed on the operation panel is based on a first webpage component. The operation panel includes the function name "Intelligent Drawing" of the first webpage component, and can display the drawing target layer selected by the user in the target application. The operation panel also includes a control to enable guided drawing functionality. If the user is a new user of the first plug-in or needs precise drawing, they can choose to enable it. The first webpage component can provide a guided mask for the user, which includes preset keywords. The user can select keywords to input, or they can choose not to select guided drawing and select keywords according to their needs. The keyword can include two input boxes: one for specifying the drawing elements the user wants to add, and another for specifying the drawing elements they do not want to add. The first webpage component also provides different drawing styles, including comics, 3D realistic animation, beautiful ukiyo-e, sci-fi, Chinese classical, and other drawing styles. Other drawing styles can be displayed by clicking "More". Different drawing styles can correspond to different image processing models. Different image processing models can be deployed on one functional server or on different servers. Figure 3 You can also choose how to display the processed image. In the image, 4 was selected, indicating that the final display includes 4 display areas.

[0113] After the user enters text or selects a style, clicking the "Generate" function control allows the first webpage component to capture the user's input and selection. The user can then use the "Generate" function control to determine how the target user wants to process the image based on the entered text and selected format. Figure 3 In the example, the user did not input any keywords, but only selected the painting model style "cybersci-fi". Therefore, the resulting processed image is an image that has been transformed into a "cybersci-fi" style image.

[0114] As an optional embodiment, the first plugin can determine that the user has clicked a target function control through the target panel, determine the target-related data required for the target function corresponding to the target function control, and this target-related data is the relevant data of the image to be processed. The relevant data of the image to be processed is obtained by calling the system API and then sent to the first web page component. In this embodiment, the target functions corresponding to the target function controls are different, and the required target-related data are not entirely the same. Specifically, the target-related data for different target functions may include common data, such as the document name of the image to be processed and the document storage path. The common data for processing in the target-related data for different target functions also includes data required for the implementation of the target function. For example, if the target function can stylize the image to be processed, the corresponding target-related data may be the pixel data and image format of the image to be processed; if the target function can add filters to the image to be processed, the corresponding target-related data may be the image size information.

[0115] After receiving the target-related data, the first web page component can access the function server corresponding to the target function, so that the function server can process the multimedia data according to the relevant data of the image to be processed. After obtaining the processed image, the function server returns the processed image to the first web page component.

[0116] As an optional embodiment of this application, Figure 3 The session target in the middle can be displayed based on the first plugin. The first plugin can interact with the target application to obtain the image selected by the user in the target application. The first plugin can display the image on the operation panel so that the user can preview the image he or she selected before clicking the "Generate" function control, detect whether there is an error when selecting the image, and prevent the image processed by the first web page component from being the image that the user wants to process.

[0117] As an optional embodiment of this application, the operation panel of the first plugin is implemented based on the first web page component;

[0118] The step of responding to receiving a selection operation for a target function control of the first webpage component through the operation panel of the first plugin, and sending the target-related data corresponding to the target function control to the first webpage component through the first plugin, includes:

[0119] In response to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, the following operation is triggered:

[0120] Based on the first web page component, the data retrieval request corresponding to the target function control is sent to the first plugin;

[0121] When the first plugin receives a data acquisition request, it accesses the target application programming interface of the target application to obtain the target-related data corresponding to the data acquisition request, and provides the target-related data to the first web page component.

[0122] In this embodiment of the application, the operation panel can be implemented based on a unified extension platform and a first web page component. When developing the first plugin, the unified extension platform generates the framework of the operation panel, which can be regarded as a container. The first web page component is embedded in this container, and the user can view the UI interface presented by the first web page component on the operation panel.

[0123] In this embodiment, when a user clicks on a target function control, since the target function control is displayed on a first webpage component, the first webpage component can capture the user's click operation. As an optional embodiment, the first webpage component can set a corresponding detector for each function control and capture user operations based on the detector.

[0124] In this embodiment, after the first webpage component determines that the target function control has been clicked, it generates one or more data acquisition requests corresponding to the target function control. In this embodiment, the first webpage component and the target application are in different processes. When the target application starts, it starts its own process. When the plugin starts, the first webpage component also starts its own process along with the webpage panel. Since the first webpage component and the target application are in different processes, the first webpage component cannot directly access the target application to obtain the relevant data of the image to be processed. Therefore, in this embodiment, the plugin is used as a communication intermediary. The first webpage component sends the data acquisition request to the first plugin. Then, the first plugin obtains the relevant data of the image to be processed by accessing the system API according to the data acquisition request. After obtaining the relevant data of the image to be processed, the first plugin sends the relevant data of the image to be processed to the first webpage component, so that the first webpage component can process the image according to the relevant data of the image to be processed.

[0125] In this embodiment, the first application can request data by calling system interfaces or by accessing system properties. For example, it can obtain the pixel data of a layer using the command line `app.getPixels()`, where `app` is the module name. This `app` module can be a system module, meaning it can be a module implemented through the functionality of the target application or through the functionality of a unified extension platform. This `app` module provides the `getPixels` method, which can be called via the command line to retrieve the pixel data of the image layer. Alternatively, the `id` attribute of the `app` module can be accessed via the command line `app.id._value` to obtain information about the current document, which is used to record the document identifier `id` to which the image belongs when generating the processed image.

[0126] In this embodiment of the application, the interaction between the first web page component and the target application is realized by using the first plugin as a communication intermediary between the first web page component and the target application.

[0127] As an optional embodiment of this application, the first webpage component sends a data acquisition request corresponding to the target functional control to the first plugin, including:

[0128] The data acquisition request is transmitted from the first web page component to the first communicator of the web page component. The first communicator then sends the data acquisition request to the second communicator of the first plugin based on the message passing mechanism. The data acquisition request is then transmitted to the first plugin through the second communicator.

[0129] Provide target-related data to the first webpage component, including:

[0130] The first plugin transmits target-related data to the second communicator, which then sends the target-related data back to the first communicator based on a message passing mechanism. The first communicator then sends the target-related data to the first web page component.

[0131] In this embodiment, to enable information transmission between the first webpage component and the first plugin, a communication bridge can be established between them. This bridge includes a first communicator located on the first webpage component's side and a second communicator located on the first plugin's side. Both the first and second communicators have the function of invoking a message passing mechanism. This message passing mechanism can be `postMessage`. When the first webpage component sends a data retrieval request to the first plugin, the request is first passed from the first webpage component to the first communicator. The first communicator then invokes the message passing mechanism to send the data retrieval request to the first plugin. The second communicator on the data retrieval request side includes a mechanism for capturing messages sent from the first communicator based on the message passing mechanism, thereby obtaining the data retrieval request. After obtaining the data retrieval request, the second communicator sends it to the first plugin.

[0132] In this embodiment, after the first plugin obtains the relevant data of the image to be processed based on the data acquisition request, it passes the relevant data of the image to be processed to the second communicator. The second communicator calls the message passing mechanism to send the relevant data of the image to be processed to the first web page component. The first communicator on the first web page component side includes a function to obtain the message sent from the second communicator based on the message passing value, thereby obtaining the relevant data of the image to be processed, and then sending the relevant data of the image to be processed to the first web page component.

[0133] In this embodiment of the application, message passing between the first web page component and the first plugin is realized based on the message passing mechanism, thereby realizing message passing between the first web page component and the target application. This enables the first web page component to obtain relevant data of the image to be processed, and to perform corresponding processing on the image based on the relevant data of the image to be processed, thereby realizing the corresponding target function.

[0134] S203. Display the processed multimedia data.

[0135] In this embodiment of the application, the processed image can be displayed so that the user can intuitively see the processing result and obtain the image of the style they need. In addition, the first plugin also provides a download function so that the user can download the processed image to their local device.

[0136] In this embodiment, by embedding a first webpage component into the plugin's operation panel, web technologies (such as HTML, CSS, and JavaScript) can be used to build a rich user interface. This allows developers to create more complex and diverse functional controls, overcoming the limitations of the plugin's original interface display capabilities and enhancing its functional extensibility. By responding to the selection operation of the target functional control, the system can determine the user's desired data processing of multimedia data based on the target function. The first plugin then sends the target-related data to the first webpage component, enabling the first webpage component to obtain the target-related data and perform processing on the multimedia data corresponding to the target function. Displaying this processed multimedia data provides users with an intuitive view of the multimedia data corresponding to the target function, improving the user experience. This embodiment opens up a broader development space for plugin functionality, meeting users' needs for high-quality and diverse plugins.

[0137] As an optional embodiment, the processed multimedia data includes at least one of the following:

[0138] The processed multimedia data is obtained through the first web page component;

[0139] If the first plugin also includes a display panel, the processed multimedia data is obtained through a first web page component or a second web page component, wherein the display panel is implemented based on the second web page component;

[0140] Display the processed multimedia data, including at least one of the following:

[0141] When the processed multimedia data is obtained through the first web page component, the processed multimedia data is displayed in the operation panel through the first web page component, or the processed multimedia data is sent to the first plugin through the first web page component, so that the processed multimedia data is displayed in the display interface of the target application through the first plugin.

[0142] When the processed multimedia data is obtained through the second web page component, the processed multimedia data is displayed in the display panel through the second web page component.

[0143] In this embodiment, when developing specific business logic, developers can choose to display the processed image on a panel or interface. Specifically, the processed image can be displayed on the operation panel of the first plugin or on the display interface of the target application. For example, the processed image can replace the original image, which is the image to be processed. If the image to be processed is a part of an image, the processed image can also replace the complete image. Alternatively, an additional layer can be generated in the target application to display the processed image. In addition, the processed image can also be displayed on other panels of the first plugin.

[0144] As an optional embodiment, the first plugin includes at least two panels. In addition to the operation panel, it may also include a display panel, which is implemented based on a second web page component and can display the processed image on the display panel. The operation panel of the first plugin is used to implement the image processing function, while the display panel focuses on presenting the processed image with the best visual effect. Users can clearly view the final effect of the image here. Different panels focus on their respective tasks, which greatly improves the convenience of operation and the user experience.

[0145] like Figure 4 As shown, Figure 4 This is a schematic diagram illustrating a display panel loading state as provided in an embodiment of this application. Figure 4 The image shows four areas, corresponding to... Figure 3 When selected, four areas are displayed, and users can also click on them. Figure 4 The page switching buttons below display four areas of other pages. While the function server is processing the image, the image generation area on the display panel shows "Generating, please wait." It should be noted that after the first webpage component calls the function server for image processing, it can then call the function of the second webpage component to display the task results page. After the second webpage component is called, it retrieves the processed image, displaying a loading status interface during the retrieval process. For example... Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the display state of a display panel according to an embodiment of this application. After acquiring the corresponding processed image, the display panel renders it, and the four areas of the display panel respectively display processed images corresponding to the same model style. Users can click... Figure 5 The page switching buttons below allow you to access other processed images. After a drawing task is generated, the processed image can be stored in the drawing record, where users can find previously generated processed images.

[0146] like Figure 6 As shown, Figure 6This is a schematic diagram of an application interface provided in an embodiment of this application. After the processed image is displayed on the display panel, the application interface of the target application can be as follows: Figure 6 As shown in the image, the interface includes three display areas and an options bar. The options bar provides functionality for the target application, which can open multiple documents. The image shows two documents open, labeled "Figure 0" and "...". Figure 1 Each document can include multiple layers. The image shows layer 0 of the second document. The three display areas are the operation panel, the display panel, and the image operation area for the target application. This image operation area displays... Figure 2 The corresponding image in layer 0, the operation panel is as follows Figure 3 As shown, the display panel is as follows Figure 5 As shown in the diagram, the user selects the entire image on layer 0 as the image to be processed. Optionally, the user can also use the target application's cropping tool to select the image on layer 0 and use the selected image as the image to be processed. In addition, in this target application, after the user selects a layer, the image on that layer may be covered by other layers. In this case, when selecting the image on that layer as the image to be processed, the session target displayed on the first plugin may be different from the image displayed by the target application.

[0147] This application embodiment realizes diversified display of processed images, which can be designed according to user needs, thereby improving the user's experience and satisfaction when using the first plugin.

[0148] In this embodiment, to display the processed image on the operation panel, the processed multimedia image can be obtained through a first webpage component, and the processed multimedia data can be displayed on the operation panel through the first webpage component. Specifically, WebView1 can send an HTTP request to the function server by calling the function server's generateImage interface to initiate a generation task. The function server can return a task ID (taskId) indicating that the generation task has started processing. After receiving the task ID indicating that the generation task has started processing, WebView1 adds a new task page to its page by calling the AddTaskPage method. The specific command line can be AddTaskPage(taskId). The task page generated by the first webpage component can periodically query the function server for the task status through HTTP polling. If the task status is "completed", the generated image is obtained from the server; if the task status is "in progress", polling continues; if the task status is "failed", polling stops and the user is prompted.

[0149] After the first web page component obtains the processed image, it can render the processed image and display it in the first web page component. Since the first web page component is embedded in the operation panel, the processed image can be displayed on the operation panel.

[0150] The processed image can also be displayed on the target application's display interface. Since the first web page component and the target application cannot directly transmit information, the processed image can be passed to the first plugin through the first web page component, and the processed image can be displayed on the target application's display interface by calling the system API through the first plugin.

[0151] As an optional embodiment, when the first web page component passes the processed image to the first plugin, the processed image can be passed to the first communicator through the first web page component, and then the processed image can be passed to the first plugin through the second communicator via the first communicator. This allows the message passing mechanism to be invoked based on the first communicator to pass the processed image to the first plugin.

[0152] In this embodiment, to display the processed image on the operation panel, the processed multimedia image can be obtained through a first webpage component, and the processed multimedia data can be displayed on the display panel through a second webpage component. Specifically, after receiving the task ID that indicates the task generation task has started processing, the first webpage component notifies the second webpage component that a new task has been created. This utilizes the method call capability between webpage components. After receiving the notification, the second webpage component adds a new task page to its page by calling the AddTaskPage method. The specific command line can be AddTaskPage(taskId). The task page generated by the second webpage component can periodically query the task status from the function server via HTTP polling. If the task status is "completed", the generated image is retrieved from the server; if the task status is "in progress", polling continues; if the task status is "failed", polling stops and the user is prompted.

[0153] It is important to understand that the first webpage component and the second webpage component are in different processes, therefore they cannot directly transmit information. As an optional embodiment, the first plugin can be used as a communication middleware to enable interaction between the first and second webpage components. The second webpage component can include a third communicator, and the second and third communicators can also transmit information based on a message passing mechanism. Therefore, interaction between the first webpage component and the first plugin can be achieved through the first and second communicators, and interaction between the second webpage component and the first plugin can be achieved through the second and third communicators. The first webpage component can use this interaction method to notify the second webpage component that a new task has been created, enabling the second webpage component to obtain the processed image and display it on the display panel. As an optional embodiment, each webpage component can include a data sharing module, allowing each webpage component to obtain data from other webpage components or use modules or functions built into other webpage components through the aforementioned interaction method. Therefore, in this embodiment of the application, the first web page component and the second web page component can interact with each other. Either web page component can obtain the processed image and transmit it to the other, but the efficiency of data transmission is not improved. After setting a web page component for displaying the processed image, the processed image can be obtained directly through the HTTP interface access function processor based on the web page component, instead of obtaining the processed image through the form of data transmission from the web page component.

[0154] like Figure 7 As shown, Figure 7 This is a schematic diagram illustrating interaction between communicators, provided in an embodiment of this application. Web page components can invoke system APIs and access data and methods of other web page components based on a connectivity bridge. Specifically, when invoking the system API, each web page component uses the system API through its own connectivity bridge and a first plugin connectivity bridge, utilizing the first plugin to access the system API. When accessing data and invoking methods of other web page components, access to data and invocation of methods can be achieved through the first plugin's distributed shared state, based on the connectivity bridge on its own side, the first plugin connectivity bridge, and the connectivity bridge of the opposite web page component.

[0155] In this embodiment of the application, the ability of the first communicator, the second communicator and the third communicator to call the message passing mechanism enables efficient and flexible interaction between the first web page component and the second web page component, thereby improving the application scenarios of the first plugin.

[0156] As an optional embodiment of this application, the method involves sending a data acquisition request to a second communicator based on a message passing mechanism using a first communicator, and then sending the data acquisition request to a first plug-in based on the second communicator, including:

[0157] The first communicator, based on a message passing mechanism, sends a data acquisition request to the first plug-in via the second communicator through the first communication link between the first and second communicators.

[0158] Based on the second communicator, target-related data is sent to the first communicator using a message passing mechanism. The first communicator then sends a data retrieval request to the first webpage component, including:

[0159] The second communicator, based on a message passing mechanism, sends a data acquisition request to the first web page component through the first communicator via a second communication link between the first and second communicators.

[0160] In this embodiment, data transmission between the first communicator and the second communicator can utilize different links to achieve asynchronous data transmission. Specifically, during the process of the first communicator sending a data acquisition request to the second communicator based on a message passing mechanism, data transmission can be performed through the first communication link between the first and second communicators. Similarly, during the process of the second communicator sending target-related data to the first communicator based on a message passing mechanism, data transmission can be performed through the second communication link between the second and first communicators.

[0161] As an optional embodiment, the first communication link may include a data transmission link between a first transmitter in the first communicator and a second receiver in the second communicator, and the second communication link may include a data transmission link between a second transmitter and a first receiver in the second communicator. Specifically, the first communicator can invoke a message passing mechanism through the first transmitter to send corresponding data to the second communicator, and the second communicator can capture the corresponding data sent by the first transmitter based on the second receiver. Similarly, the second communicator can invoke a data transmission mechanism through the second transmitter to send response data to the first communicator, and the first communicator can capture the corresponding data sent by the second transmitter based on the first receiver.

[0162] As an optional embodiment of this application, the first communicator sends a data acquisition request to the second communicator of the first plug-in based on a message passing mechanism, including:

[0163] When the amount of data in the data acquisition request exceeds a preset threshold, the data acquisition request is compressed based on the first communicator, and the compressed data acquisition request is sent to the second communicator based on the message passing mechanism of the first communication module. The compressed data acquisition request is decompressed by the second communicator to obtain the decompressed data acquisition request.

[0164] The first plugin transmits target-related data to the second communicator, which then sends the target-related data back to the first communicator based on a message passing mechanism, including:

[0165] When the amount of target-related data exceeds a preset threshold, the target-related data is compressed using the second communicator. The compressed target-related data is then sent to the first communicator via the second communicator using a message passing mechanism. The first communicator decompresses the compressed target-related data to obtain the decompressed data acquisition request.

[0166] Considering that the message passing mechanism does not support the transmission of large amounts of data, in this embodiment, before each communicator sends corresponding data to other communicators based on the transmitter calling the message passing mechanism, the transmitter can first determine the size of the data. If the data size is greater than a preset threshold, the transmitter compresses the data. After each communicator receives data sent by other communicators through the message passing mechanism based on the receiver, it can determine whether the sent data is in compressed format. If it is in compressed format, the receiver can decompress the data to obtain the original data.

[0167] As an optional embodiment, before determining the size of the data, the transmitter can first determine the data type. If the data is of the target type, it further determines whether the data size exceeds a preset threshold. If it exceeds the preset threshold, the data is divided into multiple data blocks, each data block is compressed, and the data is sent in multiple batches. The target type can be a binary array (TypedArray). Since this type of data generally has a large size, transmitting it through block compression can improve transmission efficiency.

[0168] In this embodiment of the application, during data transmission, the data is compressed by the transmitter and decompressed by the receiver, which improves the efficiency of data transmission and avoids the problem of transmission failure when transmitting large amounts of data in the message transmission mechanism.

[0169] As an optional embodiment of this application, a first proxy module of a first plugin is registered in the first webpage component, and a second proxy module of the first plugin is registered in the first plugin. The first webpage component transmits a data acquisition request to the first communicator, including:

[0170] When the first webpage component initiates a data retrieval request, the first proxy module intercepts the data retrieval request sent by the first webpage component and performs the following operations:

[0171] The data request method of the data acquisition request is determined, the data acquisition request is encapsulated according to the agreed protocol corresponding to the data acquisition request, and the encapsulated data acquisition request is transmitted to the first communicator. The agreed protocol is the agreed data format between the first proxy module and the second proxy module.

[0172] The data acquisition request is sent to the first plugin via the second communicator, including:

[0173] The second communicator passes the encapsulated data retrieval request to the second proxy module, so that the second proxy module can perform the following operations:

[0174] The data request method is determined based on the encapsulated data retrieval request, and the target-related data corresponding to the data retrieval request is obtained based on the data request method.

[0175] In this embodiment of the application, a first proxy module of a first plugin is registered in the first web page component, and a second proxy module of the first plugin is registered in the first plugin. Since the first plugin can access the system API, the first proxy module and the second proxy module can be regarded as system proxy modules. The proxy module includes a module that proxies the target application function and a module that proxies the unified extension platform function.

[0176] In this embodiment, when the first webpage component sends a data acquisition request to the first plugin, it typically obtains the relevant data of the image to be processed by calling functions and accessing attributes. However, when the communicator transmits command-line data such as function calls and attribute access based on the message passing mechanism, there may be transmission failures. To solve this problem, this embodiment converts the command-line data of the data acquisition request into string data and establishes an agreement for the first and second proxy modules to define the format for sending or receiving data.

[0177] The SystemModuleProxy class can be instantiated through the connection bridge on the first web component side, and the first proxy module of the system module can be registered. The proxy module can also be called a proxy object. When a data acquisition request is issued in the first web component, the SystemModuleProxy proxy object will dynamically intercept the data acquisition request, determine the data acquisition request method, obtain the corresponding agreement protocol according to the data acquisition method, encapsulate the data acquisition request based on the agreement protocol, and automatically send the encapsulated data acquisition request to the plugin through the first communicator.

[0178] Similarly, the SystemModuleProxy class can be instantiated through the connection bridge on the first plugin side, and a second proxy module of the system module can be registered. The second proxy module receives the encapsulated data acquisition request transmitted by the second communicator, determines the data request format of the data acquisition request based on the agreed protocol, and obtains the relevant data of the image to be processed based on the data request format. It should be noted that, in this embodiment, the first plugin obtains the relevant information of the image through the second proxy module. By agreeing on a protocol between the first and second proxy modules, the specific format conversion of the data acquisition request is encapsulated in the first and second proxy modules, so that the web plugin can obtain the relevant data of the image to be processed in the same way as the first plugin, thus providing the same development experience as the native image.

[0179] The first proxy module can convert command-line data into a string according to the agreed protocol and encapsulate the string. After the second proxy module obtains the encapsulated data, it reads the encapsulated data according to the agreed protocol and converts the string data into command-line data to obtain the original data access request. Based on the execution of the corresponding command line by the second proxy module, it calls functions or accesses attributes to obtain the relevant data of the image to be processed.

[0180] In this embodiment, a data acquisition request may include two data request methods: calling a function and accessing an attribute. It should be understood that the command lines corresponding to calling a function and accessing an attribute have different appearances and contain different information. Therefore, in this embodiment, the data format of the data acquisition request can be converted based on the data request method. For example... Figure 8 As shown, Figure 8 This is a schematic diagram of a connectivity bridge on the web component side provided in an embodiment of this application. The system API proxy module in the connectivity bridge of the web component is the first proxy module. The system API proxy module includes a UXP proxy module and a target application proxy module. The UXP proxy module includes a file module and a storage module. The file module includes an interface for processing files and related attribute information. The storage module includes an interface for storing cached data and related attribute information. The communication base is implemented based on a communicator, which can call the plugin system API, access other web component data, access other web component methods, and share data and methods.

[0181] It should be noted that the communicator in this embodiment has both sending and receiving functions. It can receive messages sent by the proxy module, pass the messages to the transmitter, and send messages processed by the receiver to the proxy module.

[0182] The communicator can instantiate message transmitters and message receivers, as mentioned earlier. The input to the message transmitter is the data to be sent. Data transmission preparation includes message creation, message encapsulation, and message transmission. After the message is sent, the transmission result can be passed to the communicator so that the communicator can confirm the transmission result. The message receiver can be created with a message detector to capture data sent by the second communicator based on the message passing mechanism. The message receiver is also used to wait for messages, receive messages, and after receiving a message, it also includes message verification and parsing to confirm the sender of the message, obtain the data in the message, further process the message, and decompress the data in the message. The communicator can obtain the decompressed data through message callbacks.

[0183] like Figure 9 As shown, Figure 9 This is a schematic diagram of a first plug-in side connectivity bridge provided in an embodiment of this application. Figure 9 The structure and Figure 8 Similarly, it should be noted that the main function of the communicator on the first plugin side is to respond to system API call requests, realize data sharing between web page components, realize method sharing between web page components, and push message notifications to web page components.

[0184] In this embodiment, by registering a first agent module and a second agent module, and setting agreed protocols for the first agent module and the second agent module under different data request methods, the first agent module can convert the data format of the data acquisition request according to the agreed protocol, and the second agent module can determine the data request method corresponding to the data acquisition request according to the agreed protocol, and obtain the relevant data of the image to be processed based on the corresponding data request method. This solves the problem of anomalies when the first communicator calls the message passing mechanism to transmit command-line data acquisition requests.

[0185] As an optional embodiment of this application, the data request method includes a first request method or a second request method. The first request method is to request data by calling a system interface, and the second request method is to request data by accessing system module attributes.

[0186] The determination of the data request method for the data acquisition request includes:

[0187] If the data acquisition request carries a first identifier, the data request method is determined to be the second request method; if the data acquisition request does not carry a first identifier, the data request method is determined to be the first request method. The first identifier is related to the attributes of the system module to be accessed.

[0188] According to the agreed protocol corresponding to the data request method, the data acquisition request is encapsulated, including:

[0189] If the data request method is the first request method, the interface path and corresponding condition parameters of the called interface in the data acquisition request are encapsulated according to the agreed protocol corresponding to the first request method to obtain the encapsulated data acquisition request;

[0190] If the data request method is determined to be the second request method, the attribute path of the accessed attribute is obtained from the data retrieval request, the attribute path is encapsulated, and the data retrieval request is obtained after format conversion.

[0191] It should be understood that the data retrieval requests issued by the first web page component will have different command line formats depending on the data retrieval method. For example, when the data retrieval method is to call the getPixels function of the app module, the command line format corresponding to the data retrieval request is app.getPixels(params). When the data retrieval method is to access the document id attribute, the command line format corresponding to the data retrieval request is app.activeDocument.id.__value.

[0192] The first proxy module can determine the data retrieval method corresponding to the data retrieval request based on the form of the command line. As an optional embodiment, when registering the first proxy module, a getter method is registered for it. This getter method is used to determine the data retrieval request method. The first proxy module intercepts the data retrieval request through the getter method and returns a standard request function `standardMsgFunc` inside the getter. The `standardMsgFunc` function is also bound to a `__value` attribute, namely `standardMsgFunc._value`.

[0193] In this embodiment, the first proxy module can be understood as an empty shell. Whether the first web page component accesses a property or calls an interface, it executes a getter method. If it accesses a property, and the executed code includes the first identifier ".__value", the getter method can obtain the property path JSONpath at the time of execution. The getter method encapsulates the property path according to the agreed protocol. This property path is the string form of the command line. jsonPath is the path of the system API being called. Taking the app module of the target application as an example, if app.activeDocument.id is accessed, the corresponding jsonPath is app.activeDocument.id. If the first webpage calls an API (i.e., calls a function), the first proxy module will also call the getter method. After this getter method is called, it detects the absence of the first identifier "value" and returns the function identifier "standardMsgFunc" to indicate that the data request format corresponding to this data retrieval request is a function call. It also obtains the API path and condition parameters of the called API, for example, in `app.getPixels(params)`, the API path is "app.getPixels" and the condition parameter is "params". The function identifier, API path, and condition parameters are then combined into a string according to the agreed protocol and encapsulated accordingly. If the first proxy module receives a simple data transmission request instead of a data retrieval request, the getter method will not be triggered, and the data can be sent to the communicator through the proxy module.

[0194] The conditional parameter specifies the specific data that the function will process. This parameter can be empty; if empty, the function will process default data. For example, `app.imaging.getPixels()` retrieves the pixel data of the entire layer, i.e., the pixel data of the complete image, while `imaging.getSelection(targetSectionParams)` retrieves the data of the selected area within the layer.

[0195] In this embodiment of the application, by determining whether the data acquisition request includes a first identifier, the data request format corresponding to the data acquisition request is determined, and different data are encapsulated into a pre-defined protocol according to the data request format, so that the second proxy module can execute the corresponding data acquisition request according to the pre-defined protocol.

[0196] As an optional embodiment of this application, when an update prompt pop-up is displayed on the operation panel of the first plugin, in response to receiving an update confirmation operation, the update data of the configuration information is obtained through the first web page component, and the configuration information of the first web page component is updated based on the update data;

[0197] The update notification pop-up is displayed when the first webpage component determines that the corresponding configuration information has changed.

[0198] In this embodiment of the application, the developer can package the first plugin and the relevant files of each corresponding web page component to generate a package file, and publish it on the content delivery network CDN for users to download;

[0199] The first time a user downloads the packaged file from the network, they install the first plugin in the target application. The panel of the first plugin embeds the corresponding web component. If the developer updates the relevant files of any web component, they can upload the updated incremental or full file to the CDN. The CDN will generate a new version of the web component file based on the uploaded file. The CDN includes a database for storing web component versions.

[0200] When users actually use the webpage, each webpage component can access the database through an interface at preset time intervals to determine the latest version number of the current webpage component. After confirming that the version has changed, the webpage component will display a pop-up window on the corresponding panel, prompting the user to refresh the interface. The pop-up window has an "OK" button. After the user clicks it, the webpage component can pull the latest version of the configuration file from the CDN. By distributing the configuration file through the CDN, the update can be quickly pushed to all users, realizing the update of the webpage component. The webpage component can also cache the latest version of the configuration file locally. By caching, the time spent repeatedly loading the configuration file is reduced, and the response speed of the webpage is accelerated. Even if the user is offline, the webpage component can still load the configuration file from the local cache.

[0201] As an optional embodiment, the target application's interface includes an options bar for a first plugin. This options bar includes a refresh control for the first plugin, which the user can click at any time. The refresh control in the options bar is a command-line configuration of the first plugin, which executes the refresh of the first plugin behind the scenes. Since the web page component is embedded in the first plugin's panel, the web page component, as a webpage, is also refreshed. At this time, each web page component corresponding to the first plugin accesses the database interface to determine if there is a new version of the file. If a new version exists, it directly pulls the latest version of the configuration file. The options bar is implemented based on the command line, and its style is set in the first plugin's manifest configuration file.

[0202] In this embodiment of the application, by embedding web page components in the panel, remote updates to the functionality of the web page components are supported, reducing the cost of plugin function updates and bug fixes, shortening the version iteration cycle, and enabling agile development and rapid distribution of the latest version of the configuration file.

[0203] As an optional embodiment, when the updated data includes data related to the display style of the functional controls on the operation panel, updating the configuration information of the first web page component based on the updated data includes:

[0204] Based on data related to the display style of the function controls in the operation panel, the styles of the relevant function controls in the operation panel are updated.

[0205] In this embodiment, when the updated data includes data related to the display style of each control on the operation panel, the display style of the control on the operation panel needs to be changed. Updating the display style can include deleting controls, adding controls, and changing the display style of controls. For example, if the data update changes the button background color to green, the text color to blue, and the font size to 16px, the configuration information of the web page components will change accordingly. After the update, the button style in the operation panel will be re-rendered according to the new configuration, ultimately displaying a green background, blue text, and a 16px font size.

[0206] In this embodiment of the application, by updating the style of the functional spaces in the operation panel, the display effect of the web page components is kept consistent with the latest data, thereby improving the user experience and making the web page interface more dynamic and flexible.

[0207] As an optional embodiment of this application, providing target-related data to a first webpage component includes:

[0208] The first plugin identifies the data identifier of the data retrieval request; it then encapsulates the target-related data and the data identifier to obtain cached data.

[0209] The target-related data and data identifier are provided to the first web page component. If the first web page component determines that the relevant information of the multimedia data has been garbage collected, the data identifier is sent to the first plugin so that the cached data can be deleted by the first plugin.

[0210] like Figure 10 As shown, Figure 10 This is a flowchart illustrating a caching mechanism provided in an embodiment of this application. In this embodiment, a SystemCache class can be registered in a first plugin to uniformly manage result caching, where the result can be related data of the image to be processed. Specifically, the caching mechanism provided in this embodiment includes the following steps:

[0211] S1. Register cache class; S2. Communicator initialization; S3. Call system API / access system module properties; S4. Communicator sends message; S5. Receive system API call / module property access request; S6. Parse message; S7. Execute API call / property access; S8. Cache result; S9. Send result; S10. Get message result; S11. Register garbage collection callback mechanism; S12. Use result; continuously check if the result is not referenced, until the result is not referenced, continue to execute S13-S17, S13 memory reclamation; S14. Communicator sends message: clear cache; S15. Receive system API call / module property access request; S16. Parse message; S17. Clear cached result (based on PosId and JsonPath).

[0212] In this embodiment, the request identifier postId can be used as the data identifier cacheKey, or the data identifier cacheKey can be generated by combining postId and jsonPath. Here, postId is a unique ID that identifies the message request when the communicator sends a message. After the result caching is complete, the result and cacheKey are returned to the first web page component via the communicator. After receiving the result, the first web page component checks whether the data is used and still referenced based on the registered garbage collection callback mechanism FinalizationRegistry. When the garbage collection callback mechanism detects that the result is no longer referenced, it reclaims the memory of the result. The first web page component sends a clear cache message to the first plugin via the first communicator. The second communicator of the first plugin receives and parses the message to obtain the request identifier and the interface path or request path corresponding to the result. Based on the interface path and request path, the cached result is cleared.

[0213] In this embodiment of the application, the system performance can be improved by caching the relevant data of the image to be processed. The results of cross-process API calls and attribute access are cached within the plugin itself, which facilitates secondary calls and access based on the results.

[0214] In this embodiment, by setting a connection bridge between the first webpage component and the first plugin, interaction among the first webpage component, the first plugin, and the target application is achieved. A complete embodiment is given below, as follows: Figure 11 As shown, Figure 11 A flowchart illustrating a multimedia data processing method provided in this application embodiment includes:

[0215] S301. Determine the multimedia data to be processed in the target application;

[0216] S302. When the first plugin is called by the target application, the first web page component receives the selection operation of the target function control for the first web page component and sends the data acquisition request corresponding to the target function control to the first agent module.

[0217] S303. The first proxy module determines the data request method of the data acquisition request, encapsulates the data acquisition request according to the corresponding agreed protocol based on the data request method, and transmits the encapsulated data acquisition request to the first communicator.

[0218] S304. The first communicator compresses the encapsulated data acquisition request through the first transmitter, encapsulates the compressed data acquisition request into a preset message format, and calls the message passing mechanism to send the message format data acquisition request to the second communicator.

[0219] S305. After receiving the data acquisition request in the message format, the second receiver of the second communicator parses and decompresses it to obtain the encapsulated data acquisition request.

[0220] S306, The second communicator sends the encapsulated data acquisition request to the second agent module;

[0221] S307. The second agent module decapsulates the encapsulated data acquisition request according to the agreed protocol to obtain the data acquisition request. Based on the data acquisition request, it calls the system interface or accesses the system property to obtain the target-related data.

[0222] S308, The second agent module transmits target-related data to the second communicator;

[0223] S309. After the second communicator obtains the target-related data, it compresses the target-related data through the second transmitter and sends the compressed target-related data to the first communicator by calling the message passing mechanism.

[0224] S310, the first receiving module of the first communicator receives the compressed target-related data, decompresses it to obtain the target-related data, and transmits it to the first web page component through the first proxy module;

[0225] S311. The first web page component calls the HTTP interface to send the target-related data to the function server corresponding to the target function control, and notifies the second web page component to obtain the processed multimedia data.

[0226] S312. The second web page component obtains the processed multimedia data generated by the functional server by calling the HTTP interface and displays it in the display panel.

[0227] like Figure 12 As shown, Figure 12This application provides a system architecture diagram. The system architecture includes multiple functional servers and terminals. Each functional server communicates with web page component 1 and web page component 2 in the terminal via a network. These functional servers can provide different functional services, such as image generation services and data processing services. The terminal has an application client installed. This application can be a multimedia data processing program, such as an image processing program. The application client has a first plugin installed, which can be a plugin that implements text-to-image generation based on web page components. The first plugin includes an operation panel and a display panel. Web page components 1 and 2 in the terminal are web page components embedded in the operation panel and display panel of the first plugin. The web page components can be WebViews; web page component 1 is referred to as WebView1, and web page component 2 as WebView2. By embedding the web page components into the operation panel and display panel, the operation panel and display panel can display web page content.

[0228] WebView1 displays icons or controls with complex styles. Users can enter a text description of the image they want to generate in WebView1, and also select the desired image style. Once ready, the user can click "Generate Control" on WebView1. Based on the user's input and selection, Web Component 1 determines which data from the image to be processed needs to be obtained and generates data retrieval requests. Multiple data retrieval requests can be generated, each used to retrieve relevant data from the corresponding image by calling an interface or accessing properties. Data retrieval requests are sent asynchronously, meaning the request and response process does not block the main thread.

[0229] Specifically, WebView1 can first determine whether the image data to be processed is a complete image displayed in the application's interface layer. If it is a complete image, WebView1 can call app.getPixels() to obtain the layer's pixel data. If the image to be processed is an area selected by the selection tool, it needs to obtain the selected area range first. WebView1 can access the boundary property app.selection.bounds.__value, and then call the image retrieval method imaging.getSelection(targetSectionParams), where targetSectionParams is the specific parameter when retrieving the image.

[0230] {

[0231] documentID: document ID

[0232] sourceBounds: Select the area (left, right, top, bottom).

[0233] targetSize: The target size is used to set the size after extracting the image from the selected area.

[0234] }

[0235] When WebView1 further obtains information about the current document to initiate model generation, the document ID can be obtained by accessing system module properties, specifically app.activeDocument.id._value;

[0236] WebView1 interacts with the plugin through a connection bridge. After obtaining relevant data, WebView1 calls the server's HTTP interface to generate an image, notifies WebView2 to display the task result loading page, and polls for the generated results. Specifically, WebView1, through the connection bridge with the first plugin and the connection bridge between the first plugin and WebView, calls the WebView2 method (function) AddTaskPage(taskInfo), using the ability of inter-WebView method calls, to obtain the image generated by the function server based on WebView2, and displays the generated image in the corresponding display panel of WebView2.

[0237] The UXP plugin development framework provided in this application allows for plugin development based on web page components. This eliminates the reliance on the UXP development tool's debugging panel, allowing for the use of the development tools embedded in WebView, which offer richer functionality and support for hot module replacement of HMRs, greatly improving debugging efficiency.

[0238] This application provides a multimedia data processing device, such as... Figure 13 As shown, the multimedia data processing device 40 may include: a determining module 401, an acquiring module 402, and a display module 403, wherein,

[0239] The determining module 401 is used to determine the multimedia data to be processed in the target application, the target application having a first plug-in with an operation panel, the operation panel of the first plug-in displaying at least one functional control based on a first web page component;

[0240] The acquisition module 402 is configured to, when the first plugin is invoked by the target application, respond to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, send the target-related data corresponding to the target function control to the first web page component through the first plugin, so that the first web page component can perform processing on the multimedia data based on the related data, corresponding to the target function control, to obtain processed multimedia data; the target-related data is the related data of the multimedia data to be processed;

[0241] Display module 403 is used to display the processed multimedia data.

[0242] As an optional embodiment of this application, the operation panel of the first plugin is implemented based on the first webpage component; when the acquisition module receives a selection operation of a target function control for the first webpage component through the operation panel of the first plugin, and sends the target-related data corresponding to the target function control to the first webpage component through the first plugin, it is specifically used for:

[0243] In response to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, the following operation is triggered:

[0244] Based on the first web page component, the data retrieval request corresponding to the target function control is sent to the first plugin;

[0245] When the first plugin receives the data acquisition request, it accesses the target application programming interface of the target application to obtain the target-related data corresponding to the data acquisition request, and provides the target-related data to the first web page component.

[0246] As an optional embodiment of this application, when the first web page component sends the data retrieval request corresponding to the target function control to the first plugin, it is specifically used for:

[0247] Based on the first web page component, the data acquisition request is transmitted to the first communicator of the web page component. The first communicator then sends the data acquisition request to the second communicator of the first plugin based on the message passing mechanism. The second communicator then transmits the data acquisition request to the first plugin.

[0248] When the acquisition module provides the target-related data to the first webpage component, it is specifically used for:

[0249] The first plugin transmits the target-related data to the second communicator, which then sends the target-related data back to the first communicator based on a message passing mechanism. Finally, the first communicator sends the target-related data to the first webpage component.

[0250] As an optional embodiment of this application, the first webpage component registers a first proxy module of the first plugin, and the first plugin registers a second proxy module of the first plugin. When the acquisition module is used to transmit the data acquisition request to the first communicator based on the first webpage component, it is specifically used for:

[0251] When the first webpage component initiates the data retrieval request, the first proxy module intercepts the data retrieval request sent by the first webpage component and performs the following operations:

[0252] The data request method of the data acquisition request is determined, and the data acquisition request is encapsulated according to the agreed protocol corresponding to the data acquisition request based on the data request method. The encapsulated data acquisition request is then transmitted to the first communicator, wherein the agreed protocol is the agreed data format between the first proxy module and the second proxy module.

[0253] When the acquisition module sends the data acquisition request to the first plugin via the second communicator, it is specifically used for:

[0254] The second communicator forwards the encapsulated data acquisition request to the second proxy module, so that the second proxy module can perform the following operations:

[0255] The data request method is determined based on the encapsulated data retrieval request, and the target-related data corresponding to the data retrieval request is obtained based on the data request method.

[0256] As an optional embodiment of this application, the data request method includes a first request method or a second request method. The first request method is to request data by calling a system interface, and the second request method is to request data by accessing system module attributes.

[0257] When determining the data request method for the data retrieval request, the acquisition module specifically uses:

[0258] If the data acquisition request carries a first identifier, the data request method is determined to be the second request method; if the data acquisition request does not carry the first identifier, the data request method is determined to be the first request method. The first identifier is related to the attributes of the system module to be accessed.

[0259] When the acquisition module converts the data format of the data acquisition request according to the data request method, it has the following functions:

[0260] If the data request method is the first request method, the interface path of the called interface and the corresponding condition parameters in the data acquisition request are encapsulated according to the agreed protocol corresponding to the first request method to obtain the encapsulated data acquisition request;

[0261] If the data request method is determined to be the second request method, the attribute path of the accessed attribute is obtained from the data retrieval request, and the attribute path is encapsulated to obtain the encapsulated data retrieval request.

[0262] As an optional embodiment of this application, the multimedia data processing apparatus further includes:

[0263] The update module is used to, in response to receiving an update confirmation operation, obtain the update data of the configuration information through the first web page component when an update prompt pop-up is displayed on the operation panel of the first plugin, and update the configuration information of the first web page component based on the update data;

[0264] This update notification pop-up is displayed when the first webpage component determines that the corresponding configuration information has changed.

[0265] As an optional embodiment of this application, when the updated data includes data related to the display style of the functional controls of the operation panel, the update module, when used to update the configuration information of the first webpage component based on the updated data, is specifically used for:

[0266] Based on the data related to the display style of the function controls in the operation panel, the style of the relevant function controls in the operation panel is updated.

[0267] As an optional embodiment of this application, when the acquisition module provides the target-related data to the first webpage component, it is specifically used for:

[0268] The first plugin determines the data identifier of the data retrieval request; it then encapsulates the target-related data and the data identifier to obtain cached data.

[0269] The target-related data and the data identifier are provided to the first web page component. If the first web page component determines that the relevant information of the multimedia data has been garbage collected, the data identifier is sent to the first plugin so that the cached data can be deleted by the first plugin.

[0270] As an optional embodiment of this application, the processed multimedia data includes at least one of the following:

[0271] The processed multimedia data is obtained through the first webpage component;

[0272] If the first plugin also includes a display panel, the processed multimedia data is obtained through the first web page component or the second web page component, wherein the display panel is implemented based on the second web page component;

[0273] The processed multimedia data displayed includes at least one of the following:

[0274] When the processed multimedia data is obtained through the first web page component, the processed multimedia data is displayed in the operation panel through the first web page component, or the processed multimedia data is sent to the first plugin through the first web page component so that the processed multimedia data is displayed in the display interface of the target application through the first plugin.

[0275] When the processed multimedia data is obtained through the second web page component, the processed multimedia data is displayed in the display panel through the second web page component.

[0276] As an optional embodiment of this application, the acquisition module is used to send the data acquisition request to the second communicator based on the first communicator using a message passing mechanism, and then send the data acquisition request to the first plug-in based on the second communicator, specifically for:

[0277] The first communicator is based on a message passing mechanism. Through the first communication link between the first communicator and the second communicator, the data acquisition request is sent to the first plug-in via the second communicator.

[0278] When the acquisition module sends the target-related data to the first communicator based on the second communicator using a message passing mechanism, and sends the data acquisition request to the first webpage component based on the first communicator, it is specifically used for:

[0279] The second communicator, based on a message passing mechanism, sends the data acquisition request to the first web page component through the first communicator via the second communication link between the first and second communicators.

[0280] As an optional embodiment of this application, when the acquisition module sends the data acquisition request to the second communicator of the first plug-in based on the message passing mechanism of the first communicator, it is specifically used for:

[0281] When the amount of data in the data acquisition request exceeds a preset threshold, the data acquisition request is compressed based on the first communicator, and the compressed data acquisition request is sent to the second communicator based on the message passing mechanism of the first communication module. The compressed data acquisition request is decompressed through the second communicator to obtain the decompressed data acquisition request.

[0282] When the acquisition module transmits the target-related data to the second communicator based on the first plugin, and the second communicator sends the target-related data back to the first communicator based on the message passing mechanism, it is specifically used for:

[0283] When the amount of data related to the target exceeds a preset threshold, the second communicator compresses the data related to the target, and the second communicator sends the compressed data related to the target to the first communicator based on a message passing mechanism. The first communicator then decompresses the compressed data related to the target to obtain a decompressed data acquisition request.

[0284] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0285] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a multimedia data processing method. Compared with related technologies, this method allows for the following: After embedding a first webpage component in the operation panel of a plugin, web technologies (such as HTML, CSS, and JavaScript) can be used to build a rich user interface. This allows developers to create more complex and diverse functional controls, overcoming the limitations of the plugin's original interface display capabilities and enhancing the plugin's functional extensibility. By responding to the selection operation of a target functional control, the system can determine the user's desired data processing based on the target function corresponding to the target functional control. The first plugin then sends the target-related data to the first webpage component, enabling the first webpage component to obtain the target-related data and perform processing on the multimedia data corresponding to the target function. The processed multimedia data is then displayed, providing users with an intuitive view of the multimedia data corresponding to the target function, thus improving the user experience. This application opens up a broader development space for plugin functionality, meeting users' needs for high-quality and diverse plugins.

[0286] In one alternative embodiment, an electronic device is provided, the electronic device may be... Figure 2 or Figure 12 The terminal shown. (As shown in the image.) Figure 14 As shown, Figure 14The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0287] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0288] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0289] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0290] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0291] Figure 14 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0292] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it achieves the following: After embedding a first web page component in the operation panel of a plugin, web technologies (such as HTML, CSS, and JavaScript) can be used to build a rich user interface. This allows developers to create more complex and diverse functional controls, overcoming the limitations of the plugin's original interface display capabilities and enhancing the plugin's functional extensibility. By responding to the selection operation of the target functional control, it is possible to determine whether the user wants to process multimedia data based on the target function corresponding to the target functional control. The first plugin then sends the target-related data to the first web page component, enabling the first web page component to obtain the target-related data and perform processing on the multimedia data corresponding to the target function. The processed multimedia data is then displayed, providing users with an intuitive view of the multimedia data corresponding to the target function, thus improving the user experience. This application opens up a broader development space for plugin functionality, meeting users' needs for high-quality and diverse plugins.

[0293] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0294] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve the following: After embedding a first web page component in the operation panel of the plugin, web technologies (such as HTML, CSS, and JavaScript) can be used to build a rich user interface, allowing developers to create more complex and diverse functional controls. This solves the limitations of the original interface display capabilities of the plugin and enhances the functional extensibility of the plugin. By responding to the selection operation of the target functional control, it can determine whether the user wants to process multimedia data based on the target function corresponding to the target functional control. The first plugin then sends the target-related data to the first web page component, enabling the first web page component to obtain the target-related data and process the multimedia data according to the target function, obtaining the processed multimedia data. By displaying the processed multimedia data, the user is provided with an intuitive display of the multimedia data corresponding to the target function, improving the user experience. This application opens up a broader development space for plugin functionality and meets users' needs for high-quality and diverse plugins.

[0295] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0296] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0297] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A multimedia data processing method, characterized in that, include: The multimedia data to be processed in the target application is determined. The target application has a first plug-in with an operation panel. The operation panel of the first plug-in displays at least one functional control based on a first web page component. When the first plugin is invoked by the target application, in response to receiving a selection operation for the target function control of the first web page component through the operation panel of the first plugin, the target-related data corresponding to the target function control is sent to the first web page component through the first plugin, so that the first web page component performs processing on the multimedia data to be processed based on the target-related data, corresponding to the target function control, to obtain the processed multimedia data; the target-related data is the related data of the multimedia data to be processed. The processed multimedia data is displayed.

2. The method according to claim 1, characterized in that, The operation panel of the first plugin is implemented based on the first web page component; The step of responding to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, and sending target-related data corresponding to the target function control to the first web page component through the first plugin, includes: In response to receiving a selection operation for a target function control of the first web page component through the operation panel of the first plugin, the following operation is triggered: Based on the first web page component, the data acquisition request corresponding to the target function control is sent to the first plugin; When the first plugin receives the data acquisition request, the first plugin accesses the target application programming interface of the target application to obtain the target-related data corresponding to the data acquisition request, and provides the target-related data to the first web page component.

3. The method according to claim 2, characterized in that, The first web page component sends a data retrieval request corresponding to the target functional control to the first plugin, including: The data acquisition request is transmitted to the first communicator of the first webpage component based on the first webpage component. The first communicator then sends the data acquisition request to the second communicator of the first plugin based on the message passing mechanism. The data acquisition request is then transmitted to the first plugin through the second communicator. Providing the target-related data to the first webpage component includes: The first plugin transmits the target-related data to the second communicator, which then sends the target-related data to the first communicator based on a message passing mechanism. The first communicator then sends the target-related data to the first web page component.

4. The method according to claim 3, characterized in that, The first webpage component has a first proxy module registered with the first plugin, and the first plugin has a second proxy module registered with the first plugin. The first webpage component transmits the data acquisition request to the first communicator, including: When the first webpage component initiates the data retrieval request, the first proxy module intercepts the data retrieval request sent by the first webpage component and performs the following operations: The data request method of the data acquisition request is determined, the data acquisition request is encapsulated according to the agreed protocol corresponding to the data request method, and the encapsulated data acquisition request is transmitted to the first communicator, wherein the agreed protocol is an agreed data format between the first proxy module and the second proxy module; Sending the data acquisition request to the first plugin via the second communicator includes: The second communicator transmits the encapsulated data acquisition request to the second proxy module, so that the second proxy module can perform the following operations: The data request method is determined based on the encapsulated data acquisition request, and the target-related data corresponding to the data acquisition request is obtained based on the data request method.

5. The method according to claim 4, characterized in that, The data request method includes a first request method or a second request method. The first request method is to request data by calling a system interface, and the second request method is to request data by accessing system module attributes. The determination of the data request method for the data acquisition request includes: If the data acquisition request carries a first identifier, the data request method is determined to be a second request method; if the data acquisition request does not carry the first identifier, the data request method is determined to be a first request method. The first identifier is related to the attributes of the system module to be accessed. The step of encapsulating the data acquisition request according to the agreed protocol corresponding to the data request method includes: If the data request method is the first request method, the interface path and corresponding condition parameters of the called interface in the data acquisition request are encapsulated according to the agreed protocol corresponding to the first request method to obtain the encapsulated data acquisition request; If the data request method is determined to be the second request method, the attribute path of the accessed attribute is obtained from the data acquisition request, and the attribute path is encapsulated to obtain the encapsulated data acquisition request.

6. The method according to claim 1, characterized in that, The method further includes: When an update prompt pop-up is displayed on the operation panel of the first plugin, in response to receiving an update confirmation operation, the update data of the configuration information is obtained through the first web page component, and the configuration information of the first web page component is updated based on the update data; The update notification pop-up is displayed when the first webpage component determines that the corresponding configuration information has changed.

7. The method according to claim 6, characterized in that, When the updated data includes data related to the display style of the functional controls on the operation panel, updating the configuration information of the first webpage component based on the updated data includes: Based on the data related to the display style of the function controls in the operation panel, the style of the relevant function controls in the operation panel is updated.

8. The method according to claim 2, characterized in that, Providing the target-related data to the first webpage component includes: The first plugin determines the data identifier of the data acquisition request; it encapsulates the target-related data and the data identifier to obtain cached data; The target-related data and the data identifier are provided to the first web page component. If the first web page component determines that the target-related data has been garbage collected, the data identifier is sent to the first plugin so that the cached data can be deleted through the first plugin.

9. The method according to claim 1, characterized in that, The processed multimedia data includes at least one of the following: The processed multimedia data is obtained through the first web page component; If the first plugin also includes a display panel, the processed multimedia data is obtained through the first web page component or through the second web page component, wherein the display panel is implemented based on the second web page component; The display of the processed multimedia data includes at least one of the following: When the processed multimedia data is obtained through the first web page component, the processed multimedia data is displayed in the operation panel through the first web page component, or the processed multimedia data is sent to the first plugin through the first web page component, so that the processed multimedia data is displayed in the display interface of the target application through the first plugin. When the processed multimedia data is obtained through the second web page component, the processed multimedia data is displayed in the display panel through the second web page component.

10. The method according to claim 3, characterized in that, The process of sending the data acquisition request to the second communicator based on the first communicator using a message passing mechanism, and then sending the data acquisition request to the first plugin based on the second communicator, includes: The first communicator, based on a message passing mechanism, sends the data acquisition request to the first plug-in via the second communicator through the first communication link between the first and second communicators; The step of sending the target-related data to the first communicator based on the second communicator using a message passing mechanism, and sending the data acquisition request to the first webpage component based on the first communicator, includes: The second communicator, based on a message passing mechanism, sends the data acquisition request to the first web page component through the first communicator via the second communication link between the first and second communicators.

11. The method according to claim 3 or 10, characterized in that, The first communicator sends the data acquisition request to the second communicator of the first plugin based on a message passing mechanism, including: When the data volume of the data acquisition request exceeds a preset threshold, the data acquisition request is compressed based on the first communicator, and the compressed data acquisition request is sent to the second communicator based on the message passing mechanism of the first communication module. The compressed data acquisition request is then decompressed by the second communicator to obtain the decompressed data acquisition request. The first plugin transmits the target-related data to the second communicator, which then sends the target-related data back to the first communicator based on a message passing mechanism, including: When the amount of target-related data exceeds a preset threshold, the target-related data is compressed using the second communicator. The compressed target-related data is then sent to the first communicator via the second communicator using a message passing mechanism. The first communicator decompresses the compressed target-related data to obtain a decompressed data acquisition request.

12. A multimedia data processing device, characterized in that, include: A determination module is used to determine the multimedia data to be processed in a target application, wherein the target application has a first plug-in with an operation panel, and the operation panel of the first plug-in displays at least one functional control based on a first web page component. The acquisition module is configured to, when the first plugin is invoked by the target application, respond to receiving a selection operation for a target function control of the first webpage component through the operation panel of the first plugin, send target-related data corresponding to the target function control to the first webpage component through the first plugin, so that the first webpage component can perform processing on the multimedia data based on the target-related data, corresponding to the target function control, to obtain processed multimedia data; the target-related data is the related data of the multimedia data to be processed. The display module is used to display the processed multimedia data.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-11.

Images