Method and device for processing audio and video functions, storage medium and processor

Abstract

The present application discloses a method and device for processing audio and video functions, a storage medium and a processor. The method includes: atomizing each audio and video function into a corresponding target meta-capability node, and setting a UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and setting a native bottom-layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; connecting the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain an audio and video pipeline topology diagram of the UI architecture layer; connecting the native bottom-layer audio and video meta-capability nodes according to the audio and video pipeline topology diagram of the UI architecture layer to obtain a native bottom-layer audio and video pipeline topology diagram; and realizing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram. The present application solves the technical problem in the related art that the UI architecture tends to build the upper-layer functions of the application, resulting in relatively low efficiency in developing audio and video functions at the bottom layer.

Description

Technical Field

[0001] The present application relates to the field of computer science and technology, and in particular to a method and device for processing audio and video functions, a storage medium and a processor. Background Art

[0002] The UI architecture is used to build user interfaces, such as Flutter. Flutter is the latest cross-platform development technology that can span multiple systems such as Android, iOS, MacOS, Windows, and Linux, and can also be packaged into a web program to run on a browser. The UI architecture adopts a more thorough cross-platform solution, that is, it implements a set of UI frameworks and then renders the UI pages directly on the GPU. As a cross-end application development framework, the UI architecture has great advantages in unifying the functional logic and interface performance of each end and improving development efficiency. However, the UI architecture itself is more inclined to build upper-level functions of applications, and is not so efficient for developing audio and video at a lower level.

[0003] Currently, no effective solution has been proposed to address the problem that related technologies tend to focus on building upper-level application functions in UI architecture, resulting in low efficiency in developing lower-level audio and video functions. Summary of the invention

[0004] The embodiments of the present application provide a method and device for processing audio and video functions, a storage medium and a processor, so as to at least solve the technical problem that the related technology tends to build upper-level functions of the application in the UI architecture, resulting in relatively low efficiency in developing lower-level audio and video functions.

[0005] According to one aspect of an embodiment of the present application, a method for processing audio and video functions is provided, including: atomizing each audio and video function into a corresponding target meta-capability node, and setting a UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and a native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; connecting the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain an audio and video pipeline topology map of the UI architecture layer, wherein the audio and video pipeline topology map of the UI architecture layer is used to transmit data with the native bottom layer; connecting the native bottom layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology map to obtain a native bottom layer audio and video pipeline topology map; and implementing the target audio and video function according to the native bottom layer audio and video pipeline topology map.

[0006] Furthermore, the UI architecture audio and video meta capability nodes are connected according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram, including: determining the UI architecture audio and video meta capability nodes to be connected according to the target audio and video functions; determining the node type of the UI architecture audio and video meta capability nodes to be connected and the type of data transmitted between the UI architecture audio and video meta capability nodes to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the type of data is one of the following: video stream data type and audio stream data type; connecting the UI architecture audio and video meta capability nodes to be connected according to the node type and the type of the data to obtain the UI architecture layer audio and video pipeline topology diagram.

[0007] Furthermore, based on the node type and the type of the data, the UI architecture audio and video meta-capability node to be connected is connected to obtain the UI architecture layer audio and video pipeline topology diagram, including: judging whether the node type meets the first preset requirement; if the node type meets the first preset requirement, then based on the type of the data, the UI architecture audio and video meta-capability node to be connected is connected to obtain the UI architecture layer audio and video pipeline topology diagram.

[0008] Furthermore, after connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, the method also includes: verifying the UI architecture layer audio and video pipeline topology map according to a second preset requirement to obtain a verification result; if the verification result is passed, transmitting the UI architecture layer audio and video pipeline topology map to the native bottom layer.

[0009] Furthermore, before connecting the native underlying audio and video meta-capability node according to the UI architecture layer audio and video pipeline topology map to obtain the native underlying audio and video pipeline topology map, the method also includes: constructing a mapping relationship between the UI architecture audio and video meta-capability node and the native underlying audio and video meta-capability node; based on the mapping relationship, calling the native underlying audio and video meta-capability node through the UI architecture audio and video meta-capability node to implement the target audio and video function.

[0010] Furthermore, according to the UI architecture layer audio and video pipeline topology map, the native underlying audio and video meta capability node is connected to obtain the native underlying audio and video pipeline topology map, including: determining the node type of the native underlying audio and video meta capability node according to the port of the native underlying audio and video meta capability node; judging whether the native underlying audio and video meta capability node meets the third preset requirement according to the UI architecture layer audio and video pipeline topology map and the node type; if the native underlying audio and video meta capability node meets the third preset requirement, connecting the native underlying audio and video meta capability node through the port to obtain the native underlying audio and video pipeline topology map.

[0011] Furthermore, the method also includes: if the native underlying audio and video meta-capability node does not meet the third preset requirement, sending an error signal to the UI architecture.

[0012] Furthermore, implementing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram includes: setting the application state of the native bottom layer through the UI architecture, wherein the application state is one of the following: startup state, running state, stop state and destruction state; when the application state of the native bottom layer is the startup state, implementing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram.

[0013] According to another aspect of an embodiment of the present application, a processing device for audio and video functions is also provided, including: a setting unit, used to atomize each audio and video function into a corresponding target meta-capability node, and set the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and set the native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; a first connection unit, used to connect the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain a UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer; a second connection unit, used to connect the native bottom layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology map to obtain a native bottom layer audio and video pipeline topology map; an implementation unit, used to implement the target audio and video function according to the native bottom layer audio and video pipeline topology map.

[0014] Furthermore, the first connection unit includes: a first determination subunit, used to determine the UI architecture audio and video meta capability node to be connected based on the target audio and video function; a second determination subunit, used to determine the node type of the UI architecture audio and video meta capability node to be connected and the type of data transmitted between the UI architecture audio and video meta capability node to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the type of data is one of the following: video stream data type and audio stream data type; the first connection subunit is used to connect the UI architecture audio and video meta capability node to be connected based on the node type and the type of data to obtain the UI architecture layer audio and video pipeline topology diagram.

[0015] Furthermore, the connection subunit includes: a judgment module, used to judge whether the node type meets the first preset requirement; a connection module, used to connect the UI architecture audio and video element capability node to be connected according to the type of the data if the node type meets the first preset requirement, so as to obtain the UI architecture layer audio and video pipeline topology diagram.

[0016] Furthermore, the device also includes: a verification unit, which is used to connect the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, and then verify the UI architecture layer audio and video pipeline topology map according to a second preset requirement to obtain a verification result; a transmission unit, which is used to transmit the UI architecture layer audio and video pipeline topology map to the native bottom layer if the verification result is passed.

[0017] Furthermore, the device also includes: a construction unit, which is used to construct a mapping relationship between the UI architecture audio and video meta capability node and the native underlying audio and video meta capability node before connecting the native underlying audio and video meta capability node according to the UI architecture layer audio and video pipeline topology map to obtain the native underlying audio and video pipeline topology map; and a calling unit, which is used to call the native underlying audio and video meta capability node through the UI architecture audio and video meta capability node according to the mapping relationship to realize the target audio and video function.

[0018] Furthermore, the second connection unit includes: a third determination subunit, used to determine the node type of the native underlying audio and video meta capability node based on the port of the native underlying audio and video meta capability node; a judgment subunit, used to judge whether the native underlying audio and video meta capability node meets the third preset requirement based on the UI architecture layer audio and video pipeline topology diagram and the node type; a second connection subunit, used to connect the native underlying audio and video meta capability node through the port to obtain the native underlying audio and video pipeline topology diagram if the native underlying audio and video meta capability node meets the third preset requirement.

[0019] Furthermore, the device also includes: a sending unit, configured to send an error signal to the UI architecture if the native underlying audio and video meta-capability node does not meet the third preset requirement.

[0020] Furthermore, the implementation unit includes: a setting subunit, used to set the application state of the native bottom layer through the UI architecture, wherein the application state is one of the following: startup state, running state, stop state and destruction state; an implementation subunit, used to implement the target audio and video function according to the native bottom layer audio and video pipeline topology diagram when the application state of the native bottom layer is the startup state.

[0021] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned methods for processing audio and video capabilities.

[0022] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the program executes any one of the above-mentioned methods for processing audio and video capabilities when running.

[0023] In an embodiment of the present application, each audio and video function is atomized into a corresponding target meta-capability node, and a UI architecture audio and video meta-capability node corresponding to the target meta-capability node is set on the UI architecture, and a native bottom layer audio and video meta-capability node corresponding to the target meta-capability node is set on the native bottom layer; the UI architecture audio and video meta-capability nodes are connected according to the target audio and video function to obtain a UI architecture layer audio and video pipeline topology diagram, wherein the UI architecture layer audio and video pipeline topology diagram is used to transmit data between the native bottom layer; according to the UI architecture layer audio and video pipeline topology diagram, the native bottom layer audio and video meta-capability nodes are connected to obtain a native bottom layer audio and video pipeline topology diagram; the target audio and video function is realized according to the native bottom layer audio and video pipeline topology diagram, which solves the technical problem in the related technology that the UI architecture tends to build upper-layer functions of the application, resulting in relatively low efficiency in developing audio and video functions at the bottom layer. By atomizing each audio and video function into a target meta-capability node, setting the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and setting the native bottom-layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer, and then constructing the UI architecture layer audio and video pipeline topology diagram and the native bottom-layer audio and video pipeline topology diagram according to the actual audio and video function requirements, it is possible to develop audio and video functions that are more bottom-level in the UI architecture, thereby improving the efficiency of audio and video development. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 is a hardware structure block diagram of a computer terminal provided according to Embodiment 1 of the present invention;

[0026] Figure 2 is a flowchart of a method for processing audio and video functions provided in Embodiment 1 of the present invention;

[0027] Figure 3 is a schematic diagram of an audio and video pipeline topology diagram provided according to Embodiment 1 of the present invention;

[0028] Figure 4 is a schematic diagram of a device for processing audio and video functions according to a second embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of a system for processing audio and video functions according to a second embodiment of the present invention;

[0030] Figure 6 It is a schematic diagram of a computer terminal provided according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Example 1

[0034] According to an embodiment of the present application, a method for processing audio and video functions is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a processing method for audio and video functions is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, ..., 102n in the figure) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0036] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the processing method of the audio and video functions in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the processing method of the audio and video functions described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0039] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0040] Under the above operating environment, this application provides Figure 2 The processing method of the audio and video functions shown. Figure 2 It is a flowchart of a method for processing audio and video functions according to the first embodiment of the present application.

[0041] Step S201, atomize each audio and video function into a corresponding target meta-capability node, and set the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and set the native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer.

[0042] Specifically, Flutter or other architectures with similar functions can be used as UI architecture. First, each audio and video capability is atomized into a corresponding target meta-capability node. The atomization here means setting the audio and video capability to the smallest independent function without the help of other functions. The target meta-capability node can be represented by Node. For example, the smallest independent capability corresponding to the camera is the ability to continuously obtain a frame of the picture. Then set the UI architecture audio and video meta-capability node (UI architecture Node) corresponding to the target meta-capability node in the UI architecture, and set the native bottom layer audio and video meta-capability node (Native Node) corresponding to the target meta-capability node on the native bottom layer. There is a one-to-one correspondence between the UI architecture Node and the Native Node. The UI architecture Node is used as an operation object to control the Native Node with actual implementation capabilities.

[0043] Step S202, connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer.

[0044] Specifically, developers can implement different audio and video functions according to different application scenarios. Connect the UI architecture audio and video meta-capability nodes according to the actual audio and video functions required (i.e. the target audio and video functions mentioned above) to obtain the UI architecture layer audio and video pipeline topology diagram. For example, to implement the photo function, you can connect the UI architecture Node corresponding to Camera with the UI architecture Node corresponding to PhotoTaker to obtain the UI architecture layer audio and video pipeline topology diagram. The UI architecture layer audio and video pipeline topology diagram is used for overall state control, data transmission between nodes, and data transmission with native underlying layers.

[0045] Step S203, according to the UI architecture layer audio and video pipeline topology diagram, connect the native bottom layer audio and video meta-capability nodes to obtain the native bottom layer audio and video pipeline topology diagram.

[0046] Specifically, the native underlying audio and video meta-capability nodes are nodes with actual implementation capabilities. Therefore, according to the UI architecture layer audio and video pipeline topology diagram obtained above, the native underlying audio and video meta-capability nodes are connected to obtain the native underlying audio and video pipeline topology diagram, which can realize the audio and video functions logically controlled by the UI architecture layer.

[0047] Step S204, realizing the target audio and video functions according to the native bottom-layer audio and video pipeline topology diagram.

[0048] In summary, by atomizing each audio and video function into a target meta-capability node, setting the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and setting the native bottom-layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer, and then constructing the UI architecture layer audio and video pipeline topology diagram and the native bottom-layer audio and video pipeline topology diagram according to the actual audio and video function requirements, it is possible to develop more bottom-level audio and video functions in the UI architecture, and the atomized audio and video functions support reuse in different scenarios, which changes the previous way of audio and video development, thereby improving the efficiency and experience of audio and video development in the UI architecture.

[0049] Optionally, in the audio and video capability processing method provided in Embodiment 1 of the present invention, the UI architecture audio and video meta capability nodes are connected according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram, including: determining the UI architecture audio and video meta capability node to be connected according to the target audio and video function; determining the node type of the UI architecture audio and video meta capability node to be connected and the type of data transmitted between the UI architecture audio and video meta capability nodes to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the data type is one of the following: video stream data type and audio stream data type; according to the node type and the data type, the UI architecture audio and video meta capability nodes to be connected are connected to obtain the UI architecture layer audio and video pipeline topology diagram.

[0050] Specifically, the type of node is set, and the node types are divided into production type, consumption type and production consumption type. For example, the camera belongs to the production type, that is, a frame of the picture is continuously obtained through the camera. Taking pictures belongs to consumption, that is, a frame of the picture is saved by taking pictures. At the same time, the data transmission between nodes is also divided into the following two video stream data types and audio stream data types, and the transmitted data is audio stream or video stream. The types of data supported by different nodes are different. For example, the camera supports video stream transmission, while the microphone supports audio stream transmission. According to the audio and video functions to be implemented, determine the UI architecture audio and video meta-capability nodes that need to be connected, and then determine the node type and data type. According to the above node type and data type, the UI architecture audio and video meta-capability nodes that need to be connected are connected to obtain the UI architecture layer audio and video pipeline topology diagram. Through the above steps, it is ensured that the obtained UI architecture layer audio and video pipeline topology diagram can realize the corresponding audio and video functions, and the advantages of the UI architecture can be better utilized.

[0051] Optionally, in the audio and video capability processing method provided in Embodiment 1 of the present invention, the UI architecture audio and video meta-capability nodes to be connected are connected according to the node type and the data type to obtain the UI architecture layer audio and video pipeline topology map, including: judging whether the node type meets the first preset requirement; if the node type meets the first preset requirement, then according to the data type, the UI architecture audio and video meta-capability nodes to be connected are connected to obtain the UI architecture layer audio and video pipeline topology map.

[0052] Specifically, for the connection between nodes, only when nodes with production capabilities are connected to nodes with consumption capabilities can data flow transmission be realized (i.e., the first preset requirement mentioned above), so it is necessary to determine whether the node type meets the requirements. After the node type meets the first preset requirement, the corresponding connection is made according to the data type. The above steps further ensure the accuracy of the audio and video pipeline topology diagram of the UI architecture layer.

[0053] Optionally, in the audio and video capability processing method provided in Embodiment 1 of the present invention, after connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, the method also includes: verifying the UI architecture layer audio and video pipeline topology map according to the second preset requirement to obtain a verification result; if the verification result is passed, transmitting the UI architecture layer audio and video pipeline topology map to the native bottom layer.

[0054] Specifically, after obtaining the audio and video pipeline topology diagram of the UI architecture layer, the audio and video pipeline topology diagram of the UI architecture layer is checked, for example, whether the node type and data type meet the requirements, and whether there is no loop phenomenon in the audio and video pipeline topology diagram of the UI architecture layer. After the verification is passed, the audio and video pipeline topology diagram of the UI architecture layer is transmitted to the native bottom layer. The above steps ensure the accuracy of the connection of the audio and video pipeline topology diagram of the UI architecture layer, and avoid the problem that the corresponding audio and video functions cannot be successfully implemented due to the connection problem of the audio and video pipeline topology diagram of the UI architecture layer.

[0055] Optionally, in the audio and video capability processing method provided in Embodiment 1 of the present invention, before connecting the native underlying audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology diagram to obtain the native underlying audio and video pipeline topology diagram, the method also includes: constructing a mapping relationship between the UI architecture audio and video meta-capability nodes and the native underlying audio and video meta-capability nodes; based on the mapping relationship, calling the native underlying audio and video meta-capability nodes through the UI architecture audio and video meta-capability nodes to realize the target audio and video functions.

[0056] Specifically, since the UI architecture audio and video meta-capability node is used as the operation object to control the Native point with actual implementation capability, it is necessary to establish a mapping relationship between the UI architecture audio and video meta-capability node and the Native node (i.e., the above-mentioned native underlying audio and video meta-capability node) to ensure a one-to-one correspondence between the UI architecture audio and video meta-capability node and the Native node, and then implement the audio and video function by calling the Native node through the UI architecture audio and video meta-capability node. Through the above steps, the corresponding audio and video functions can be realized by controlling the UI architecture audio and video meta-capability node on the UI architecture side, thereby improving the efficiency of audio and video function development.

[0057] Optionally, in the audio and video capability processing method provided in Example 1 of the present invention, the native underlying audio and video meta capability nodes are connected according to the UI architecture layer audio and video pipeline topology diagram to obtain the native underlying audio and video pipeline topology diagram, including: determining the node type of the native underlying audio and video meta capability node according to the port of the native underlying audio and video meta capability node; judging whether the native underlying audio and video meta capability node meets the third preset requirement according to the UI architecture layer audio and video pipeline topology diagram and the node type; if the native underlying audio and video meta capability node meets the third preset requirement, connecting the native underlying audio and video meta capability node through the port to obtain the native underlying audio and video pipeline topology diagram.

[0058] Specifically, after obtaining the audio and video pipeline topology diagram of the UI architecture layer, the audio and video pipeline topology diagram of the UI architecture layer will be further checked at the native bottom layer to ensure accuracy. Similarly, when connecting the native bottom layer audio and video meta-capability nodes, it is still necessary to confirm the node type of the native bottom layer audio and video meta-capability nodes and the type of data transmitted (i.e., the third preset requirement mentioned above). The native bottom layer audio and video meta-capability node has an external port, through which the node type of the native bottom layer audio and video meta-capability node can be determined. For example, Source Port represents the production type, and Sink Port represents the consumption type. When the third preset requirement mentioned above is met, the corresponding native bottom layer audio and video meta-capability nodes are connected to obtain the native bottom layer audio and video meta-capability nodes. Through the above steps, the accuracy of the native bottom layer audio and video pipeline topology diagram is guaranteed, ensuring that the corresponding audio and video functions can be achieved by controlling the UI architecture audio and video meta-capability nodes on the UI architecture side.

[0059] Optionally, in the method for processing audio and video capabilities provided in the first embodiment of the present invention, the method further includes: if the native underlying audio and video meta-capability node does not meet the third preset requirement, sending an error signal to the UI architecture.

[0060] Specifically, when the third preset requirement is not met, the native bottom layer will send an error signal to the UI architecture to remind the developer to handle the problem as soon as possible.

[0061] Optionally, in the audio and video capability processing method provided in Embodiment 1 of the present invention, implementing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram includes: setting the native bottom-layer application state through the UI architecture, wherein the application state is one of the following: startup state, running state, stop state and destruction state; when the native bottom-layer application state is the startup state, implementing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram.

[0062] Specifically, after obtaining the corresponding native bottom-level audio and video pipeline topology diagram, the native bottom-level application state can be set on the UI architecture side. When in the startup state, the audio and video meta-capability nodes of the UI architecture can be operated to control the Native nodes with actual implementation capabilities to implement audio and video functions. Combining the streaming audio and video development concept with the cross-end development technology of the UI architecture improves the audio and video development experience on the UI architecture; giving the UI architecture more audio and video logic control capabilities can better play the cross-end advantages of the UI architecture; the audio and video capabilities are atomic, supporting multiple scenarios, and greatly improving the audio and video development efficiency of the UI architecture.

[0063] In summary, in the audio and video capability processing method provided in the first embodiment of the present invention, each audio and video function is atomized into a corresponding target meta-capability node, and a UI architecture audio and video meta-capability node corresponding to the target meta-capability node is set on the UI architecture and a native bottom layer audio and video meta-capability node corresponding to the target meta-capability node is set on the native bottom layer; the UI architecture audio and video meta-capability nodes are connected according to the target audio and video function to obtain a UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer; according to the UI architecture layer audio and video pipeline topology map, the native bottom layer audio and video meta-capability nodes are connected to obtain a native bottom layer audio and video pipeline topology map; the target audio and video function is realized according to the native bottom layer audio and video pipeline topology map, which solves the technical problem that the related technology tends to build upper-layer functions of the application on the UI architecture, resulting in relatively low efficiency in developing audio and video functions at the bottom layer. By atomizing each audio and video function into a target meta-capability node, setting the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and setting the native bottom-layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer, and then constructing the UI architecture layer audio and video pipeline topology diagram and the native bottom-layer audio and video pipeline topology diagram according to the actual audio and video function requirements, it is possible to develop audio and video functions that are more bottom-level in the UI architecture, thereby improving the efficiency of audio and video development.

[0064] like Figure 3As shown, it is a node topology diagram provided according to an implementation of the present invention. The audio and video capabilities of Camera, Microphone, Renderer, Previewer, PhotoTaker, and VideoRecorder are atomized into Nodes, and then these Nodes are linked in a directed graph order in the UI architecture and Natve to obtain a topology diagram, so that the shooting function with the capabilities of taking pictures, recording videos, and rendering filter images can be realized.

[0065] Through the embodiments of the present application, some specific audio and video capabilities are atomized into target meta-capability nodes, the corresponding node of the node is set in the native bottom layer (i.e., the above-mentioned native bottom layer audio and video meta-capability node), and the function mapping node corresponding to the native bottom layer node is set in the UI architecture layer (i.e., the above-mentioned UI architecture audio and video meta-capability node); according to the required audio and video functions, the UI architecture audio and video meta-capability nodes are connected to obtain the UI architecture layer audio and video pipeline topology diagram; at the same time, the UI architecture layer pipeline topology diagram will communicate with the native bottom layer to transmit data; according to the UI architecture layer audio and video pipeline topology diagram, a native bottom layer audio and video pipeline topology diagram is constructed; the target audio and video function is realized through the native bottom layer pipeline topology diagram, and the native bottom layer pipeline topology diagram is operated and controlled by the UI architecture audio and video meta-capability node and the UI architecture layer audio and video pipeline topology diagram. Through this application, it is possible to realize audio and video functions that are more bottom-level in the UI architecture development. By operating the UI architecture audio and video meta-capability node, it is possible to control the native bottom layer audio and video meta-capability node with actual implementation capability to realize the corresponding audio and video function.

[0066] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0067] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0068] Example 2

[0069] According to an embodiment of the present application, a device for processing audio and video functions for implementing the above-mentioned processing of audio and video functions is also provided, such as Figure 4 As shown, the device comprises:

[0070] The setting unit 401 is used to atomize each audio and video function into a corresponding target meta-capability node, and set the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and set the native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer.

[0071] The first connection unit 402 is used to connect the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer.

[0072] The second connection unit 403 is used to connect the native bottom-layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology diagram to obtain the native bottom-layer audio and video pipeline topology diagram.

[0073] The implementation unit 404 is used to implement the target audio and video functions according to the native bottom-layer audio and video pipeline topology diagram.

[0074] The audio and video capability processing device provided in the second embodiment of the present invention atomizes each audio and video function into a corresponding target meta-capability node through a setting unit 401, and sets a UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and sets a native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; the first connection unit 402 connects the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain an audio and video pipeline topology diagram of the UI architecture layer, wherein the audio and video pipeline topology diagram of the UI architecture layer is used to transmit data between the native bottom layer; the second connection unit 403 connects the native bottom layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology diagram to obtain a native bottom layer audio and video pipeline topology diagram; the implementation unit 404 implements the target audio and video function according to the native bottom layer audio and video pipeline topology diagram, thereby solving the technical problem in the related technology that the UI architecture tends to build upper-layer functions of the application, resulting in relatively low efficiency in developing audio and video functions at the bottom layer. By atomizing each audio and video function into a target meta-capability node, setting the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and setting the native bottom-layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer, and then constructing the UI architecture layer audio and video pipeline topology diagram and the native bottom-layer audio and video pipeline topology diagram according to the actual audio and video function requirements, it is possible to develop audio and video functions that are more bottom-level in the UI architecture, thereby improving the efficiency of audio and video development.

[0075] Optionally, in the audio and video capability processing device provided in Embodiment 2 of the present invention, the first connection unit 402 includes: a first determination subunit, used to determine the UI architecture audio and video meta capability node to be connected based on the target audio and video function; a second determination subunit, used to determine the node type of the UI architecture audio and video meta capability node to be connected and the type of data transmitted between the UI architecture audio and video meta capability nodes to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the data type is one of the following: video stream data type and audio stream data type; a connection subunit, used to connect the UI architecture audio and video meta capability node to be connected based on the node type and the data type, and obtain the UI architecture layer audio and video pipeline topology diagram.

[0076] Optionally, in the audio and video capability processing device provided in Embodiment 2 of the present invention, the connection subunit includes: a judgment module, used to judge whether the node type meets the first preset requirement; a connection module, used to connect the UI architecture audio and video meta-capability node to be connected based on the type of data if the node type meets the first preset requirement, so as to obtain the UI architecture layer audio and video pipeline topology diagram.

[0077] Optionally, in the audio and video capability processing device provided in Embodiment 2 of the present invention, the device also includes: a verification unit, which is used to connect the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, and then verify the UI architecture layer audio and video pipeline topology map according to a second preset requirement to obtain a verification result; and a transmission unit, which is used to transmit the UI architecture layer audio and video pipeline topology map to the native bottom layer if the verification result is passed.

[0078] Optionally, in the audio and video capability processing device provided in Embodiment 2 of the present invention, the device also includes: a construction unit, used to connect the native underlying audio and video meta capability nodes according to the UI architecture layer audio and video pipeline topology map to obtain the native underlying audio and video pipeline topology map, and construct a mapping relationship between the UI architecture audio and video meta capability nodes and the native underlying audio and video meta capability nodes; and a calling unit, used to call the native underlying audio and video meta capability nodes through the UI architecture audio and video meta capability nodes according to the mapping relationship to realize the target audio and video functions.

[0079] Optionally, in the audio and video capability processing device provided in Embodiment 2 of the present invention, the second connection unit 303 includes: a third determination subunit, used to determine the node type of the native underlying audio and video meta capability node based on the port of the native underlying audio and video meta capability node; a judgment subunit, used to judge whether the native underlying audio and video meta capability node meets the third preset requirement based on the UI architecture layer audio and video pipeline topology diagram and node type; a second connection subunit, used to connect the native underlying audio and video meta capability node through the port to obtain the native underlying audio and video pipeline topology diagram if the native underlying audio and video meta capability node meets the third preset requirement.

[0080] Optionally, in the audio and video capability processing device provided in the second embodiment of the present invention, the device further includes: a sending unit, configured to send an error signal to the UI architecture if the native underlying audio and video meta-capability node does not meet the third preset requirement.

[0081] Optionally, in the processing device for audio and video capabilities provided in Embodiment 2 of the present invention, the implementation unit 404 includes: a setting sub-unit, used to set the application state of the native bottom layer through the UI architecture, wherein the application state is one of the following: a startup state, a running state, a stop state, and a destruction state; an implementation sub-unit, used to implement the target audio and video function according to the native bottom layer audio and video pipeline topology diagram when the application state of the native bottom layer is the startup state.

[0082] It should be noted that the above-mentioned setting unit 401, the first connection unit 402, the second connection unit 403 and the implementation unit 304 correspond to steps S201 to S204 in Example 1, and the examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules as part of the device can be run in the computer terminal 10 provided in Example 1.

[0083] like Figure 5 As shown, it is a processing system for audio and video functions provided according to the second embodiment of the invention, and the system includes a UI architecture, Native and Pipeline. Pipeline is an audio and video processing pipeline, which is used to transmit data information; Graph represents a topological graph; Message Channel is used for communication between UI architecture and Native. Sample Buffer is a data frame circulating in the pipeline; the atomic audio and video function is called Node. Node has a one-to-one correspondence with the underlying Native on the UI architecture side. The Node on the UI architecture side serves as an operation object to control the Native Node with actual implementation capabilities, and the two communicate through the Message Channel inside the Pipeline. Developers can quickly implement corresponding audio and video functions by creating, linking and controlling Node on the UI architecture side. The Node of the implemented audio and video function supports reuse in different scenarios, which changes the previous audio and video development method, and thus improves the efficiency and experience of audio and video development in the UI architecture.

[0084] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0085] Example 3

[0086] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0087] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0088] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the processing method of audio and video functions: atomize each audio and video function into a corresponding target meta-capability node, and set the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and set the native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; connect the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain the UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer; connect the native bottom layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology map to obtain the native bottom layer audio and video pipeline topology map; realize the target audio and video function according to the native bottom layer audio and video pipeline topology map.

[0089] The above-mentioned computer terminal can also execute the program code of the following steps in the processing method of audio and video functions: connecting the UI architecture audio and video meta capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram including: determining the UI architecture audio and video meta capability node to be connected according to the target audio and video function; determining the node type of the UI architecture audio and video meta capability node to be connected and the type of data transmitted between the UI architecture audio and video meta capability nodes to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the data type is one of the following: video stream data type and audio stream data type; connecting the UI architecture audio and video meta capability nodes to be connected according to the node type and the data type to obtain the UI architecture layer audio and video pipeline topology diagram.

[0090] The above-mentioned computer terminal can also execute the program code of the following steps in the processing method of audio and video functions: according to the node type and the type of data, the UI architecture audio and video meta-capability nodes to be connected are connected to obtain the UI architecture layer audio and video pipeline topology diagram, including: judging whether the node type meets the first preset requirement; if the node type meets the first preset requirement, then according to the type of data, the UI architecture audio and video meta-capability nodes to be connected are connected to obtain the UI architecture layer audio and video pipeline topology diagram.

[0091] The above-mentioned computer terminal can also execute the program code of the following steps in the audio and video function processing method: after connecting the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain the UI architecture layer audio and video pipeline topology map, the UI architecture layer audio and video pipeline topology map is verified according to the second preset requirement to obtain the verification result; if the verification result is passed, the UI architecture layer audio and video pipeline topology map is transmitted to the native bottom layer.

[0092] The above-mentioned computer terminal can also execute the program code of the following steps in the audio and video function processing method: before connecting the native underlying audio and video meta-capability nodes based on the UI architecture layer audio and video pipeline topology diagram to obtain the native underlying audio and video pipeline topology diagram, construct a mapping relationship between the UI architecture audio and video meta-capability nodes and the native underlying audio and video meta-capability nodes; based on the mapping relationship, call the native underlying audio and video meta-capability nodes through the UI architecture audio and video meta-capability nodes to realize the target audio and video functions.

[0093] The above-mentioned computer terminal can also execute the program code of the following steps in the processing method of audio and video functions: according to the UI architecture layer audio and video pipeline topology diagram, the native underlying audio and video meta capability node is connected to obtain the native underlying audio and video pipeline topology diagram, including: according to the port of the native underlying audio and video meta capability node, the node type of the native underlying audio and video meta capability node is determined; according to the UI architecture layer audio and video pipeline topology diagram and the node type, whether the native underlying audio and video meta capability node meets the third preset requirement; if the native underlying audio and video meta capability node meets the third preset requirement, the native underlying audio and video meta capability node is connected through the port to obtain the native underlying audio and video pipeline topology diagram.

[0094] The computer terminal may also execute the program code of the following steps in the method for processing audio and video functions: if the native underlying audio and video meta-capability node does not meet the third preset requirement, an error signal is sent to the UI architecture.

[0095] The above-mentioned computer terminal can also execute the program code of the following steps in the method for processing audio and video functions: realizing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram includes: setting the application state of the native bottom layer through the UI architecture, wherein the application state is one of the following: startup state, running state, stop state and destruction state; when the application state of the native bottom layer is the startup state, realizing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram.

[0096] Figure 6 is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 6 As shown, the computer terminal 10 may include: one or more ( Figure 6 Only one processor and memory are shown.

[0097] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the processing method and device of the audio and video functions in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the processing method of the audio and video functions mentioned above is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: atomize each audio and video function into a corresponding target meta-capability node, and set the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and set the native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; connect the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain the UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer; connect the native bottom layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology map to obtain the native bottom layer audio and video pipeline topology map; realize the target audio and video function according to the native bottom layer audio and video pipeline topology map.

[0099] Optionally, the processor may also execute the program code of the following steps: connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram, including: determining the UI architecture audio and video meta-capability nodes to be connected according to the target audio and video functions; determining the node type of the UI architecture audio and video meta-capability nodes to be connected and the type of data transmitted between the UI architecture audio and video meta-capability nodes to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the data type is one of the following: video stream data type and audio stream data type; connecting the UI architecture audio and video meta-capability nodes to be connected according to the node type and the data type to obtain the UI architecture layer audio and video pipeline topology diagram.

[0100] Optionally, the processor may also execute the program code of the following steps: connecting the UI architecture audio and video meta-capability nodes to be connected based on the node type and the data type, and obtaining the UI architecture layer audio and video pipeline topology diagram, including: determining whether the node type meets the first preset requirement; if the node type meets the first preset requirement, connecting the UI architecture audio and video meta-capability nodes to be connected based on the data type, and obtaining the UI architecture layer audio and video pipeline topology diagram.

[0101] Optionally, the above-mentioned processor can also execute the program code of the following steps: after connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, the method also includes: according to the second preset requirement, verifying the UI architecture layer audio and video pipeline topology map to obtain the verification result; if the verification result is passed, the UI architecture layer audio and video pipeline topology map is transmitted to the native bottom layer.

[0102] Optionally, the above-mentioned processor can also execute the program code of the following steps: before connecting the native underlying audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology diagram to obtain the native underlying audio and video pipeline topology diagram, the method also includes: constructing a mapping relationship between the UI architecture audio and video meta-capability nodes and the native underlying audio and video meta-capability nodes; based on the mapping relationship, calling the native underlying audio and video meta-capability nodes through the UI architecture audio and video meta-capability nodes to realize the target audio and video functions.

[0103] Optionally, the processor may also execute the program code of the following steps: connecting the native underlying audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology diagram to obtain the native underlying audio and video pipeline topology diagram, including: determining the node type of the native underlying audio and video meta-capability node according to the port of the native underlying audio and video meta-capability node; judging whether the native underlying audio and video meta-capability node meets the third preset requirement according to the UI architecture layer audio and video pipeline topology diagram and the node type; if the native underlying audio and video meta-capability node meets the third preset requirement, connecting the native underlying audio and video meta-capability node through the port to obtain the native underlying audio and video pipeline topology diagram.

[0104] Optionally, the processor may also execute the following program code: if the native underlying audio and video meta-capability node does not meet the third preset requirement, an error signal is sent to the UI architecture.

[0105] Optionally, the processor may also execute program code of the following steps: realizing the target audio and video functions according to the native bottom-layer audio and video pipeline topology diagram, including: setting the native bottom-layer application state through the UI architecture, wherein the application state is one of the following: startup state, running state, stop state and destruction state; when the native bottom-layer application state is the startup state, realizing the target audio and video functions according to the native bottom-layer audio and video pipeline topology diagram.

[0106] It can be understood by those skilled in the art that Figure 6 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 6 The structure of the electronic device is not limited. For example, the computer terminal 10 may also include Figure 6 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 6 Different configurations shown.

[0107] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0108] Example 4

[0109] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the method for processing the audio and video functions provided in the first embodiment.

[0110] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0111] Optionally, in this embodiment, the above-mentioned storage medium is configured to store program codes for executing the following steps: atomizing each audio and video function into a corresponding target meta-capability node, and setting a UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and a native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; connecting the UI architecture audio and video meta-capability nodes according to the target audio and video function to obtain a UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data between the native bottom layer; connecting the native bottom layer audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology map to obtain a native bottom layer audio and video pipeline topology map; and realizing the target audio and video function according to the native bottom layer audio and video pipeline topology map.

[0112] The above-mentioned storage medium is also configured to store program codes for executing the following steps: connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram, including: determining the UI architecture audio and video meta-capability nodes to be connected according to the target audio and video functions; determining the node type of the UI architecture audio and video meta-capability nodes to be connected and the type of data transmitted between the UI architecture audio and video meta-capability nodes to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the data type is one of the following: video stream data type and audio stream data type; connecting the UI architecture audio and video meta-capability nodes to be connected according to the node type and the data type to obtain the UI architecture layer audio and video pipeline topology diagram.

[0113] The above-mentioned storage medium is also configured to store program code for executing the following steps: connecting the UI architecture audio and video meta-capability nodes to be connected based on the node type and the data type, and obtaining the UI architecture layer audio and video pipeline topology map including: judging whether the node type meets the first preset requirement; if the node type meets the first preset requirement, then connecting the UI architecture audio and video meta-capability nodes to be connected based on the data type, and obtaining the UI architecture layer audio and video pipeline topology map.

[0114] The above-mentioned storage medium is also configured to store program codes for executing the following steps: after connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology map, the UI architecture layer audio and video pipeline topology map is verified according to the second preset requirement to obtain the verification result; if the verification result is passed, the UI architecture layer audio and video pipeline topology map is transmitted to the native bottom layer.

[0115] The above-mentioned storage medium is also configured to store program codes for executing the following steps: before connecting the native underlying audio and video meta-capability nodes according to the UI architecture layer audio and video pipeline topology diagram to obtain the native underlying audio and video pipeline topology diagram, construct a mapping relationship between the UI architecture audio and video meta-capability nodes and the native underlying audio and video meta-capability nodes; based on the mapping relationship, call the native underlying audio and video meta-capability nodes through the UI architecture audio and video meta-capability nodes to realize the target audio and video functions.

[0116] The above-mentioned storage medium is also configured to store program codes for executing the following steps: according to the UI architecture layer audio and video pipeline topology diagram, connecting the native underlying audio and video meta capability nodes to obtain the native underlying audio and video pipeline topology diagram, including: determining the node type of the native underlying audio and video meta capability node according to the port of the native underlying audio and video meta capability node; judging whether the native underlying audio and video meta capability node meets the third preset requirement according to the UI architecture layer audio and video pipeline topology diagram and the node type; if the native underlying audio and video meta capability node meets the third preset requirement, connecting the native underlying audio and video meta capability node through the port to obtain the native underlying audio and video pipeline topology diagram.

[0117] The storage medium is also configured to store program codes for executing the following steps: if the native underlying audio and video meta-capability node does not meet the third preset requirement, an error signal is sent to the UI architecture.

[0118] The above-mentioned storage medium is also configured to store program code for executing the following steps: realizing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram, including: setting the application state of the native bottom layer through the UI architecture, wherein the application state is one of the following: startup state, running state, stop state and destruction state; when the application state of the native bottom layer is the startup state, realizing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram.

[0119] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0120] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0125] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing audio and video functions, characterized in that: include: Atomize each audio and video function into a corresponding target meta-capability node, and set a UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and a native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer, wherein the UI architecture is used to construct a user interface; Connecting the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain a UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data with the native bottom layer; According to the UI architecture layer audio and video pipeline topology diagram, the native bottom layer audio and video meta-capability nodes are connected to obtain the native bottom layer audio and video pipeline topology diagram; Implementing the target audio and video functions according to the native underlying audio and video pipeline topology diagram; Wherein, realizing the target audio and video function according to the native bottom-layer audio and video pipeline topology diagram includes: The native underlying audio and video meta-capability nodes connected in the underlying audio and video pipeline topology diagram are logically controlled through the UI architecture to achieve the target audio and video functions.

2. The method according to claim 1, characterized in that The UI architecture audio and video meta-capability nodes are connected according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram including: Determine the UI architecture audio and video meta-capability node to be connected according to the target audio and video function; Determine the node type of the UI architecture audio and video meta capability node to be connected and the type of data transmitted between the UI architecture audio and video meta capability node to be connected, wherein the node type is one of the following: production type, consumption type and production consumption type, and the data type is one of the following: video stream data type and audio stream data type; According to the node type and the type of the data, the UI architecture audio and video meta-capability nodes to be connected are connected to obtain the UI architecture layer audio and video pipeline topology diagram.

3. The method according to claim 2, characterized in that According to the node type and the data type, the UI architecture audio and video meta-capability node to be connected is connected to obtain the UI architecture layer audio and video pipeline topology diagram including: Determining whether the node type meets a first preset requirement, wherein the first preset requirement is that for a connection between nodes, only a node with production capability is connected to a node with consumption capability to realize data flow transmission; If the node type meets the first preset requirement, the UI architecture audio and video meta-capability node to be connected is connected according to the type of the data to obtain the UI architecture layer audio and video pipeline topology diagram.

4. The method according to claim 1, characterized in that: After the UI architecture audio and video meta-capability nodes are connected according to the target audio and video functions to obtain the UI architecture layer audio and video pipeline topology diagram, the method further includes: According to the second preset requirement, the audio and video pipeline topology diagram of the UI architecture layer is checked to obtain a verification result, wherein the second preset requirement is whether the node type and the data type meet the requirements, and whether there is no loop phenomenon in the audio and video pipeline topology diagram of the UI architecture layer; If the verification result is passed, the audio and video pipeline topology diagram of the UI architecture layer is transmitted to the native bottom layer.

5. The method according to claim 4, characterized in that Before connecting the native bottom-layer audio and video meta-capability node according to the UI architecture layer audio and video pipeline topology diagram to obtain the native bottom-layer audio and video pipeline topology diagram, the method further includes: Constructing a mapping relationship between the UI architecture audio and video meta capability node and the native bottom layer audio and video meta capability node; According to the mapping relationship, the native underlying audio and video meta capability node is called through the UI architecture audio and video meta capability node to implement the target audio and video function.

6. The method according to claim 5, characterized in that According to the UI architecture layer audio and video pipeline topology diagram, the native bottom layer audio and video meta-capability nodes are connected to obtain the native bottom layer audio and video pipeline topology diagram including: Determine the node type of the native bottom-layer audio and video meta capability node according to the port of the native bottom-layer audio and video meta capability node; According to the UI architecture layer audio and video pipeline topology diagram and the node type, determine whether the native bottom layer audio and video meta capability node meets the third preset requirement, wherein the third preset requirement is to confirm the node type of the native bottom layer audio and video meta capability node and the type of transmitted data when connecting to the native bottom layer audio and video meta capability node; If the native bottom-layer audio and video meta-capability node meets the third preset requirement, the native bottom-layer audio and video meta-capability node is connected through the port to obtain the native bottom-layer audio and video pipeline topology diagram.

7. The method according to claim 6, characterized in that The method further comprises: If the native underlying audio and video meta-capability node does not meet the third preset requirement, an error signal is sent to the UI framework.

8. The method according to claim 1, characterized in that: Implementing the target audio and video functions according to the native underlying audio and video pipeline topology diagram includes: Setting the application state of the native bottom layer through the UI architecture, wherein the application state is one of the following: a start state, a running state, a stop state, and a destroyed state; When the application state of the native bottom layer is the startup state, the target audio and video function is implemented according to the native bottom layer audio and video pipeline topology diagram.

9. A device for processing audio and video functions, characterized in that: include: A setting unit, used to atomize each audio and video function into a corresponding target meta-capability node, and set the UI architecture audio and video meta-capability node corresponding to the target meta-capability node on the UI architecture and set the native bottom layer audio and video meta-capability node corresponding to the target meta-capability node on the native bottom layer; A first connection unit is used to connect the UI architecture audio and video meta-capability nodes according to the target audio and video functions to obtain a UI architecture layer audio and video pipeline topology map, wherein the UI architecture layer audio and video pipeline topology map is used to transmit data with the native bottom layer; A second connection unit is used to connect the native bottom-layer audio and video meta-capability node according to the UI architecture layer audio and video pipeline topology diagram to obtain the native bottom-layer audio and video pipeline topology diagram; An implementation unit is used to implement the target audio and video function according to the native underlying audio and video pipeline topology diagram; wherein the implementation unit is also used to logically control the native underlying audio and video meta-capability nodes connected in the underlying audio and video pipeline topology diagram through the UI architecture to implement the target audio and video function.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the method for processing audio and video functions as described in any one of claims 1 to 8.

11. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method for processing audio and video functions described in any one of claims 1 to 8 when running.

Images