Media resource control method, electronic device and computer readable medium

Abstract

A media resource control method, an electronic device and a computer readable medium. In the method, by means of an interface added in a Linux operating system and used for transmitting a calling instruction and data, a kernel driver of a kernel layer of the Linux operating system is called to start a media device in an electronic device, and media resources acquired by the media device are provided to a Windows application program by means of the interface, wherein the interface for transmitting the calling instruction and the data is a function node that is preset in the Linux operating system.

Description

Media resource control method, electronic device, and computer-readable medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 17, 2023, with application number 202311053188.1 and application name “Media Resource Control Method, Electronic Device and Computer-Readable Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the field of computer technology, and in particular to a media resource control method, electronic equipment, and computer-readable medium. Background Art

[0003] With the development of computer technology, in order to reduce the amount of development, applications developed for one operating system can be compatible with running on other operating systems through methods such as API conversion, creating containers or creating virtual machines. For example, a Windows application developed for the Windows system can be run on operating systems such as Linux, macOS and Free BSD through Wine (Windows environment). Taking the Linux system as an example, Wine can use API conversion technology to create Linux functions corresponding to Windows, so that DLL (a dynamic link library file format) can be called to run Windows applications. It can serve as a compatibility layer for running Windows applications in the Linux system.

[0004] However, Windows applications currently developed using the Media Foundation (MF) framework provided by the Windows operating system cannot access camera information and open the camera through the Linux kernel driver. The MF framework typically runs in Wine on Linux. As a result, these Windows applications often cannot access the data stream captured by the camera when they need to conduct video calls, ultimately rendering the video function of these applications unusable.

[0005] Summary of the Invention

[0006] The present application provides a media resource control method, electronic device, and computer-readable medium. Through the interface provided by the MF framework that can transmit call instructions and data between the kernel layer of the operating system, calls to kernel structures in the kernel layer, such as the video device kernel driver, are implemented. This enables some Windows applications developed based on the MF framework to implement video functions, etc., thereby improving the compatibility of such Windows applications with the MF framework.

[0007] In a first aspect, the present application provides a media resource control method, which is applied to an electronic device, wherein the electronic device is installed with a first operating system and a first application, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and the method includes: detecting a first request from the first application to open a first media device; in response to the first request, transmitting a first call instruction to a first driver of the kernel layer of the first operating system through a first transmission interface, wherein the first transmission interface is developed based on the first operating system, and the first call instruction is used to instruct the first driver to open the first media device requested by the first request; and providing the first media resources collected by the first media device to the first application through the first transmission interface.

[0008] For example, the first operating system is a Linux operating system, the second operating system is a Windows operating system, and the first application is, for example, a Windows application developed based on the Windows operating system. The media resource control method provided by the first aspect is that the Windows application (for example, the first application) on the Linux operating system installed on the electronic device can call the kernel driver (for example, the first driver) of the kernel layer of the first operating system through the interface (for example, the first transmission interface) added in the first operating system for transmitting call instructions and data to open the media device (for example, the first media device) in the electronic device, and can provide the media resources (for example, the first media resources) collected by the media device to the required Windows application through the interface. Wherein, the interface for transmitting call instructions and data can be a functional node preset in the first operating system, and the corresponding first transmission interface can be, for example, a video source acquisition node, and the first media device can be, for example, a camera device opened corresponding to the video source acquisition node.

[0009] Based on the media resource control method provided in the first aspect above, it is possible to enable Windows applications to call kernel structures such as video device kernel drivers in the kernel layer of operating systems such as Linux, thereby enabling some Windows applications to implement video functions, etc., thereby improving the compatibility of such Windows applications.

[0010] In a possible implementation of the first aspect, the first operating system includes an MF framework running in a compatibility layer, the MF framework is provided for the second operating system, and the MF framework includes multiple transmission interfaces, including the first transmission interface.

[0011] Specifically, the interface added to the first operating system for transmitting call instructions and data (e.g., the first transmission interface) can be set within the corresponding structure of the MF framework running in the compatibility layer of the first operating system (e.g., the Linux operating system). Correspondingly, the first application can be a Windows application developed based on the MF framework, referred to as an MF application. This helps improve the compatibility of the MF framework running in non-Windows operating systems.

[0012] In a possible implementation of the first aspect above, detecting a first request from a first application to open a first media device includes: receiving a user operation on a video control on a first interface, where the first interface is an interface displayed corresponding to the electronic device running the first application; and detecting the first request to open the first media device triggered by the user operation.

[0013] In a possible implementation of the first aspect above, providing the first media resource collected by the first media device to the first application through the first transmission interface includes: identifying that the media resource format set by the first application does not match the format of the first media resource collected by the first media device; receiving the first media resource transmitted through the first transmission interface through the second transmission interface in the MF framework, wherein the second transmission interface is an interface with a format conversion function; performing format conversion on the first media resource through the second transmission interface to obtain a second media resource, and providing the second media resource to the first application through the second transmission interface.

[0014] For example, the second transmission interface with format conversion function can be a format conversion node. When this functional node is added to the pipeline corresponding to the camera source, it can convert the format of the media resources (e.g., the first media resource) collected by the media device (e.g., the first media device) on the electronic device before providing it to the required MF application. The media resources collected by the first media device can be, for example, a video data stream, and the format conversion implemented by the corresponding format conversion node can be to convert the video data stream from the original video format supported by the camera to a video format supported by the required MF application.

[0015] In a possible implementation of the first aspect, an output end of the first transmission interface in the MF framework is connected to an input end of the second transmission interface to form a data transmission channel for transmitting the first media resource and the second media resource.

[0016] For example, the data transmission channel formed by connecting the first transmission interface and the second transmission interface may be a camera source pipeline formed by connecting corresponding preset function nodes in the media resources of the MF framework.

[0017] In a possible implementation of the first aspect above, the MF framework further includes a resource reading module, the first media device is a target device selected from multiple media devices obtained through the resource reading module in the MF framework, and the method of obtaining multiple media devices through the resource reading module of the MF framework includes: accessing the dynamic link library of the first operating system through the resource reading module to obtain the first dynamic link library file; reading the calling interface corresponding to the first device enumerator in the first dynamic link library file through the resource reading module, and calling the first device enumerator based on the calling interface to obtain device information of multiple media devices.

[0018] In a possible implementation of the first aspect above, the method of obtaining multiple media devices through the resource reading module of the MF framework also includes: reading, through the resource reading module, configuration information instructing to create a second device enumerator in the first dynamic link library file, and creating the second device enumerator based on the configuration information to obtain device information of multiple media devices.

[0019] In a possible implementation of the first aspect, the device information of the plurality of media devices includes at least one of the following: an identification identifier of the media device; a name of the media device; and a model of the media device.

[0020] In a possible implementation of the first aspect above, the method includes: detecting a second request from a first application to open a locally stored first media file; in response to the second request, transmitting a second call instruction to a second driver of the kernel layer of the first operating system through a third transmission interface in the MF framework, wherein the second call instruction is used to instruct the second driver to open the second media device requested by the second request; and providing the content read by the second media device by calling the storage path of the first media file to the first application through the third transmission interface, wherein the second media device is a virtual device created by virtualization technology.

[0021] For example, the third transmission interface for transmitting the second call instruction can be an Open Local File node. When this function node is added to the pipeline corresponding to the camera source, it can support calling the video device kernel driver in the kernel layer of the Linux operating system to open a virtual camera, such as the second media device, and thus support the corresponding MF application to open local files. If the MF application is, for example, a conferencing application, the Open Local File node's support for opening local files can enable users to share files with other participants.

[0022] In a possible implementation of the first aspect above, providing the content read by the second media device by calling the storage path of the first media file to the first application through the third transmission interface includes: identifying that the screen size required for the second media device to call the storage path of the first media file to read does not match the screen size corresponding to the display of the second interface of the first application; receiving the content read by calling the storage path of the first media file transmitted through the third transmission interface through the fourth transmission interface in the MF framework, wherein the fourth transmission interface is an interface with a cropping function; cropping the content of the first media file to the required screen size through the fourth transmission interface, and providing the cropped content of the second media file to the first application through the fourth transmission interface.

[0023] For example, the fourth transmission interface with a cropping function can be a cropping node. When added to the pipeline corresponding to the camera source, this function node can crop the size of a corresponding media file (e.g., the first media file) to meet the requirements of the MF application for displaying the corresponding content of the media file in the corresponding window. The first media file can be, for example, an image or video file, etc., without limitation herein.

[0024] In a possible implementation of the first aspect above, the method includes: detecting a third request from the first application to share desktop display content; in response to the third request, transmitting a third call instruction to a third driver of the kernel layer of the first operating system through the fifth transmission interface in the MF framework, wherein the third call instruction instructs the third driver to open a second media device to record the desktop display content; and providing the third media resources generated corresponding to the desktop display content recorded by the second media device to the first application through the fifth transmission interface.

[0025] For example, the fifth transmission interface for transmitting the third call instruction can be a desktop screen recording node. When this function node is added to the pipeline corresponding to the camera source, it can support the MF application in providing a shared desktop function. For example, the desktop display content recorded by the virtual camera (i.e., the second media device) opened by the desktop screen recording node can be provided to the MF application. This is not further described here.

[0026] In a possible implementation of the first aspect, the first media device includes a camera device.

[0027] In a possible implementation of the first aspect, the second media device includes a virtual camera.

[0028] In a second aspect, the present application provides an electronic device having a first operating system and a first application installed thereon, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and the electronic device includes a first transmission interface developed based on the first operating system, wherein the first transmission interface is used to send a first call instruction to a first driver of the kernel layer of the first operating system, wherein the first call instruction is used to respond to a first request of the first application to open a first media device, and instruct the first driver to open the first media device requested by the first request; and the first transmission interface is used to provide the first media resources collected by the first media device to the first application.

[0029] In a possible implementation of the second aspect, the first operating system includes an MF framework running in a compatibility layer, the MF framework is provided for the second operating system, and the MF framework includes multiple transmission interfaces, including the first transmission interface.

[0030] In a possible implementation of the second aspect above, the multiple transmission interfaces include a second transmission interface, wherein the second transmission interface is an interface with a format conversion function, and the second transmission interface is used to receive a first media resource transmitted through the first transmission interface, and the second transmission interface is used to perform format conversion on the first media resource to obtain a second media resource, and provide the second media resource to the first application.

[0031] In a possible implementation of the second aspect above, the multiple transmission interfaces include a third transmission interface, wherein the third transmission interface is used to transmit a second call instruction to a second driver of the kernel layer of the first operating system, wherein the second call instruction is used to respond to a second request of the first application to open a locally stored first media file, and instruct the second driver to open the second media device requested by the second request; and the third transmission interface is used to provide the content read by the second media device calling the storage path of the first media file to the first application, wherein the second media device is a virtual device created through virtualization technology.

[0032] In a possible implementation of the second aspect above, the multiple transmission interfaces include a fourth transmission interface, wherein the fourth transmission interface is an interface with a cropping function, and the fourth transmission interface is used to receive content read by calling the storage path of the first media file transmitted through the third transmission interface, and the fourth transmission interface is used to crop the content of the first media file to correspond to the required screen size, and provide the cropped content of the second media file to the first application.

[0033] In a possible implementation of the second aspect above, the multiple transmission interfaces include a fifth transmission interface, wherein the fifth transmission interface is used to transmit a third call instruction to a third driver of the kernel layer of the first operating system, wherein the third call instruction is used to respond to a third request of the first application to share desktop display content, and instruct the third driver to open a second media device to record the desktop display content; and the fifth transmission interface is used to provide the third media resources generated corresponding to the desktop display content recorded by the second media device to the first application.

[0034] In a third aspect, the present application provides an electronic device comprising: one or more processors; one or more memories; one or more memories storing one or more programs, which, when one or more programs are executed by one or more processors, enables the electronic device to execute the media resource control method provided in the above-mentioned first aspect and various possible implementations of the first aspect.

[0035] In a fourth aspect, the present application provides a computer-readable medium having instructions stored thereon, which, when executed on a computer, causes the computer to execute the media resource control method provided in the above-mentioned first aspect and various possible implementations of the first aspect.

[0036] The beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions of the first aspect and various possible implementations of the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1a is a schematic diagram showing an interface in which a Windows application cannot obtain a data stream collected by a camera during operation.

[0038] FIG1b is a schematic diagram of a Linux operating system architecture.

[0039] FIG2 a shows a schematic diagram of a Linux system architecture provided in an embodiment of the present application.

[0040] FIG2 b is a schematic diagram showing a process of setting functional nodes on a camera source pipeline based on application requirements, provided by an embodiment of the present application.

[0041] FIG2 c is a schematic diagram showing a conference interface of an MF application 211 provided in an embodiment of the present application.

[0042] FIG3 is a schematic diagram showing an implementation flow of a media resource control method provided in an embodiment of the present application.

[0043] FIG4 a is a schematic diagram showing the changes in the conference interface displayed before and after format conversion processing is performed on the collected camera data stream when the video function is enabled in an MF application 211 according to an embodiment of the present application.

[0044] FIG4 b is a schematic diagram showing a conference interface displayed when the MF application 211 opens a local file according to an embodiment of the present application.

[0045] FIG4 c is a schematic diagram showing a conference interface displayed when the MF application 211 provides a desktop sharing method according to an embodiment of the present application.

[0046] FIG5 is a schematic diagram showing an implementation flow of Wine traversing camera information of an electronic device provided in an embodiment of the present application.

[0047] FIG6 is a schematic diagram showing an implementation flow of Wine setting a video source acquisition node on a camera source pipeline according to an embodiment of the present application.

[0048] FIG7 is a schematic diagram showing an implementation flow of adding a format conversion node to a camera source pipeline in Wine according to an embodiment of the present application.

[0049] FIG8 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0051] FIG1a shows a schematic diagram of an interface in which a Windows application cannot obtain a data stream collected by a camera during operation.

[0052] As shown in Figure 1a, an electronic device 100 running a Linux operating system can display a conference interface 01 when running some conferencing application. When a user clicks the video control 02 on conference interface 01 to initiate a video conference, the conferencing application must open the camera to obtain the data stream captured by the camera. It can be understood that a stream is a sequence of media data of a uniform type. For example, the aforementioned data stream can be a sequence of image data captured by a camera, which in some embodiments can also be referred to as a video stream.

[0053] Figure 1b shows a schematic diagram of an existing operating system architecture. Referring to Figure 1b, the application 101 running in the application layer of the operating system can be a conference application or an office application, etc. The application is a Windows application developed using the MF framework provided by the Windows operating system.

[0054] Taking the Linux operating system as an example, the operating system compatibility layer shown in FIG1b can be compatible with the MF framework 102 running the Windows operating system. The MF framework 102 can support accessing media resources 106 through a source reader 104 to obtain media source data. The media source data can include local files (such as video files), videos loaded from the network, and data captured by the electronic device's hardware capture device (such as a camera).

[0055] In addition, as shown in FIG1b , the MF framework 102 of the Windows operating system may further include a media session 103 and a sink writer 105. The media session 103 is created to control the media pipeline, respond to requests from some Windows applications, and read media source data managed by a media source 106. In some embodiments, the media session 103 may also open local files through a media foundation transform (MFT) 107, and may also transmit a receive instruction to a media sink 108 to obtain data such as video loaded from the network.

[0056] The receiver writer 105 can obtain the media source data through the media receiver 108. No further details will be given here.

[0057] However, when a Windows application developed based on the MF framework, such as application 101 (hereinafter referred to as MF application), needs to turn on the camera, the MF framework shown in FIG1b does not have an interface for accessing the kernel layer of the Linux operating system, and is unable to call the kernel driver and related audio and video processing framework of the kernel layer, for example, it is unable to call the video device kernel driver 109 shown in FIG1b. Therefore, when Windows applications currently developed based on the MF framework of the Windows operating system are compatible and run in other operating systems such as Linux, they cannot turn on the camera, and therefore cannot support the video function. Referring to FIG1a above, application 101, such as a conference application, cannot turn on the camera, and the corresponding conference interface 01 may display a black screen or a prompt message, such as the prompt message shown in FIG1a, which can refer to the content such as "Unable to obtain the camera".

[0058] In order to solve the above problems, the present application provides a media resource control method. Specifically, the method adds one or more interfaces in the MF framework that can transmit call instructions and data between the kernel layer of the operating system, so as to enable the MF framework to call kernel structures in the kernel layer, such as the video device kernel driver, and thus support some MF applications to access other operating system kernels to open cameras, thereby enabling MF applications to implement video functions. Among them, the method of adding the above interfaces in the MF framework can be implemented by presetting some functional nodes in the media resources of the MF framework.

[0059] For example, in an embodiment of the present application, a video source acquisition node can be preset in the media resources of the MF framework. When the MF application turns on the video function in response to a user operation, the media resources of the MF framework can construct a pipeline (Pipeline) for transmitting data streams between the MF framework and the video device kernel driver of the kernel layer based on the video source acquisition node, such as the camera source pipeline shown in Figure 2b below. The corresponding position of the camera source pipeline can refer to Figure 2a below, and is not limited here. At this time, the pipeline can be used as one of the media resources to open the camera for the MF application and obtain the video data stream captured by the camera. In an embodiment of the present application, the above-mentioned pipeline constructed in the media resources according to the needs of the MF application can also be called a camera source (camera resource).

[0060] In this way, when a Windows application developed based on the MF framework of the Windows operating system is compatible and runs on other operating systems, it can call the kernel driver of other operating systems to open the camera and realize the video function.

[0061] It can be understood that when the format of the media source data requested by the MF application through the MF framework is not consistent with the format supported by the camera opened by the kernel driver, a format conversion node can be preset in the media resource of the above-mentioned MF framework. At this time, when the MF application uses the video function, the above-mentioned format conversion node can be added to the pipeline constructed in the above-mentioned media resource to perform format conversion on the video data stream collected by the camera opened by the video device kernel driver, and then provide it to the MF application. Similarly, in some embodiments, the above-mentioned media resources, or the constructed camera source pipeline, can also include other functional nodes for implementing different data processing functions, such as frame rate adjustment nodes, cropping nodes, local file opening nodes, and desktop screen recording nodes, etc., for further realizing format conversion, frame rate adjustment, image cropping, etc. of the media source data. There is no limitation here.

[0062] The aforementioned camera source pipeline includes various functional nodes added to meet application needs, such as the aforementioned format conversion node, frame rate adjustment node, and cropping node. This allows operating systems like Linux to support Windows applications based on the MF framework to open image or video files of various formats, improving Windows application compatibility with image or video formats. Furthermore, the system can adjust the frame rate to suit Windows applications, improving the quality of images or videos displayed by Windows applications.

[0063] Furthermore, the aforementioned pipeline-based processing, achieved by adding functional nodes, allows processing of the raw camera data stream at the underlying operating system level, thereby reducing the amount of data output from the underlying operating system to the compatibility layer. This reduces the amount of data input to the MF framework in the compatibility layer, reducing the storage space and transmission channel bandwidth occupied by the transmitted data stream in Wine. This, in turn, improves the processing performance of the compatibility layer, for example, by increasing the efficiency of the MF framework in processing media source data.

[0064] It is understood that the kernel driver supported by the video source acquisition node added to the camera source pipeline may include a video device kernel driver provided by system libraries such as the Linux kernel's v4l2.lib, such as a Universal Serial Bus (USB) video class (UVC) driver. Furthermore, operating systems such as Linux, while providing these kernel drivers, may also provide audio and video processing frameworks required for processing video and image data, such as gstreamer and FFmpeg, without limitation.

[0065] It can be understood that the media resource control method provided in this application is applicable to electronic devices including but not limited to desktop, laptop and handheld computers, notebooks, netbooks, tablets, mobile phones, augmented reality (AR) / virtual reality (VR) devices, smart TVs, smart watches and other wearable devices, servers, car equipment, portable game consoles, portable music players, and other electronic devices with one or more processors and the ability to play media files such as audio and video.

[0066] In some embodiments, the media resource control method provided in this application can also be applied to other compatible layer architectures, such as deepin-wine, kylin-wine, crossover, etc., without limitation here.

[0067] Figure 2a shows a schematic diagram of a Linux system architecture according to an embodiment of the present application. It is understood that the architecture shown in Figure 2a is not intended to limit the Linux operating system architecture. In other embodiments, the Linux system architecture may also include more or fewer structures than shown in Figure 2a.

[0068] As shown in FIG. 2 a , the Linux operating system 200 may include an application layer 210 , a compatibility layer 220 , and a kernel layer 230 .

[0069] The application program running in the application layer 210 may include a Windows application program developed based on the MF framework of the Windows operating system, hereinafter referred to as an MF application, with reference to the MF application 211 shown in FIG. 2 a .

[0070] Compatibility layer 220 can be implemented using Wine, and may also be referred to as Wine 220 hereinafter. As shown in Figure 2a , Wine 220 can run a MF framework 221. This MF framework 221 may include a media session 222 created to obtain media source data, as well as a source reader 223 and a receiver writer 224. In some embodiments, when MF application 211 needs to open a camera, it can obtain a camera data stream via source reader 223 or media session 222. When MF application 211 needs to open a local media file or a video loaded from the network, it can read the corresponding media file from media resource 225 as media source data via source reader 223. In an embodiment of the present application, source reader 223 can obtain processed camera data streams via functional nodes added to the camera source 226 pipeline. A specific example of the camera source 226 pipeline can be found in Figure 2b and the related description below. In other embodiments, MF application 211 may also obtain media source data such as video received by media receiver 228 via receiver writer 224, without limitation herein.

[0071] It can be understood that the above media session 222 can be created in response to the MF application 211's request to obtain media source data such as images, audio and video, and is used to provide a transmission channel for sending the corresponding media source data to the MF application 211.

[0072] In some embodiments, when the MF application 211 uses the video function to request to open the camera, it can first call the system's dynamic link library file, such as mfplat.dll, through the source reader 223 or the created media session 222, to read the device enumerator interface written in the file to call the system device enumerator, or read the configuration information instructing to create the enumerator in the file to create an enumerator for identifying the video device, etc. The details will be described in detail below and will not be repeated here. The above-mentioned system device enumerator is used to find and create media conversion and media sources registered on the system, such as the media source such as the camera data stream obtained through the camera source 226 pipeline described below. mfplat.dll can also be configured with attribute values ​​that need to be set when adding some functional nodes to the camera source 226 pipeline, etc., which are not limited here.

[0073] In some embodiments, the MF framework 221 may also include a media base transformation 227, which is used to uniformly record information such as the name and acquisition path of media source data such as images, audio, and video in the system. The information recorded in the media base transformation 227 may include media source data such as video received by the media receiver 228.

[0074] Continuing with reference to FIG. 2 a , the kernel layer 230 may include a video device kernel driver 231. In a Linux operating system, the kernel layer 230 may be implemented based on the Linux kernel. The video device kernel driver 231 in the kernel layer 230 may include a USB video class driver, etc. In the kernel layer 230, these kernel drivers may be provided by v4l.lib or v4l2.lib.

[0075] In addition, operating systems such as Linux can also provide some audio and video processing frameworks required for processing video and image data, including gstreamer, FFmpeg, etc., which can be used to encode, decode, process and transmit the video data stream collected by the camera, etc., without limitation here.

[0076] In other embodiments, Wine 220 in the system architecture shown in 2a can also be set in other operating systems, such as macOS and Free BSD, etc., which is not limited here.

[0077] It is understood that the camera source 226 added to the MF framework 221 can be used as a subclass of the media resource 225 to provide media source data for the media session 222 or the source reader 223. The camera source 226 can be implemented based on a pipeline, and some preset functional nodes can be added to the pipeline of the camera source 226 according to the requirements of the MF application 211.

[0078] FIG2 b shows a schematic diagram of a process for setting functional nodes on a camera source pipeline based on application requirements according to an embodiment of the present application.

[0079] As shown in FIG2B , the configuration file corresponding to the pipeline corresponding to the camera source 226 may have multiple function nodes preset, such as a video source acquisition node and a format conversion node for opening a physical camera, an open local file node and a desktop screen recording node for calling a virtual camera, and a cropping node for cropping the image / video image to fit the screen size. The configuration file may be, for example, a dynamic link library file such as mfplat.dll, which is not limited here. The virtual camera may be a camera created using virtualization technology that has similar image acquisition functions as a physical camera. The virtual camera may be created and opened using the corresponding kernel driver in V4L2 provided by the kernel layer 230.

[0080] It can be understood that the process of opening local files implemented by the above-mentioned open local file node is essentially to map local files, such as images or video files, to the created virtual camera display, and then provide them to the required MF application 211. Taking the conference application as an example, the function of opening local files supported by the above-mentioned open local file node can support the user to share files with other participants. The process of sharing the desktop implemented by the above-mentioned desktop recording node can be to map the desktop display content to the created virtual camera recording and display, and then provide it to the required MF application 211, which will not be elaborated here.

[0081] For example, using a conference application as an example, when MF application 211 needs to enable the video function during a conference, the application can pass certain relevant node attributes to camera source 226 via source reader 223. Based on the node attributes passed by MF application 211, camera source 226 can then add pre-defined video source acquisition nodes and format conversion nodes, such as those for opening the physical camera, to the pipeline. This allows these nodes to call the video device kernel driver of kernel layer 230 to open the camera and perform format conversion on the camera data stream, ultimately achieving the video function of MF application 211. The video source acquisition node for acquiring the physical camera data stream can be linked to the kernel driver corresponding to the camera device selected by MF application 211 to enable the camera. If the video format supported by MF application 211 is inconsistent with the format supported by the camera of electronic device 100, the format conversion node added to the pipeline corresponding to camera source 226 can perform format conversion on the video data stream acquired from the camera video source before providing it to MF application 211.

[0082] It is understood that in some embodiments, other MF applications 211 can also obtain image or video data streams captured by the camera through the media resource 225 based on the above-mentioned media session 222. However, the format of the image or video data stream obtained in this manner is limited to the format supported by the camera device configured on the electronic device 100, and the frame rate of the image or video data stream can only be provided to the application according to the frame rate configured by the camera device, and cannot be dynamically adjusted according to the settings of the MF application 211 in the application layer 210. In contrast, the source reader 223 or media session 222 calls the video device kernel driver 231 through the video source acquisition node added to the camera source 226 pipeline to open the camera, which can avoid the problem of the MF application 211 being unable to obtain the camera. In addition, this method of opening the camera can also solve the problem of the image or video screen not being able to display normally when the format supported by the physical camera configured on the electronic device 100 is inconsistent with the format set by the application. The specific interface displayed when the MF application 211 provides a video function based on the media resource control method provided in this application can be referred to as shown in Figure 2c below and the related description, and will not be repeated here.

[0083] Continuing with Figure 2b , when MF application 211 needs to open local media source data such as video files, MF application 211 can pass other relevant node attributes to camera source 226 via source reader 223. Camera source 226 can then, based on the node attributes passed by MF application 211, add preset local file opening nodes and cropping nodes that can invoke a virtual camera to the pipeline, thereby enabling MF application 211 to open local files such as images or videos. For another example, when the user of MF application 211 needs to share their desktop with other participants, MF application 211 can also pass other relevant node attributes to camera source 226 via source reader 223. Camera source 226 can then add a preset desktop recording node to the pipeline, enabling MF application 211 to support desktop sharing by recording the desktop display content. In other embodiments, the functional node supporting MF 211 in implementing the shared desktop function can also be other types of nodes, such as access nodes that can access the desktop display data rendered by the rendering process, without limitation.

[0084] FIG2 c shows a schematic diagram of a conference interface of an MF application 211 according to an embodiment of the present application.

[0085] As shown in Figure 2c, based on the media resource control method provided in this application, the operating system on electronic device 100 can support the video function of MF application 211 based on the MF framework. Therefore, when a user clicks video control 201 on conference interface 201 shown in Figure 2c, conference interface 201 can capture images based on the open camera, so that the person shown in Figure 2c or other images captured by the camera are displayed on conference interface 201.

[0086] Based on the system architecture shown in Figure 2a and the implementation principle of adding functional nodes to the camera source pipeline shown in Figure 2b, the following takes an application request to open a camera as an example, and combines other drawings to introduce in detail the specific implementation process of the media resource control method provided in the embodiment of the present application.

[0087] FIG3 is a schematic diagram illustrating an implementation process of a media resource control method according to an embodiment of the present application. It is understood that the execution entity of each step in the process shown in FIG3 can be Wine 220 in the system architecture shown in FIG2a above, or can be the electronic device 100 installed with the operating system shown in FIG2a above. In other embodiments, Wine 220 can also be set in other operating systems that need to be compatible with running Windows applications, and the execution entity of each step in the implementation process shown in FIG3 can also be other electronic devices installed with the corresponding operating system, without limitation herein.

[0088] It should also be stated that the steps in the methods and processes in the embodiments of the present application are numbered for ease of reference, rather than to limit the order of precedence. If there is a sequence between the steps, the written description shall prevail.

[0089] As shown in Figure 3, the implementation process includes the following steps:

[0090] 301: In response to the application's request to open the camera, traverse the camera information.

[0091] For example, in response to an application's request to open a camera, the electronic device 100 or the aforementioned Wine 220 can load the dynamic link library file mfplat.dll through the source reader 223 or the media session 222 and read the enumerator interface preset in the file, thereby calling the system device enumerator to traverse various types of hardware device information on the electronic device 100. In addition, the mfplat.dll can also preset instructions to create an enumerator component object model (COM) capable of identifying video devices. Wine 220 can create a video device enumerator based on the COM component and obtain relevant information about one or more physical cameras, referred to as camera information, from the various types of hardware device information obtained by the traversal. The camera information may include, but is not limited to, the identity identifier (ID), name, model, etc. of each camera device.

[0092] It is understood that the MF application 211 can be a Windows application developed based on the MF framework and compatible with the Linux operating system, such as a conferencing application. As an example, referring to FIG. 2c , the conferencing application's request to turn on the camera can correspond to a user clicking the video control 202 on the conferencing application's conference interface 201. Wine 220 of the Linux operating system can receive the camera turn-on request sent by the MF application 211 in response to the user's operation.

[0093] It can be understood that the above-mentioned "traversal" includes the meanings of querying, copying, and obtaining as the name implies. In the embodiment of the present application, traversing the camera information means querying the relevant information of the camera device, and may also include copying and obtaining the queried camera information, which is not limited here. After traversing the camera information, Wine220 can continue to execute the following step 302 to return the camera information to the required application (such as the above-mentioned MF application 211). Please refer to the relevant description in the following steps for details, which will not be repeated here. In addition, the above-mentioned system device enumerator based on the preset enumerator interface call can be implemented by other frameworks (such as the DirectShow framework) running in Wine 220, for example, and is not limited here.

[0094] 302: Return the traversed camera information to the application.

[0095] For example, after executing step 301 to retrieve information about one or more configured cameras, the electronic device 100 or the Wine 220 may return the retrieved camera information corresponding to each camera to the application requesting the camera to be opened. For example, Wine 220 may return the retrieved camera information to the MF application 211 requesting the camera to be opened.

[0096] It is understood that when the traversed camera information includes information related to multiple cameras, Wine 220 can return the camera information to the application in the form of a device list. Specifically, the process of Wine 220 assigning a device list to the traversed camera information and setting the relevant properties in the device list can be referred to in the description below in conjunction with the corresponding figures, and will not be repeated here.

[0097] 303: Receive information related to the target camera selected by the application.

[0098] For example, after receiving the camera information returned by Wine 220, MF application 211, such as a conferencing application, that requests to open a camera, can select one or more cameras as hardware capture devices to implement the video function based on its needs. The cameras selected by the application are hereinafter referred to as target cameras. The target camera-related information received at this time may include information such as the video formats and frame rates supported by the cameras.

[0099] As an example, MF application 211 may select the front camera of electronic device 100 as the hardware capture device for capturing the video data stream in response to a user clicking a video control. In some embodiments, if electronic device 100 is also equipped with a rear camera, when a user clicks a switch control on the conference interface of MF application 211, MF application 211 may also select the rear camera of electronic device 100 as the hardware capture device in response to the switch operation, without limitation herein.

[0100] 304: Parse the device path of the target camera.

[0101] For example, the electronic device 100 or the Wine 220 can obtain the relevant configuration file of the device and parse the device path of the camera device based on the camera information selected by the application, such as the camera device ID in the information. The parsed device path may include the call path of the video device that can open the target camera.

[0102] 305: Add a video source acquisition node to the initialized camera source pipeline and set the camera video source.

[0103] For example, after the electronic device 100 or the aforementioned Wine 220 parses and obtains the device path of the target camera, it can target the camera according to the device path and set the opened target camera as the video source. Specifically, Wine 220 can implement the method of setting the video source by adding a video source acquisition node to the pipeline corresponding to the camera source 226 of the MF framework 221.

[0104] As an example, during the development of the Linux operating system 200, a v4l2src (video for linux2 source) node can be created based on the gstreamer framework developed for Linux v4l2 as a preset video source acquisition node. Furthermore, when the pipeline corresponding to the camera source 226 in Wine 220 needs to add a video source acquisition node according to the needs of the application, the relevant path of the preset node can be read to complete the addition. Among them, v4l2src can be used to obtain elements of video data that can be collected by the corresponding camera from the video device kernel driver provided by v4l2.lib. After Wine 220 sets the attributes of the corresponding element according to the application requirements, the gstreamer framework supported by the Linux kernel can collect the data stream of the corresponding camera device based on the set attributes. Specifically, Wine 220 can implement the process of adding the above-mentioned element to the pipeline corresponding to the camera source 226 through the gstreamer framework and linking it to the video source acquisition node.

[0105] The pipeline corresponding to the camera source 226 can convert the state of the node element to the ready state after receiving the video source acquisition node added by the gstreamer framework link. At this time, the video source acquisition node on the pipeline can enter the ready state.

[0106] It is understandable that before setting the video source, the pipeline corresponding to the camera source 226 of the MF framework 221 needs to be initialized first, including initializing the pipeline interface and initializing various preset function nodes that can be added to the pipeline.

[0107] It can be understood that the detailed implementation process of the above steps 303 to 305 corresponding to setting the video source acquisition node according to the camera selected by the MF application 211 will be described in detail below in conjunction with another flowchart and will not be repeated here.

[0108] 306: Add other functional nodes required by the application to the camera source pipeline.

[0109] For example, the aforementioned additional functional nodes include format conversion nodes and frame rate adjustment nodes. For example, if the format set by the application is not supported by the physical camera, Wine 220 can add a preset format conversion node to the pipeline corresponding to camera source 226 to perform format conversion processing on the video data stream captured by the target camera. The format conversion function implemented by this format conversion node can be implemented by developing corresponding code using the GStreamer framework.

[0110] It is understandable that the initial format converted by the above format conversion node may default to the format supported by the physical camera for capturing video data streams, and the target format of the corresponding conversion may be determined based on the negotiation result between the application and Wine 220 .

[0111] Specifically, the process of adding a format conversion node and performing format negotiation based on step 306 may be described in detail below in conjunction with the corresponding drawings, and will not be repeated here.

[0112] 307: Transmitting the camera data stream processed based on the preset function node to the application.

[0113] For example, camera source 226 adds preset functional nodes to the corresponding pipeline based on the application's needs, including the aforementioned video source acquisition node and format conversion node. At this point, when the MF framework 221 in Wine 220 acquires the camera data stream through camera source 226, camera source 226, based on the video source acquisition node added to the pipeline, acquires the individual frames of imagery provided by the Linux kernel after acquisition and format conversion based on the gstreamer framework. Referring to FIG. 2c , MF application 211 can display the video image on conference interface 201 based on the camera data stream obtained by the video stream acquisition node on camera source 226, resolving the issue of MF applications being unable to acquire camera data when running on compatible operating systems.

[0114] Furthermore, based on the format conversion node added to the pipeline, the video data stream formed by encoding each frame of the image can be converted into the format required by the application. For example, based on the above-mentioned format conversion node, the video data stream in YUV422 format captured by the camera can be converted into a video data stream in YUV420 format used by the MF application 211. When the format supported by the camera is inconsistent with the format set corresponding to the video source obtained by the MF application 211, even if the MF application 211 can open the camera, the conference interface may still display the garbled image 401 shown in Figure 4a. Based on the format conversion node added to the camera source 226 pipeline, after the camera data obtained by the video source acquisition node is format-converted, the MF application 211 obtains the processed camera data stream with a matching format provided by Wine 220, and can then display the clear image 402 in the conference interface shown in Figure 4a.

[0115] In other embodiments, the pipeline corresponding to the camera source 226 may further include a pre-defined functional node, such as a frame rate adjustment node, to further perform frame rate adjustment and other processing on the camera data stream transmitted to the MF application 211. This is not a limitation here. In this way, the MF application 211 can obtain the camera data stream processed by the corresponding functional node from Wine 220 and play or display it without further format conversion or frame rate adjustment, thereby improving the efficiency of the obtained video data stream. Furthermore, the data volume of the processed data stream may be significantly reduced, thereby reducing the channel bandwidth occupied during the transmission of the data stream.

[0116] It is understood that the aforementioned video source acquisition node is a node capable of opening the physical camera of the electronic device 100. In other embodiments, the camera source pipeline may also include a node capable of calling a virtual camera to open local files. It is understood that this virtual camera can be a camera created using virtualization technology and having similar image acquisition functions as a physical camera.

[0117] As an example, referring to FIG4b , MF application 211 can open a local image 403 or video file through Wine 220. The local image can be opened using an Open Local File node added to the pipeline corresponding to camera source 226 for invoking a virtual camera. Furthermore, in this scenario, a Crop node for cropping the image size can also be added to the pipeline corresponding to camera source 226, thereby cropping the opened image to a size suitable for the currently displayed screen. This is not a limitation here.

[0118] In other embodiments, the above-mentioned preset function nodes may also include function nodes such as desktop screen recording. Continuing with the MF application 211 as an example, when a user needs to share the desktop with other participants in a meeting, or operate some application interfaces opened on the desktop, the desktop screen recording node added to the corresponding pipeline of the camera source 226 of Wine 220 can be used to call the virtual camera to record the current desktop display content and provide it to the MF application 211. In this way, other participants can synchronously see the recorded images such as the desktop displayed in the conference interface. The desktop can refer to the desktop 404 displayed on the conference interface shown in Figure 4c.

[0119] The specific implementation process of traversing the camera information in the above step 301 is described in detail below with reference to FIG. 5 .

[0120] FIG5 shows a schematic diagram of an implementation flow of Wine 220 traversing camera information of an electronic device according to an embodiment of the present application.

[0121] It is understood that when the Windows application running on the electronic device 100 needs to enable the video function, it can initiate a request to the operating system's Wine 220 to call the video device kernel driver to open the camera. At this time, Wine 220 can respond to the MF application 211's request to open the camera and execute the steps in the process shown in Figure 5 to traverse and query the various attribute data of one or more cameras configured on the electronic device 100.

[0122] As shown in Figure 5, the implementation process may include the following steps:

[0123] 501: Loading or creating a device enumerator.

[0124] Exemplarily, Wine 220 can provide an interface file for the call path corresponding to the system device enumerator, such as mfplat.dll. The dynamic link library file can pre-set the relevant interface parameters for calling the system device enumerator provided by other multimedia frameworks, such as the relevant interface parameters for the system device enumerator provided by the DirectShow framework, etc., which are not limited here. In addition, mfplat.dll can also preset instructions for creating an enumerator COM component that can identify video devices. For example, Wine 220 can load the preset enumerator COM component to create a video device enumerator by executing the function instruction "create class enumerator". The COM component can be used to create a video device enumerator, and the enumerator can further obtain the relevant information of the physical camera one by one from the various types of hardware device information obtained by the above traversal, that is, the camera information obtained in the above step 301.

[0125] It is understood that Wine 220 can load the system device enumerator by executing step 401 to obtain relevant information about the camera device and other hardware devices recorded in the system of the electronic device 100. The video device enumerator created by Wine 220 based on the preset COM component can filter the enumeration results of the system device enumerator to obtain video device enumerators such as camera devices.

[0126] 502: Allocate device list storage space for the relevant information of the i-th device obtained through enumeration.

[0127] Exemplarily, the device enumerator can enumerate the video device-related information in the electronic device 100 one by one based on preset enumeration conditions. For example, the device enumerator can first enumerate the ID, name, and other information of the first camera device. At this time, Wine 220 can allocate a device list for the camera device to store the camera information such as the ID and name. The device list can be a data table or data storage space preset in Wine 220 for returning camera information to the MF application 211. The camera information obtained by the enumerator by traversing the video devices in the electronic system device, such as the camera ID and other information, can be stored in the preset device list for providing to the required application, such as the MF application 211.

[0128] In other embodiments, the device enumerator may also sequentially enumerate the camera information corresponding to the 2nd / 3rd / …ith camera devices. Correspondingly, Wine 220 may reallocate the size of the device list space based on the sequentially enumerated camera information or other device information. The more device information enumerated, the larger the device list space allocated by Wine 220 may be, so as to fully store the enumerated device information. Therefore, after Wine 220 completes step 502 to allocate the device list for the relevant information of the i-th device, it may continue to perform the judgment process of the following step 503 to determine whether there are any unenumerated cameras or other devices in the electronic device 100. For details, please refer to the relevant description below.

[0129] It is understood that the enumeration results may include different video device-related information for different application requirements. For example, if MF application 211 requires opening a camera, the enumeration results may include information such as the ID, name, and model of each camera. For another example, if MF application 211 requires opening a local file, the enumeration results may include information about virtual cameras created by the system, etc., without limitation.

[0130] 503: Determine whether there is relevant information of the (i+1)th device obtained through enumeration.

[0131] If the judgment result is yes, it indicates that there are devices such as cameras that have not been added to the device list. At this time, Wine 220 can continue to perform the following step 504 to reallocate the size of the device list space;

[0132] If the determination result is no, it indicates that all camera devices in the electronic device 100 have been enumerated. At this time, Wine 220 may execute the following step 508 to release the device enumerator.

[0133] For example, Wine 220 can continue to traverse the enumeration results corresponding to other devices such as cameras in the electronic device 100 based on the device enumerator to determine whether relevant information about the i+1th device has been obtained through enumeration. In the above example, if i = 1, Wine 220 performs the judgment process of this step 503, which can be to determine whether relevant information about the second device, such as camera information, has been obtained through enumeration. If i = 2, Wine 220 performs the judgment process of this step 503, which can be to determine whether relevant information about the third device has been obtained through enumeration, which will not be repeated here.

[0134] 504: Increase the list space size for the (i+1)th camera device.

[0135] Exemplarily, Wine 220 may continue to increase the size of the device list space for the (i+1)th device obtained through enumeration, so as to store the camera information corresponding to the newly added (i+1)th camera device.

[0136] In the aforementioned example, if i = 1, Wine 220 executes step 504 to increase the size of the device list space for the second camera device enumerated, including its ID and name. This may also be referred to as the list size in the context of this embodiment. Similarly, if i = 2, Wine 220 executes step 504 to increase the size of the device list space for the third camera device enumerated, including its ID and name. This is not further described here.

[0137] 505: Copy the camera information obtained by enumeration to the device list space.

[0138] For example, Wine 220 can copy the enumerated camera information in the device list space allocated for each camera device obtained by enumeration, wherein the camera information may include but is not limited to the device ID, name, model and other information.

[0139] 506: Determine whether the copy is successful.

[0140] If the judgment result is yes, that is, when the copy is successful, Wine 220 can execute the following step 507 to return the camera information to the MF application 211;

[0141] If the determination result is no, ie, the copying fails, Wine 220 may execute the following step 510 to release the device list.

[0142] 507: Set other properties of the device list.

[0143] For example, after all camera information obtained by the device enumerator is copied to the device list, Wine 220 can set other properties for each camera in the device list. For example, Wine 220 can set a name for the camera to be recognized by the MF application 211 and another name to be recognized by the system, based on the needs of the MF application 211 or the system, and this is not limited here.

[0144] After Wine 220 completes step 507 , it may return to step 503 and continue traversing the (i+2)th device.

[0145] 508: Release the device enumerator.

[0146] For example, if Wine 220 determines that the result of step 503 is negative, i.e., all camera devices in the electronic device 100 have been enumerated, the device enumerator may be released. It is worth noting that, in the context of the present embodiment, "release" may refer to deregistering information, reclaiming list space, or destroying a program. For example, releasing the device enumerator may refer to destroying the device enumerator program that has been called or created.

[0147] 509: Returns the device list to the application.

[0148] For example, Wine 220 can send the read path corresponding to the device list to the requested application, allowing the application to read the device information stored in the device list space. For example, Wine 220 can send the read path corresponding to the camera device list to MF application 211, allowing MF application 211 to obtain the enumerated camera information. This camera information can correspond to the camera information traversed in step 301 above.

[0149] 510: Release the device list.

[0150] For example, referring to the description of step 508 above, if Wine 220 determines that the enumerated camera information cannot be copied to the device list during step 506, the device list may be released, i.e., the storage space provided by the device list may be reclaimed. In this embodiment of the present application, "releasing" a device list may refer to the process of clearing the camera information stored in the device list to reclaim space in the device list.

[0151] 511: Release the device enumerator.

[0152] The execution process of releasing the device enumerator may refer to the relevant description in the above step 508 and will not be described in detail here.

[0153] 512: Returns an error value.

[0154] For example, after executing the above steps to release the device list and the device enumerator, Wine 220 may return an error value to the application requesting to open the camera, wherein the application may be, for example, MF application 211, and the returned error value may be, for example, "NULL".

[0155] 3, after Wine 220 obtains camera information through the device enumerator, it can continue to execute step 302 and return the obtained camera information to the application. The return process can be described in detail in step 509, which will not be repeated here.

[0156] When the MF application 211 receives the camera information returned by Wine 220, it can select a camera that is compatible with the video function to be provided as the target camera. At this time, Wine 220 can continue to set the video source acquisition node according to the selection of the application.

[0157] Specifically, the specific implementation process of setting the video source acquisition node to the pipeline of the camera source 226 corresponding to the above steps 303 to 305 is described in detail below with reference to FIG. 6 .

[0158] FIG6 shows a schematic diagram of an implementation process of Wine 220 setting a video source acquisition node on a camera source pipeline according to an embodiment of the present application.

[0159] It can be understood that when the Windows application running on the electronic device 100 selects the camera device to be used from the multiple camera information traversed and obtained by Wine 220, Wine 220 can set a video source acquisition node on the pipeline of the camera source 226 according to the camera device selected by the application and the properties adapted to the corresponding settings of the application. The video source acquisition node can obtain the video data stream from the camera selected by the application as the video source, so that the application can smoothly display the corresponding video screen. Taking the conference application as an example, the MF application 211 can select the front camera of the electronic device 100 when providing the video function. Correspondingly, Wine 220 can parse the camera device path and call the camera pipeline to complete the setting of the video source acquisition node.

[0160] Specifically, as shown in FIG6 , the implementation process may include the following steps:

[0161] 601: Get the attribute information corresponding to the device selected by the application.

[0162] For example, in response to MF application 211's request to open a camera, Wine 220, based on the device enumerator, traverses the camera information of electronic device 100 and provides it to MF application 211. MF application 211 can then select one or more cameras based on its needs. At this point, MF application 211 can provide identification information such as the ID and name of the selected camera device as attribute information to Wine 220. Correspondingly, Wine 220 can receive the attribute information fed back by MF application 211.

[0163] 602: Obtain the device identifier included in the device attribute information.

[0164] For example, after Wine 220 receives the attribute information corresponding to the camera device selected by MF application 211, it can parse the device identifier of the camera device from the information, such as the ID, name, model, etc. of the camera device, which can be used to identify the corresponding device.

[0165] 603: Resolve the device path based on the device identifier.

[0166] For example, the device path can be the file access path corresponding to the camera device selected by MF application 211. For example, " / dev / video0" and " / dev / video1" can correspond to device paths for different cameras. Based on the device identifier obtained in the above steps, Wine 220 can find the file access path corresponding to the target camera and use it as the parsed device path.

[0167] It is understood that the parsed device path can support the media resource 225 of Wine 220 to access the driver file under the path and open the target camera. The driver file under the device path can be loaded by, for example, the video device kernel driver corresponding to the target camera in v4l2.lib provided by the Linux kernel to drive the target camera to start working. For details, please refer to the relevant description below and will not be repeated here.

[0168] 604: Call the pipeline initialization interface to construct an initialized camera source pipeline.

[0169] For example, the media resource 225 in Wine 220 can call a pipeline initialization interface to connect one or more function nodes to be added to construct an initialization pipeline capable of transmitting data. The function node at one end of the initialization pipeline can receive data input, and the function node at the other end can output data processed by each function node in the pipeline. For example, the video source acquisition node connected to the initialization pipeline can receive data stream input from a camera. At the output end of the pipeline, this node or other function nodes can output the received or processed data stream to the MF application 211 that requested to open the camera.

[0170] It is understood that the initialized camera source pipeline may include one or more functional nodes, and the specific functional nodes added may be customized according to the requirements of the MF application 211. For example, in some embodiments, the initialized camera source pipeline may include only a video source acquisition node. In other embodiments, the initialized camera source pipeline may include a video source acquisition node and a format conversion node, etc., which are not limited here.

[0171] In other embodiments, the initialization pipeline constructed above may also include the open local file node and the cropping node shown in FIG. 2 b above, or include a desktop screen recording node, etc., which is not limited here.

[0172] It is understood that after Wine 220 completes adding the video source acquisition node to the constructed camera source pipeline by executing the steps shown in FIG6 , it can also continue to execute the process shown in FIG7 below to continue adding other functional nodes such as the format conversion node required by the MF application 211 to the pipeline. The specific addition process will be described in detail below and is not detailed here.

[0173] 605: Initialize the video source acquisition node on the camera source pipeline.

[0174] For example, after Wine 220 executes the above steps to construct an initialized camera source pipeline, it has already completed the initial addition of some functional nodes, such as the video source acquisition node. At this point, Wine 220 can initialize the functions of the video source acquisition node and other functional nodes in the pipeline, including initializing the interface parameters for the corresponding data stream provided by the camera serving as the video source. After completing the initialization of each functional node, the camera source pipeline can successfully receive data.

[0175] In other embodiments, if the initialized camera source pipeline includes an open file node or a screen recording node, Wine 220 may initialize the open file node or the screen recording node by executing step 605, without limitation.

[0176] 606: Encapsulate the initialized camera source pipeline and construct an output end.

[0177] For example, after Wine 220 completes initialization of the camera source pipeline and each functional node in the pipeline by executing steps 604 to 605, it can execute the relevant code for encapsulating the camera source pipeline and construct an output terminal for the pipeline to output the processed data stream. For example, the output terminal can refer to the "output ports" shown in Figure 2b above.

[0178] 607: Return to the video source.

[0179] For example, Wine 220 can then return the media resource 225 of the corresponding ready camera source pipeline to the MF application 211. It will be appreciated that the camera source pipeline constructed above can be used to implement the functions corresponding to the camera source 226 described above. In this embodiment of the present application, the camera source 226 can be a resource subclass provided by the media resource 225. In other embodiments, the camera source can also be used as another media resource to accept calls from the media session 222 or source reader 223, etc., to provide a camera data stream based on physical camera acquisition.

[0180] Referring to FIG. 3 , after Wine 220 adds a video source acquisition node to the corresponding pipeline, it can open the camera and acquire the camera data stream based on the added video source acquisition node. If the video source data format configured by MF application 211 for the corresponding function differs from the video format supported by the target camera's acquired data stream, Wine 220 can continue executing step 306 to add other functional nodes, such as a format conversion node, to the pipeline corresponding to camera source 226.

[0181] Specifically, the specific implementation process of adding a format conversion node to the camera source pipeline corresponding to the above step 306 is described in detail below with reference to FIG. 7 .

[0182] FIG7 shows a schematic diagram of an implementation process of Wine 220 adding a format conversion node to a camera source pipeline according to an embodiment of the present application.

[0183] It is understandable that when a Windows application running on the electronic device 100, such as the MF application 211, provides a video function, the format of the corresponding image or video data stream may not be supported by the physical camera of the electronic device 100. For example, if the video format of the video function supported by the MF application 211 is YUV420, and the format supported by the physical camera of the electronic device 100 is YUV422, the video data collected by the physical camera needs to be format converted before being provided to the MF application 211. This allows the video content collected by the camera to be smoothly displayed in the conference interface of the MF application 211, avoiding garbled video images displayed on the interface.

[0184] Specifically, as shown in FIG7 , the implementation process may include the following steps:

[0185] 701: Returns the list of video formats supported by the target camera to the application.

[0186] Exemplarily, after Wine 220 completes adding the video source acquisition node and initializing the constructed camera source pipeline and function nodes based on the target camera selected by the MF application 211, it can provide the format of the video data stream supported by the target camera to the MF application 211. For example, Wine 220 can return the video format information supported by the target camera to the MF application 211 in the form of a video format list. As an example, the list of video formats supported by the target camera can include one or more of YUV420, YUV422, YUV444, YUV411, YUV410, etc., without limitation. Correspondingly, after receiving the video format list returned by Wine 220, the MF application 211 can determine whether these video formats are adapted to its own functional requirements based on its own needs.

[0187] In other embodiments, Wine 220 may also return hardware configuration information such as the frame rate of the target camera capturing the video data stream to the MF application 211, which is not limited here.

[0188] 702: Read the video format set by the application.

[0189] For example, after receiving the video format list returned by Wine 220, MF application 211 can determine whether the video formats supported by the target camera match the video format required by its own function. If so, MF application 211 can select a video format from the received video format list and feedback it to Wine 220 as the set video format. If not, MF application 211 can set a video format based on its own functional requirements and feedback it to Wine 220 as the set video format.

[0190] 703: Determine whether the target camera supports the video format set by the application.

[0191] If the determination result is yes, that is, the target camera supports the video format set by the application, Wine 220 may execute the following step 708 to determine whether the camera pipeline is ready.

[0192] If the determination result is no, that is, the target camera does not support the video format set by the application, Wine 220 may continue to execute the following steps 704 to 707 to add a format conversion node to the camera source pipeline.

[0193] Exemplarily, Wine 220 may compare the video format set by the MF application 211 with various video formats supported by the target camera to determine whether the target camera has a matching video format, ie, the above-mentioned determination of whether the target camera supports the video format set by the application.

[0194] It is understood that the video format list returned by Wine 220 to the application (e.g., MF application 211) in step 701 may not include all video formats supported by the target camera. That is, the video format list may contain omissions of corresponding video formats. The determination process performed by Wine 220 in this step can compensate for any omissions in the video format list and determine whether the target camera supports the video format set by the application.

[0195] 704: Pause the data stream transmission process in the camera source pipeline.

[0196] For example, when Wine 220 determines that the target camera does not support the format set by MF application 211 during step 703, it can suspend the transmission of the camera data stream in the pipeline. Wine 220 can suspend the transmission of the data stream in the pipeline by, for example, shutting down the video source acquisition node and temporarily not receiving the data stream collected from the camera.

[0197] 705: Add a format conversion node to the camera source pipeline.

[0198] For example, after Wine 220 pauses data stream transmission in the pipeline, it can add a preset format conversion node to the camera source pipeline. It is understood that the added format conversion node can receive the camera data stream output by the video source acquisition node, perform format conversion, and then output the processed camera data stream to the media session 222 or source reader 223 through the output of the camera source. Ultimately, Wine 220 provides it to the MF application 211.

[0199] 706: Based on the video formats supported by the camera and the formats set by the application, the video formats before and after conversion are determined through negotiation.

[0200] For example, Wine 220 can obtain all video formats supported by the target camera through the video source acquisition node, match them with the formats supported by the format conversion node, and then determine a video format supported by the target camera as the pre-conversion video format and notify the video source acquisition node. In this way, the video source acquisition node can set the target camera to use this video format to capture the data stream.

[0201] Furthermore, Wine 220 may determine the format set by MF application 211 as the target format after conversion. If the formats supported by the format conversion node do not include the video format set by MF application 211, Wine 220 may also request MF application 211 to reset the video format supported by the format conversion node as the target format after conversion. This is not a limitation.

[0202] It is understandable that Wine 220 can perform the process of negotiating the video format in this step through the media resource 225, and this is not limited here.

[0203] 707: Restart the camera source pipeline.

[0204] For example, Wine 220 may initiate a request to obtain the camera data stream through the media session 222, triggering a restart of the camera source pipeline, so that the pipeline resumes transmitting the camera data stream.

[0205] 708: Determine if the camera source pipeline is ready.

[0206] For example, after restarting the camera source pipeline, Wine 220 can determine that the pipeline corresponding to the camera source 226 in the media resource 225 is ready. At this time, Wine 220 can perform the following step 709 to receive the request from the MF application 211 to open the camera, obtain the camera data stream, etc.

[0207] 709: Receive the application's request to read the camera data stream.

[0208] For example, Wine 220 may receive a request from MF application 211 to read the camera data stream in response to MF application 211's request to enable the video function. Wine 220 may then transmit the camera data stream based on the prepared camera source pipeline.

[0209] 710: Copy the data stream at the output end of the camera source pipeline.

[0210] 711: Returns the data stream to the application.

[0211] For example, the source reader 223 in Wine 220 can copy the camera data stream transmitted by the camera source pipeline from the output end of the pipeline, and then return the copied camera data stream to the MF application 211.

[0212] In other embodiments, Wine 220 may also copy the camera data stream transmitted by the camera source pipeline from the output end of the pipeline through the media session 222, and return the copied camera data stream to other required applications, which is not limited here.

[0213] It is understood that the above data stream may include multiple image frames collected and processed in sequence. Therefore, the process of returning the data stream may be, for example, a process of returning processed image data frame by frame, which is not limited here.

[0214] FIG8 shows a schematic structural diagram of an electronic device 100 according to an embodiment of the present application.

[0215] As shown in FIG8 , the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, and a temperature sensor 180H.

[0216] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0217] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0218] The controller can generate an operation control signal based on the instruction opcode and timing signal to complete the control of instruction fetching and execution. In an embodiment of the present application, the processor 110 can control Wine 220 through the controller to execute each step of the implementation process corresponding to the above-mentioned media resource control method provided in this application, thereby enabling the MF application to be compatible with operating systems such as Linux and also realize the camera and video opening function. It can also support the opening of local files or shared desktops quickly and stably through the local file opening node and desktop screen recording node in the pipeline, without problems such as size mismatch and frame rate mismatch.

[0219] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0220] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a universal serial bus (USB) interface.

[0221] The USB interface 130 is an interface that complies with USB standards, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transfer data between the electronic device 100 and peripheral devices.

[0222] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0223] The charging management module 140 is configured to receive charging input from a charger. While charging the battery 142 , the charging management module 140 can also provide power to the electronic device through the power management module 141 .

[0224] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160.

[0225] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0226] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. applied to the electronic device 100.

[0227] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0228] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0229] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0230] The electronic device 100 can implement a shooting function through the ISP, camera 193, video codec, GPU, display screen 194, and application processor. In the embodiment of the present application, the shooting function implemented by the electronic device 100 through the above-mentioned ISP, camera 193 and other devices can support Wine 220 to execute the media resource control method provided in the present application, and respond to the MF application 211 to turn on the video function of the camera.

[0231] The ISP is used to process data fed back by the camera 193 .

[0232] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0233] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0234] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0235] The internal memory 121 can be used to store computer executable program code, which includes instructions. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.), etc.

[0236] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0237] The audio module 170 is used to convert digital audio information into analog audio signals for output, and is also used to convert analog audio input into digital audio signals.

[0238] Keys 190 include a power button, a volume button, and the like. Keys 190 may be mechanical keys or touch-sensitive keys. Electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of electronic device 100.

[0239] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0240] The embodiments of the present application also provide a computer program product for implementing the media resource control methods provided in the above embodiments.

[0241] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as computer program modules or module codes executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0242] A computer program module or module code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0243] Module code can be implemented with high-level modular language or object-oriented programming language to communicate with the processing system. When necessary, module code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0244] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to a floppy disk, an optical disk, an optical disk, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0245] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technique disclosed according to the embodiment of the present application. The appearances of the phrase "in one embodiment" in various places in the specification do not necessarily all refer to the same embodiment.

[0246] The disclosure of the embodiments of the present application also relates to an operating device for executing the text. The device can be constructed specifically for the required purpose or it can include a general-purpose computer that is selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer-readable medium, such as, but not limited to, any type of disk, including a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic or optical card, an application-specific integrated circuit (ASIC) or any type of medium suitable for storing electronic instructions, and can be coupled to a computer system bus. In addition, the computer mentioned in the specification can include a single processor or can be an architecture involving multiple processors for increased computing power.

[0247] In addition, the language used in this specification has been primarily selected for readability and instructional purposes and may not be selected to describe or limit the disclosed subject matter. Therefore, the present disclosure of embodiments is intended to illustrate, not to limit, the scope of the concepts discussed herein.

Claims

1. A media resource control method, applied to an electronic device, characterized in that: The electronic device is installed with a first operating system and a first application, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and The method comprises: detecting a first request by the first application to open a first media device; In response to the first request, a first calling instruction is transmitted to a first driver of the kernel layer of the first operating system through a first transmission interface, wherein: The first transmission interface is developed based on a first operating system, and the first calling instruction is used to instruct the first driver to open the first media device requested by the first request; The first media resource collected by the first media device is provided to the first application through the first transmission interface.

2. The method according to claim 1, characterized in that The first operating system includes an MF framework running in a compatibility layer, the MF framework is provided by the second operating system, and the MF framework is provided by the second operating system, and The MF framework includes multiple transmission interfaces, and the multiple transmission interfaces include the first transmission interface.

3. The method according to claim 1 or 2, characterized in that: The detecting a first request by the first application to open a first media device comprises: Receiving a user operation on a video control on a first interface, wherein the first interface is an interface displayed corresponding to the electronic device running the first application; A first request for opening a first media device triggered by the user operation is detected.

4. The method according to claim 2, characterized in that: Providing the first media resource collected by the first media device to the first application through the first transmission interface includes: Recognizing that a format of a media resource set by the first application does not match a format of a first media resource collected by the first media device; receiving, through a second transmission interface in the MF framework, a first media resource transmitted through the first transmission interface, wherein the second transmission interface is an interface having a format conversion function; The first media resource is format-converted through the second transmission interface to obtain a second media resource, and the second media resource is provided to the first application through the second transmission interface.

5. The method according to claim 4, characterized in that The output end of the first transmission interface in the MF framework is connected to the input end of the second transmission interface to form a data transmission channel for transmitting the first media resource and the second media resource.

6. The method according to claim 2, characterized in that The MF framework further includes a resource reading module, the first media device is a target device selected from a plurality of media devices acquired by the resource reading module in the MF framework, and The method of acquiring multiple media devices through the resource reading module of the MF framework includes: Accessing the dynamic link library of the first operating system through the resource reading module to obtain a first dynamic link library file; The resource reading module reads a calling interface corresponding to the first device enumerator in the first dynamic link library file, and the first device enumerator is called based on the calling interface to obtain device information of the plurality of media devices.

7. The method according to claim 6, characterized in that The method of acquiring multiple media devices through the resource reading module of the MF framework also includes: The resource reading module reads configuration information instructing to create a second device enumerator in the first dynamic link library file, and creates the second device enumerator based on the configuration information to obtain device information of the plurality of media devices.

8. The method according to claim 6 or 7, characterized in that: The device information of the plurality of media devices includes at least one of the following: an identifier of the media device; the name of the media device; The model of the media device.

9. The method according to claim 2, characterized in that: The method comprises: detecting a second request by the first application to open a locally stored first media file; In response to the second request, transmitting a second call instruction to a second driver of the kernel layer of the first operating system through a third transmission interface in the MF framework, wherein the second call instruction is used to instruct the second driver to open a second media device requested by the second request; The content read by the second media device by calling the storage path of the first media file is provided to the first application through the third transmission interface, wherein the second media device is a virtual device created by virtualization technology.

10. The method according to claim 9, characterized in that The providing the content read by the second media device through the third transmission interface by calling the storage path of the first media file to the first application includes: Recognizing that the screen size required for the second media device to read the content by calling the storage path of the first media file does not match the screen size corresponding to the display of the second interface of the first application; receiving, through a fourth transmission interface in the MF framework, content read by calling the storage path of the first media file transmitted through the third transmission interface, wherein the fourth transmission interface is an interface with a cropping function; The content of the first media file is cropped to correspond to the required screen size through the fourth transmission interface, and the cropped content of the second media file is provided to the first application through the fourth transmission interface.

11. The method according to claim 2, characterized in that The method comprises: detecting a third request for the first application to share desktop display content; In response to the third request, transmitting a third calling instruction to a third driver of the kernel layer of the first operating system through a fifth transmission interface in the MF framework, wherein the third calling instruction instructs the third driver to open a second media device to record desktop display content; The third media resource generated corresponding to the desktop display content recorded by the second media device is provided to the first application through the fifth transmission interface.

12. The method according to any one of claims 1 to 8, characterized in that The first media device includes a camera device.

13. The method according to any one of claims 9 to 11, characterized in that The second media device includes a virtual camera.

14. An electronic device, characterized in that: The electronic device is installed with a first operating system and a first application, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and The electronic device includes a first transmission interface developed based on the first operating system, wherein: The first transmission interface is used to send a first call instruction to a first driver of the kernel layer of the first operating system, wherein the first call instruction is used to respond to a first request of a first application to open a first media device and instruct the first driver to open the first media device requested by the first request; and The first transmission interface is used to provide the first media resource collected by the first media device to the first application.

15. The electronic device according to claim 14, characterized in that: The first operating system includes an MF framework running in a compatibility layer, the MF framework is provided by the second operating system, and, The MF framework includes multiple transmission interfaces, and the multiple transmission interfaces include the first transmission interface.

16. The electronic device according to claim 15, characterized in that: The plurality of transmission interfaces include a second transmission interface, wherein the second transmission interface is an interface having a format conversion function, and, The second transmission interface is used to receive a first media resource transmitted through the first transmission interface, and, The second transmission interface is used to convert the format of the first media resource to obtain a second media resource, and provide the second media resource to the first application.

17. The electronic device according to claim 15, characterized in that: The plurality of transmission interfaces include a third transmission interface, wherein: The third transmission interface is used to transmit a second calling instruction to a second driver of the kernel layer of the first operating system, wherein the second calling instruction is used to respond to a second request of the first application to open a first media file stored locally, and instruct the second driver to open a second media device requested by the second request; and The third transmission interface is used to provide the content read by the second media device by calling the storage path of the first media file to the first application, wherein the second media device is a virtual device created by virtualization technology.

18. The electronic device according to claim 17, characterized in that: The plurality of transmission interfaces include a fourth transmission interface, wherein the fourth transmission interface is an interface having a clipping function, and, The fourth transmission interface is used to receive the content read by calling the storage path of the first media file transmitted through the third transmission interface, and, The fourth transmission interface is used to crop the content of the first media file to correspond to the required screen size, and provide the cropped content of the second media file to the first application.

19. The electronic device according to claim 15, characterized in that: The plurality of transmission interfaces include a fifth transmission interface, wherein: The fifth transmission interface is used to transmit a third calling instruction to a third driver of the kernel layer of the first operating system, wherein the third calling instruction is used to respond to a third request of the first application to share the desktop display content and instruct the third driver to open the second media device to record the desktop display content; and The fifth transmission interface is used to provide the third media resource generated corresponding to the desktop display content recorded by the second media device to the first application.

20. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the media resource control method described in any one of claims 1 to 13.

21. A computer readable medium, characterized in that The readable medium stores instructions, which, when executed on a computer, enable the computer to execute the media resource control method according to any one of claims 1 to 13.

Images