Media resource control method, electronic equipment and computer readable medium

Abstract

The invention relates to the technical field of computers, in particular to a media resource control method, electronic equipment and a computer readable medium. According to the method, a kernel driver of a kernel layer of an operating system such as Linux is called to open media equipment in the electronic equipment through an interface which is added in the operating system such as Linux and used for transmitting calling instructions and data, and media resources collected by the media equipment can be provided for a required Windows application program through the interface. Wherein the interface for transmitting the calling instruction and the data can be a function node preset in an operating system such as Linux or other forms. Based on the method, a Windows application program can call kernel structures such as a kernel driver of video equipment in a kernel layer of an operating system such as Linux, so that some Windows application programs can realize a video function and the like, and the compatibility of the Windows application programs is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a media resource control method, electronic equipment and computer-readable medium. Background Art

[0002] With the development of computer technology, in order to reduce the amount of development, applications developed for one operating system can be compatible and run on other operating systems through API conversion, creating containers or creating virtual machines. For example, Windows applications developed for Windows systems can be run on operating systems such as Linux, macOS and Free BSD through Wine (windows environment). Taking the Linux system as an example, Wine can use API conversion technology to make Linux functions corresponding to Windows, so that DLL (a dynamic link library file format) can be called to run Windows applications. It can be used as a compatibility layer for running Windows applications in the Linux system.

[0003] However, currently, Windows applications developed using the media foundation (MF) framework provided by the Windows operating system cannot obtain camera information and open the camera through the kernel driver of the Linux system based on the MF framework, wherein the MF framework is usually run in Wine of the Linux system. Therefore, such Windows applications are usually unable to obtain the data stream collected by the camera when a video session is required, which ultimately makes the video function of such applications unavailable. Summary of the invention

[0004] The present application provides a media resource control method, an electronic device and a computer-readable medium, which realizes calling kernel structures such as video device kernel drivers in the kernel layer through an interface provided by the MF framework that can transmit call instructions and data between the kernel layer of the operating system, thereby enabling some Windows applications developed based on the MF framework to realize video functions, etc., thereby improving the compatibility of such Windows applications and the MF framework.

[0005] In a first aspect, the present application provides a media resource control method, which is applied to an electronic device, wherein the electronic device is installed with a first operating system and a first application, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and the method includes: detecting a first request from the first application to open a first media device; in response to the first request, transmitting a first call instruction to a first driver of a kernel layer of the first operating system through a first transmission interface, wherein the first transmission interface is developed based on the first operating system, and the first call instruction is used to instruct the first driver to open the first media device requested by the first request; and providing the first media resources collected by the first media device to the first application through the first transmission interface.

[0006] For example, the first operating system is a Linux operating system, the second operating system is a Windows operating system, and the first application is, for example, a Windows application developed based on the Windows operating system. The media resource control method provided in the first aspect, that is, a Windows application (such as the first application) on a Linux operating system installed on an electronic device, can call the kernel driver (such as the first driver) of the kernel layer of the first operating system through an interface (such as the first transmission interface) added in the first operating system for transmitting call instructions and data to open a media device (such as the first media device) in the electronic device, and can provide the media resources (such as the first media resources) collected by the media device to the Windows application on demand through the interface. Among them, the interface for transmitting call instructions and data can be a function node preset in the first operating system, and the corresponding first transmission interface can be, for example, a video source acquisition node, and the first media device can be, for example, a camera device corresponding to the video source acquisition node.

[0007] Based on the media resource control method provided in the first aspect above, it is possible to implement Windows applications to call kernel structures such as video device kernel drivers in the kernel layer of operating systems such as Linux, thereby enabling some Windows applications to implement video functions, etc., thereby improving the compatibility of such Windows applications.

[0008] In a possible implementation of the first aspect, the first operating system includes an MF framework running in a compatibility layer, the MF framework is provided for the second operating system, and the MF framework includes multiple transmission interfaces, including the first transmission interface.

[0009] That is, the interface for transmitting call instructions and data added in the first operating system (e.g., the first transmission interface) can be set in the corresponding structure of the MF framework running in the compatibility layer of the first operating system (e.g., the Linux operating system). Correspondingly, the first application can be a Windows application developed based on the MF framework, referred to as an MF application. This is conducive to improving the compatibility of the MF framework running in non-Windows operating systems.

[0010] In a possible implementation of the first aspect above, detecting a first request for a first application to open a first media device includes: receiving a user operation of a video control on a first interface, wherein the first interface is an interface displayed corresponding to the electronic device running the first application; and detecting the first request for opening the first media device triggered by the user operation.

[0011] In a possible implementation of the first aspect above, providing a first media resource collected by a first media device to a first application through a first transmission interface includes: identifying that a media resource format set by the first application does not match a format of the first media resource collected by the first media device; receiving the first media resource transmitted through the first transmission interface through a second transmission interface in the MF framework, wherein the second transmission interface is an interface with a format conversion function; performing format conversion on the first media resource through the second transmission interface to obtain a second media resource, and providing the second media resource to the first application through the second transmission interface.

[0012] For example, the second transmission interface with format conversion function can be a format conversion node. When the function node is added to the pipeline corresponding to the camera source, the media resource (e.g., the first media resource) collected by the media device (e.g., the first media device) on the electronic device can be converted into a format and then provided to the required MF application. The media resource collected by the first media device can be, for example, a video data stream, and the corresponding format conversion implemented by the format conversion node can be to convert the video data stream from the original video format supported by the camera to the video format supported by the required MF application.

[0013] In a possible implementation of the first aspect above, an output end of the first transmission interface in the MF framework is connected to an input end of the second transmission interface to form a data transmission channel for transmitting the first media resource and the second media resource.

[0014] For example, the data transmission channel formed by connecting the first transmission interface and the second transmission interface may be a camera source pipeline formed by connecting corresponding preset function nodes in the media resources of the MF framework.

[0015] In a possible implementation of the first aspect above, the MF framework also includes a resource reading module, the first media device is a target device selected from multiple media devices obtained by the resource reading module in the MF framework, and the method of obtaining multiple media devices through the resource reading module of the MF framework includes: accessing the dynamic link library of the first operating system through the resource reading module to obtain the first dynamic link library file; reading the calling interface corresponding to the first device enumerator in the first dynamic link library file through the resource reading module, and calling the first device enumerator based on the calling interface to obtain device information of multiple media devices.

[0016] In a possible implementation of the first aspect above, the method of obtaining multiple media devices through the resource reading module of the MF framework also includes: reading configuration information indicating the creation of a second device enumerator in the first dynamic link library file through the resource reading module, and creating a second device enumerator based on the configuration information to obtain device information of multiple media devices.

[0017] In a possible implementation of the first aspect above, the device information of the plurality of media devices includes at least one of the following: an identity identifier of the media device; a name of the media device; and a model of the media device.

[0018] In a possible implementation of the first aspect above, the method includes: detecting a second request from a first application to open a locally stored first media file; in response to the second request, transmitting a second call instruction to a second driver of a kernel layer of the first operating system through a third transmission interface in the MF framework, wherein the second call instruction is used to instruct the second driver to open a second media device requested by the second request; and providing the content read by the second media device from the storage path of the first media file to the first application through the third transmission interface, wherein the second media device is a virtual device created by virtualization technology.

[0019] For example, the third transmission interface for transmitting the second call instruction may be an open local file node. When this function node is added to the pipeline corresponding to the camera source, it may support calling the video device kernel driver in the kernel layer of the Linux operating system to open a virtual camera, such as the second media device, and further support the corresponding MF application to open a local file. When the MF application is, for example, a conference application, the open local file node supports the function of opening a local file, which may support the operation of users sharing files with other participants.

[0020] In a possible implementation of the first aspect above, providing the content read by the second media device by calling the storage path of the first media file to the first application through the third transmission interface includes: identifying that the screen size required for the content read by the second media device by calling the storage path of the first media file does not match the screen size corresponding to the display of the second interface of the first application; receiving the content read by calling the storage path of the first media file transmitted through the third transmission interface through the fourth transmission interface in the MF framework, wherein the fourth transmission interface is an interface with a cropping function; cropping the content of the first media file corresponding to the required screen size through the fourth transmission interface, and providing the cropped content of the second media file to the first application through the fourth transmission interface.

[0021] For example, the fourth transmission interface with the cropping function may be a cropping node. When the function node is added to the pipeline corresponding to the camera source, the size of the corresponding media file (e.g., the first media file) may be cropped to meet the requirement of the MF application to display the corresponding content of the media file in the corresponding window. The first media file may be, for example, an image or a video file, etc., which is not limited here.

[0022] In a possible implementation of the first aspect above, the method includes: detecting a third request for the first application to share desktop display content; in response to the third request, transmitting a third call instruction to a third driver of the kernel layer of the first operating system through the fifth transmission interface in the MF framework, wherein the third call instruction instructs the third driver to open a second media device to record the desktop display content; and providing the third media resources generated corresponding to the desktop display content recorded by the second media device to the first application through the fifth transmission interface.

[0023] For example, the fifth transmission interface for transmitting the third call instruction may be a desktop screen recording node. When this function node is added to the pipeline corresponding to the camera source, it can support the MF application to provide a shared desktop function, such as providing the desktop display content recorded by opening the virtual camera (i.e., the second media device) through the desktop screen recording node to the MF application, which will not be elaborated here.

[0024] In a possible implementation of the first aspect, the first media device includes a camera device.

[0025] In a possible implementation of the first aspect, the second media device includes a virtual camera.

[0026] In a second aspect, the present application provides an electronic device having a first operating system and a first application installed thereon, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and the electronic device includes a first transmission interface developed based on the first operating system, wherein the first transmission interface is used to send a first call instruction to a first driver of a kernel layer of the first operating system, wherein the first call instruction is used to respond to a first request from the first application to open a first media device, and instruct the first driver to open the first media device requested by the first request; and the first transmission interface is used to provide a first media resource collected by the first media device to the first application.

[0027] In a possible implementation of the second aspect, the first operating system includes an MF framework running in a compatibility layer, the MF framework is provided for the second operating system, and the MF framework includes multiple transmission interfaces, and the multiple transmission interfaces include the first transmission interface.

[0028] In a possible implementation of the second aspect above, the multiple transmission interfaces include a second transmission interface, wherein the second transmission interface is an interface with a format conversion function, and the second transmission interface is used to receive a first media resource transmitted through the first transmission interface, and the second transmission interface is used to perform format conversion on the first media resource to obtain a second media resource, and provide the second media resource to the first application.

[0029] In a possible implementation of the second aspect above, the multiple transmission interfaces include a third transmission interface, wherein the third transmission interface is used to transmit a second call instruction to a second driver of the kernel layer of the first operating system, wherein the second call instruction is used to respond to a second request of the first application to open a first media file stored locally, and instruct the second driver to open a second media device requested by the second request; and the third transmission interface is used to provide the content read by the second media device calling the storage path of the first media file to the first application, wherein the second media device is a virtual device created by virtualization technology.

[0030] In a possible implementation of the second aspect above, the multiple transmission interfaces include a fourth transmission interface, wherein the fourth transmission interface is an interface with a cropping function, and the fourth transmission interface is used to receive content read by calling the storage path of the first media file transmitted through the third transmission interface, and the fourth transmission interface is used to crop the content of the first media file to a required screen size, and provide the cropped content of the second media file to the first application.

[0031] In a possible implementation of the second aspect above, multiple transmission interfaces include a fifth transmission interface, wherein the fifth transmission interface is used to transmit a third call instruction to a third driver of the kernel layer of the first operating system, wherein the third call instruction is used to respond to a third request from the first application to share desktop display content, and instruct the third driver to open a second media device to record the desktop display content; and the fifth transmission interface is used to provide a third media resource generated corresponding to the desktop display content recorded by the second media device to the first application.

[0032] In a third aspect, the present application provides an electronic device comprising: one or more processors; one or more memories; one or more memories storing one or more programs, wherein when one or more programs are executed by one or more processors, the electronic device executes the media resource control method provided in the above-mentioned first aspect and various possible implementations of the first aspect.

[0033] In a fourth aspect, the present application provides a computer-readable medium having instructions stored thereon, which, when executed on a computer, causes the computer to execute the media resource control method provided in the first aspect and various possible implementations of the first aspect.

[0034] The beneficial effects of the second to fourth aspects mentioned above can be referred to the relevant descriptions in the first aspect mentioned above and various possible implementations of the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1a The figure shows an interface diagram of a Windows application that cannot obtain the data stream collected by the camera during operation.

[0036] Figure 1b Shown is a schematic diagram of the Linux operating system architecture.

[0037] Figure 2a Shown is a schematic diagram of a Linux system architecture provided in an embodiment of the present application.

[0038] Figure 2b The figure is a schematic diagram of a process for setting functional nodes on a camera source pipeline based on application requirements provided by an embodiment of the present application.

[0039] Figure 2c The figure shows a conference interface diagram of a MF application 211 provided in an embodiment of the present application.

[0040] Figure 3 The figure is a schematic diagram of an implementation process of a media resource control method provided in an embodiment of the present application.

[0041] Figure 4aThe figure shows a schematic diagram of the change of the conference interface displayed corresponding to the format conversion process before and after the collected camera data stream is processed when the video function is turned on by an MF application 211 provided in an embodiment of the present application.

[0042] Figure 4b The figure shows a conference interface diagram corresponding to the opening of a local file by the MF application 211 provided in an embodiment of the present application.

[0043] Figure 4c The figure shows a schematic diagram of a conference interface displayed when the MF application 211 provided in an embodiment of the present application shares a desktop.

[0044] Figure 5 The figure is a schematic diagram of an implementation process of Wine traversing camera information of an electronic device provided in an embodiment of the present application.

[0045] Figure 6 The figure is a schematic diagram of an implementation process of Wine setting a video source acquisition node on a camera source pipeline provided by an embodiment of the present application.

[0046] Figure 7 The figure is a schematic diagram of an implementation process of adding a format conversion node to a camera source pipeline in Wine according to an embodiment of the present application.

[0047] Figure 8 Shown is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0049] Figure 1a The present invention shows an interface schematic diagram in which a data stream collected by a camera cannot be obtained during the operation of a Windows application.

[0050] like Figure 1a As shown, an electronic device 100 equipped with a Linux operating system can display a conference interface 01 when running some conference applications. When a user clicks a video control 02 on the conference interface 01 to conduct a video conference, the conference application needs to turn on the camera to obtain a data stream captured by the camera. It can be understood that a stream is a sequence of media data of a uniform type. The aforementioned data stream can be, for example, a sequence of image data captured by a camera, which can also be referred to as a video stream in some embodiments.

[0051] Figure 1b A schematic diagram of an existing operating system architecture is shown. Figure 1bAs shown, the application 101 running in the application layer of the operating system may be a conference application or an office application, etc. The application is a Windows application developed through the MF framework provided by the Windows operating system.

[0052] Taking the Linux operating system as an example, Figure 1b The compatibility layer of the operating system shown is compatible with the MF framework 102 running the Windows operating system. The MF framework 102 can support accessing the media resource 106 to obtain media source data through a source reader 104. The media source data may include local files (such as video files, etc.), videos loaded from the network, etc., and data collected by a hardware capture device (such as a camera) of an electronic device.

[0053] In addition, refer to Figure 1b As shown, the MF framework 102 of the Windows operating system may also include a media session 103 and a sink writer 105. The media session 103 is created to control the media pipeline to respond to some Windows application requests and read the media source data managed by the media source 106. In some embodiments, the media session 103 may also open a local file through a media foundation transform (MFT) 107, and may also transmit a receiving instruction to a media sink 108 to obtain data such as video loaded from the network.

[0054] The receiver writer 105 can obtain the media source data through the media receiver 108. No further details will be given here.

[0055] However, when a Windows application developed based on the MF framework such as application 101 (hereinafter referred to as MF application) needs to open the camera, Figure 1b The MF framework shown in the figure does not have an interface to access the Linux operating system kernel layer, and therefore cannot call the kernel driver and related audio and video processing framework of the kernel layer, for example, it cannot call Figure 1b The video device kernel driver 109 is shown. Therefore, when Windows applications currently developed based on the MF framework of the Windows operating system are compatible and run in other operating systems such as Linux, they cannot open the camera and therefore cannot support the video function. Figure 1a As shown, the application 101, such as the conference application, cannot open the camera, and the corresponding conference interface 01 may display a black screen or display a prompt message. The prompt message can be referred to, for example, Figure 1a It shows "Unable to obtain camera" and other content.

[0056] In order to solve the above problems, the present application provides a media resource control method. Specifically, the method adds one or more interfaces in the MF framework that can transmit call instructions and data between the kernel layer of the operating system to realize the MF framework's call to kernel structures in the kernel layer such as the video device kernel driver, and then can support some MF applications to access other operating system kernels to open cameras, so that the MF application can realize the video function. Among them, the method of adding the above interfaces in the MF framework can be implemented by presetting some function nodes in the media resources of the MF framework.

[0057] For example, in the embodiment of the present application, a video source acquisition node can be preset in the media resource of the MF framework. When the MF application turns on the video function in response to a user operation, the media resource of the MF framework can construct a pipeline for transmitting data streams between the MF framework and the video device kernel driver of the kernel layer based on the video source acquisition node, such as the following: Figure 2b The camera source pipeline shown in the example, the corresponding position of the camera source pipeline can be referred to below Figure 2a As shown, no limitation is made here. At this time, the pipeline can be used as one of the media resources to open the camera for the MF application and obtain the video data stream collected by the camera. In the embodiment of the present application, the pipeline constructed in the media resource according to the needs of the MF application can also be called a camera resource.

[0058] In this way, when a Windows application developed based on the MF framework of the Windows operating system is compatible and runs on other operating systems, it can call the kernel driver of other operating systems to open the camera and realize the video function.

[0059] It can be understood that when the format of the media source data obtained by the MF application through the MF framework request is not consistent with the format supported by the camera opened by the kernel driver, a format conversion node can also be preset in the media resource of the above MF framework. At this time, when the MF application uses the video function, the above format conversion node can be added to the pipeline constructed in the above media resource to convert the format of the video data stream collected by the camera opened by the video device kernel driver, and then provide it to the MF application. By analogy, in some embodiments, the above media resources, or the constructed camera source pipeline, can also include other functional nodes for realizing different data processing functions, such as frame rate adjustment nodes, cropping nodes, local file opening nodes, and desktop screen recording nodes, etc., which are used to further realize the format conversion, frame rate adjustment, image cropping and other processing of media source data, etc. There is no limitation here.

[0060] The various functional nodes added to the camera source pipeline corresponding to the application requirements include the format conversion node, frame rate adjustment node, and cropping node. In this way, operating systems such as Linux can also support Windows applications based on the MF framework to open image or video files of various formats, thereby improving the compatibility of Windows applications with image or video formats. In addition, the system can also adapt to Windows applications to adjust the frame rate, which is conducive to improving the quality of images or videos displayed by Windows applications.

[0061] In addition, the above processing process implemented by adding function nodes to construct Pipeline can realize the processing of the original camera data stream collected at the bottom of the operating system, thereby reducing the amount of data output by the bottom operating system to the compatibility layer. In this way, the amount of data input to the MF framework of the compatibility layer is correspondingly reduced, which is conducive to reducing the storage space and transmission channel bandwidth occupied by the data stream to be transmitted in Wine, and thus is conducive to improving the processing performance of the compatibility layer, such as improving the processing efficiency of the MF framework for media source data.

[0062] It can be understood that the video source acquisition node added to the camera source pipeline supports calling kernel drivers, which may include video device kernel drivers provided by system libraries such as v4l2.lib of the Linux kernel, such as USB video class (UVC) drivers, etc. In addition, while providing the above kernel drivers, operating systems such as Linux can also provide some audio and video processing frameworks required for processing video and image data, such as gstreamer, FFmpeg, etc., which are not limited here.

[0063] It can be understood that the media resource control method provided by the present application is applicable to electronic devices including but not limited to desktop, laptop and handheld computers, notebooks, netbooks, tablet computers, mobile phones, augmented reality (AR) / virtual reality (VR) devices, smart TVs, smart watches and other wearable devices, servers, car equipment, portable game consoles, portable music players, and other electronic devices with one or more processors and audio and video and other media file playback functions.

[0064] In some embodiments, the media resource control method provided in the present application can also be applied to other compatible layer architectures, such as deepin-wine, kylin-wine, crossover, etc., without limitation here.

[0065] Figure 2aAccording to the embodiment of the present application, a schematic diagram of a Linux system architecture is shown. It can be understood that Figure 2a The architecture shown is not intended to be a limitation of the Linux operating system architecture. In other embodiments, the Linux system architecture may also include Figure 2a More or less structures as shown.

[0066] like Figure 2a As shown, the Linux operating system 200 may include an application layer 210 , a compatibility layer 220 and a kernel layer 230 .

[0067] The application program running in the application layer 210 may include a Windows application program developed based on the MF framework of the Windows operating system, hereinafter referred to as the MF application program. Figure 2a MF application 211 is shown.

[0068] The compatibility layer 220 may be implemented by Wine, and may also be referred to as Wine 220 hereinafter. Figure 2a As shown, the MF framework 221 can be run in Wine 220, and the MF framework 221 may include a media session 222 created for obtaining media source data, and may also include a source reader 223 and a receiver writer 224. In some embodiments, when the MF application 211 needs to open the camera, the camera data stream can be obtained through the source reader 223 or the media session 222. When the MF application 211 needs to open local media files or media files such as videos loaded from the network, the corresponding media files can be read from the media resource 225 as media source data through the source reader 223. In an embodiment of the present application, the source reader 223 can obtain the processed camera data stream through the functional node added to the camera source 226 pipeline. The specific example of the camera source 226 pipeline can be found below Figure 2b In some other embodiments, the MF application 211 may also obtain the media source data such as the video received by the media receiver 228 through the receiver writer 224, which is not limited here.

[0069] It can be understood that the above-mentioned media session 222 can be created in response to the MF application 211's request to obtain media source data such as images, audio and video, and is used to provide a transmission channel for sending corresponding media source data to the MF application 211.

[0070] In some embodiments, when the MF application 211 uses the video function to request to open the camera, it can first call the system's dynamic link library file, such as mfplat.dll, through the source reader 223 or the created media session 222, to read the device enumerator interface written in the file for the system device enumerator to call, or read the configuration information instructing to create the enumerator in the file to create an enumerator for identifying the video device, etc., which will be described in detail below and will not be repeated here. The system device enumerator is used to find and create media conversion and media sources registered on the system, such as the media source such as the camera data stream obtained through the camera source 226 pipeline described below. The mfplat.dll can also be configured with attribute values ​​that need to be set when adding some function nodes to the camera source 226 pipeline, etc., which are not limited here.

[0071] In some embodiments, the MF framework 221 may also include a media basic conversion 227, which is used to uniformly record the names, acquisition paths, and other information of media source data such as images, audio and video in the system. The information recorded in the media basic conversion 227 may include media source data such as video received by the above-mentioned media receiver 228.

[0072] Continue to refer Figure 2a As shown, the kernel layer 230 may include a video device kernel driver 231. In a Linux operating system, the kernel layer 230 may be implemented based on a Linux kernel. Among them, the video device kernel driver 231 in the kernel layer 230 may include a USB video class driver, etc. In the kernel layer 230, these kernel drivers may be provided by v4l.lib or v4l2.lib.

[0073] In addition, operating systems such as Linux can also provide some audio and video processing frameworks required for processing video and image data, including gstreamer, FFmpeg, etc., which can be used to encode, decode, process and transmit the video data stream collected by the camera, etc., without limitation here.

[0074] In other embodiments, Wine 220 in the system architecture shown in 2a can also be set in other operating systems, such as macOS and Free BSD, etc., which is not limited here.

[0075] It can be understood that the camera source 226 added in the MF framework 221 can be used as a subclass of the media resource 225 to provide media source data for the media session 222 or the source reader 223. The camera source 226 can be implemented based on a pipeline, and some preset function nodes can be added to the pipeline of the camera source 226 according to the requirements of the MF application 211.

[0076] Figure 2bAccording to an embodiment of the present application, a schematic diagram of a process of setting functional nodes on a camera source pipeline based on application requirements is shown.

[0077] like Figure 2b As shown, a plurality of function nodes may be preset in the configuration file corresponding to the pipeline corresponding to the camera source 226, such as a video source acquisition node and a format conversion node for opening a physical camera, an opening local file node and a desktop screen recording node for calling a virtual camera, and a cropping node for cropping the size of the image / video screen to the screen size. The above configuration file may be, for example, a dynamic link library file such as mfplat.dll, which is not limited here. Among them, the above virtual camera may be a camera created by virtualization technology and having functions such as image acquisition similar to a physical camera. The virtual camera may be created and opened by the corresponding kernel driver in the v4l2 provided by the above kernel layer 230.

[0078] It can be understood that the process of opening local files implemented by the above-mentioned opening local file node is essentially to map local files, such as images or video files, to the created virtual camera display, and then provide them to the required MF application 211. Taking the conference application as an example, the function of opening local files supported by the above-mentioned opening local file node can support the operation of users sharing files with other participants. The process of sharing the desktop implemented by the above-mentioned desktop recording node can be to map the desktop display content to the created virtual camera recording and display, and then provide it to the required MF application 211, which will not be elaborated here.

[0079] For example, taking the conference application as an example, when the MF application 211 needs to turn on the video function during the conference, the application can pass some relevant node attributes to the camera source 226 through the source reader 223. At this time, the camera source 226 can add the preset video source acquisition node, format conversion node, etc. for opening the physical camera to the pipeline according to the node attributes passed by the MF application 211, so as to realize the purpose of calling the video device kernel driver of the kernel layer 230 through these nodes to open the camera and perform format conversion processing on the camera data stream, and finally realize the purpose of the video function of the MF application 211. Among them, the video source acquisition node for obtaining the physical camera data stream can be linked to the kernel driver corresponding to the camera device selected by the MF application 211 to turn on the camera. When the video format supported by the MF application 211 is not unified with the format supported by the camera of the electronic device 100, the format conversion node added to the pipeline corresponding to the above-mentioned camera source 226 can realize the purpose of performing format conversion processing on the video data stream obtained from the camera video source and then providing it to the MF application 211.

[0080] It can be understood that in some embodiments, other MF applications 211 can also obtain the image or video data stream captured by the camera through the media resource 225 based on the above-mentioned media session 222. However, the format of the image or video data stream obtained in this way is limited to the format supported by the camera device configured on the electronic device 100, and the frame rate of the above-mentioned image or video data stream can only be provided to the application according to the frame rate configured by the camera device, and cannot be dynamically adjusted according to the settings of the MF application 211 in the application layer 210. In contrast, the source reader 223 or the media session 222 calls the video device kernel driver 231 to open the camera through the video source acquisition node added to the camera source 226 pipeline, which can avoid the problem that the MF application 211 cannot obtain the camera. In addition, this method of opening the camera can also solve the problem that the image or video screen cannot be displayed normally when the format supported by the physical camera configured on the electronic device 100 is inconsistent with the format set by the application. The corresponding interface displayed when the specific MF application 211 provides a video function based on the media resource control method provided in this application can refer to the following Figure 2c The above and related descriptions will not be repeated here.

[0081] Continue to refer Figure 2b As shown, when the MF application 211 needs to open local media source data such as video files, the MF application 211 can pass other related node attributes to the camera source 226 through the source reader 223. At this time, the camera source 226 can add the preset local file opening node and cropping node that can call the virtual camera to the pipeline according to the node attributes passed by the MF application 211, so as to realize the function of the MF application 211 supporting the opening of local pictures or videos and other files. For another example, when the user of the MF application 211 needs to share the desktop with other participants, the MF application 211 can also pass other related node attributes to the camera source 226 through the source reader 223. At this time, the camera source 226 can add the preset desktop recording node to the pipeline, and realize the function of the MF application 211 supporting the shared desktop by recording the desktop display content. In other embodiments, the above-mentioned functional node that supports MF211 to realize the shared desktop function can also be a node in other forms, such as an access node that can access the desktop display data corresponding to the rendering process, etc., which is not limited here.

[0082] Figure 2c According to an embodiment of the present application, a schematic diagram of a conference interface of the MF application 211 is shown.

[0083] like Figure 2c As shown, based on the media resource control method provided by the present application, the operating system installed on the electronic device 100 can support the video function of the MF application 211 based on the MF framework. Figure 2cWhen the video control 201 is clicked on the conference interface 201 shown in FIG. 1 , the conference interface 201 can collect images based on the opened camera, so that the conference interface 201 is displayed as follows: Figure 2c The person shown or other images captured by the camera.

[0084] Based on the above Figure 2a The system architecture shown and Figure 2b The implementation principle of adding a functional node to the camera source pipeline is shown. Taking an application request to open a camera as an example, the specific implementation process of the media resource control method provided in the embodiment of the present application is described in detail in combination with other drawings.

[0085] Figure 3 According to an embodiment of the present application, a schematic diagram of an implementation process of a media resource control method is shown. It can be understood that Figure 3 The execution entities of each step in the process shown can be the above Figure 2a The Wine 220 in the system architecture shown can also be installed with the above Figure 2a In other embodiments, Wine 220 may also be provided in other operating systems that need to be compatible with Windows applications. Figure 3 The execution subject of each step in the illustrated implementation process may also be other electronic devices installed with the corresponding operating system, which is not limited here.

[0086] It should also be stated that the steps in the methods and processes in the embodiments of the present application are numbered for ease of reference, rather than to limit the order of precedence. If there is an order between the steps, the text description shall prevail.

[0087] like Figure 3 As shown, the implementation process includes the following steps:

[0088] 301: In response to the application's request to open the camera, traverse the camera information.

[0089] Exemplarily, the electronic device 100 or the above-mentioned Wine 220 can respond to the application's request to open the camera, load the dynamic link library file mfplat.dll through the source reader 223 or the media session 222, and read the enumerator interface preset in the file, and then call the system device enumerator to traverse various types of hardware device information on the electronic device 100. In addition, the mfplat.dll can also preset instructions to create an enumerator component object model (COM) that can identify video devices. Wine 220 can create a video device enumerator based on the COM component, and obtain relevant information of one or more physical cameras from the various types of hardware device information obtained by the above traversal, referred to as camera information. The camera information may include but is not limited to the identification identifier (ID), name, model, etc. of each camera device.

[0090] It can be understood that the MF application 211 can be a Windows application developed based on the MF framework and compatible with the Linux operating system, such as a conference application. Figure 2c As shown, the request for opening the camera by the conference application may correspond to a request for opening the camera triggered by a user clicking a video control 202 on the conference interface 201 of the conference application. Wine 220 of the Linux operating system may receive the opening request sent by the MF application 211 in response to the user operation.

[0091] It can be understood that the above-mentioned "traversal" includes the meanings of querying, copying, and obtaining as the name implies. In the embodiment of the present application, traversing the camera information means querying the relevant information of the camera device, and may also include copying and obtaining the queried camera information, which is not limited here. After traversing the camera information, Wine220 can continue to execute the following step 302 to return the camera information to the required application (such as the above-mentioned MF application 211). Please refer to the relevant description in the following steps for details, which will not be repeated here. In addition, the above-mentioned system device enumerator based on the preset enumerator interface call can be implemented by other frameworks (such as the DirectShow framework) running in Wine 220, for example, which is not limited here.

[0092] 302: Return the traversed camera information to the application.

[0093] Exemplarily, after executing step 301 to traverse and obtain relevant information of one or more configured cameras, the electronic device 100 or the Wine 220 may return the obtained camera information corresponding to each camera to the application requesting to open the camera. For example, Wine 220 may return the traversed camera information to the MF application 211 requesting to open the camera.

[0094] It can be understood that when the traversed camera information includes relevant information of multiple cameras, Wine 220 can return the camera information to the application in the form of a device list. Specifically, the process of Wine 220 allocating a device list for the traversed camera information and setting relevant attributes in the device list can be referred to the description below in conjunction with the corresponding drawings, which will not be repeated here.

[0095] 303: Receive information related to the target camera selected by the application.

[0096] Exemplarily, the MF application 211 requesting to open the camera, such as a conference application, can select one or more cameras as hardware capture devices to implement the video function according to its own needs after receiving the camera information returned by Wine 220. Among them, the camera selected by the application can be described as the target camera below. The target camera related information received at this time may include the video format and frame rate information supported by the camera.

[0097] As an example, the MF application 211 may select the front camera of the electronic device 100 as the hardware capture device for collecting the video data stream in response to the user clicking the video control. In some embodiments, if the electronic device 100 is also equipped with a rear camera, when the user clicks the switch control on the conference interface of the MF application 211, the MF application 211 may also select the rear camera of the electronic device 100 as the hardware capture device in response to the switch operation, which is not limited here.

[0098] 304: Parse the device path of the target camera.

[0099] Exemplarily, the electronic device 100 or the above-mentioned Wine 220 can obtain the relevant configuration file of the device and parse the device path of the camera device based on the camera information selected by the application, such as the camera device ID in the information. The parsed device path may include the call path of the video device that can open the target camera, etc.

[0100] 305: Add a video source acquisition node to the initialized camera source pipeline and set the camera video source.

[0101] For example, after the electronic device 100 or the Wine 220 parses and obtains the device path of the target camera, the target camera can be set as the video source according to the device path. The specific way of setting the video source can be achieved by adding a video source acquisition node to the pipeline corresponding to the camera source 226 of the MF framework 221.

[0102] As an example, in the development process of Linux operating system 200, a v4l2src (video for linux2 source) node can be created based on the gstreamer framework developed by Linux v4l2 as a preset video source acquisition node. Furthermore, when the pipeline corresponding to the camera source 226 in Wine 220 needs to add a video source acquisition node according to the needs of the application, the relevant path of the preset node can be read to complete the addition. Among them, v4l2src can be used to obtain the element (element) of the video data that can be collected by the corresponding camera from the video device kernel driver provided by v4l2.lib. After Wine 220 sets the attributes of the corresponding element according to the application requirements, the gstreamer framework supported by the Linux kernel can collect the data stream of the corresponding camera device based on the set attributes. Specifically, Wine 220 can add the above-mentioned element (element) to the pipeline corresponding to the camera source 226 through the gstreamer framework, and link it to the process of obtaining the video source acquisition node.

[0103] The pipeline corresponding to the camera source 226 can convert the state of the node element to the ready state after receiving the video source acquisition node added by the gstreamer framework link. At this time, the video source acquisition node on the pipeline can enter the ready state.

[0104] It can be understood that before setting the video source, the pipeline corresponding to the camera source 226 of the MF framework 221 needs to be initialized first, including initializing the pipeline interface and initializing various preset function nodes that can be added to the pipeline.

[0105] It can be understood that the detailed implementation process of the above steps 303 to 305 corresponding to the setting of the video source acquisition node according to the camera selected by the MF application 211 will be described in detail below in conjunction with another flowchart and will not be repeated here.

[0106] 306: Add other function nodes required by the application to the camera source pipeline.

[0107] Exemplarily, the other function nodes added above include, for example, a format conversion node, a frame rate adjustment node, etc. Taking the format conversion node as an example, when the format set by the application is a format not supported by the physical camera, Wine 220 can add a preset format conversion node to the pipeline corresponding to the camera source 226 to perform format conversion processing on the video data stream collected by the target camera. The format conversion function corresponding to the format conversion node can be implemented by developing corresponding code through the gstreamer framework.

[0108] It can be understood that the initial format converted by the above format conversion node can default to the format of the video data stream captured by the physical camera, and the target format of the corresponding conversion can be determined based on the negotiation result between the application and Wine 220.

[0109] Specifically, the process of adding a format conversion node and performing format negotiation based on step 306 may be described in detail below in conjunction with the corresponding drawings, and will not be described in detail here.

[0110] 307: Transmitting the camera data stream processed based on the preset function node to the application.

[0111] For example, the preset function nodes added by the camera source 226 on the corresponding pipeline based on the needs of the application program include the above-mentioned video source acquisition node and format conversion node. At this time, when the MF framework 221 in Wine 220 acquires the camera data stream through the camera source 226, the camera source 226 acquires each frame of image provided by the Linux kernel after the Linux kernel acquires and converts the format based on the gstreamer framework based on the video source acquisition node added on the pipeline. Figure 2c As shown, the MF application 211 can obtain the camera data stream obtained by the video stream acquisition node on the camera source 226 and display the video screen on the conference interface 201, solving the problem that the MF application cannot obtain the camera when it is compatible with other operating systems.

[0112] Furthermore, based on the format conversion node added to the pipeline, the video data stream formed by encoding each frame of the image can be converted into the format required by the application. For example, based on the above format conversion node, the video data stream in the YUV422 format collected by the camera can be converted into the video data stream in the YUV420 format used by the MF application 211. When the format supported by the camera is inconsistent with the format set corresponding to the video source obtained by the MF application 211, even if the MF application 211 can open the camera, the conference interface may still display Figure 4aThe garbled screen 401 shown. Based on the format conversion node added to the camera source 226 pipeline, after the camera data obtained by the video source acquisition node is format converted, the MF application 211 obtains the processed camera data stream with the matching format provided by Wine 220, and then it can be displayed. Figure 4a A clear picture 402 in the conference interface is shown.

[0113] In other embodiments, preset function nodes such as frame rate adjustment nodes may be added to the pipeline corresponding to the camera source 226 to further perform frame rate adjustment and other processing on the camera data stream to be transmitted to the MF application 211, which is not limited here. In this way, the MF application 211 can obtain the camera data stream processed by the corresponding function node from Wine 220, and can play or display the obtained data without further format conversion or frame rate adjustment, and the use efficiency of the obtained video data stream will also be higher. In addition, the amount of data after the data stream is processed may be greatly reduced, so the channel bandwidth occupied by the process of transmitting the data stream is also small.

[0114] It is understood that the video source acquisition node is a node that can open the physical camera of the electronic device 100. In other embodiments, a node that can call a virtual camera to open a local file can also be added to the camera source pipeline. It is understood that the virtual camera can be a camera created by virtualization technology and has functions such as collecting images similar to a physical camera.

[0115] As an example, refer to Figure 4b As shown, the MF application 211 can open a local picture 403 or a video file through Wine 220. The local picture can be opened through an open local file node for calling a virtual camera added to the pipeline corresponding to the camera source 226. In addition, in this scenario, a cropping node for cropping the picture size can also be added to the pipeline corresponding to the camera source 226 to crop the opened picture to a screen size suitable for the current display, which is not limited here.

[0116] In other embodiments, the preset function nodes may also include function nodes such as desktop recording. Taking MF application 211 as an example, when a user needs to share the desktop with other participants in a meeting and operate some application interfaces opened on the desktop, the desktop recording node added to the corresponding pipeline of the camera source 226 of Wine 220 can be used to call the virtual camera to record the current desktop display content and provide it to MF application 211. In this way, other participants can synchronously see the desktop and other recorded images displayed in the conference interface, and the desktop can be referenced. Figure 4c Desktop 404 is displayed on the conference interface.

[0117] Regarding the specific implementation process of traversing the camera information executed in the above step 301, the following is combined with Figure 5 Provide specific instructions.

[0118] Figure 5 According to an embodiment of the present application, a schematic diagram of an implementation process of Wine 220 traversing camera information of an electronic device is shown.

[0119] It is understandable that when the Windows application running on the electronic device 100 needs to turn on the video function, it can initiate a request to the Wine 220 of the operating system to call the video device kernel driver to turn on the camera. At this time, Wine 220 can respond to the request of the MF application 211 to turn on the camera and execute Figure 5 Each step in the process shown traverses and queries various attribute data of one or more cameras configured in the electronic device 100.

[0120] like Figure 5 As shown, the implementation process may include the following steps:

[0121] 501: Loading or creating a device enumerator.

[0122] Exemplarily, Wine 220 can provide an interface file of a call path corresponding to a system device enumerator, such as mfplat.dll. The dynamic link library file can pre-set the relevant interface parameters for calling the system device enumerator provided by other multimedia frameworks, such as the relevant interface parameters of the system device enumerator provided by the DirectShow framework, etc., which are not limited here. In addition, mfplat.dll can also preset instructions for creating an enumerator COM component that can identify video devices. For example, Wine 220 can load a preset enumerator COM component to create a video device enumerator by executing the function instruction "create class enumerator". The COM component can be used to create a video device enumerator, and the enumerator can further obtain the relevant information of the physical camera one by one from the various types of hardware device information obtained by the above traversal, that is, the camera information obtained in the above step 301.

[0123] It can be understood that Wine 220 can load the system device enumerator by executing this step 401, and obtain the relevant information of the camera device and other hardware devices recorded in the system of the electronic device 100. The video device enumerator created by Wine 220 based on the preset COM component can filter and obtain the video device enumerator such as the camera device from the enumeration results of the above system device enumerator.

[0124] 502: Allocate device list storage space for the relevant information of the i-th device obtained through enumeration.

[0125] Exemplarily, the device enumerator can enumerate the video device related information in the electronic device 100 one by one based on the preset enumeration conditions. For example, the device enumerator can first enumerate the ID, name and other information of the first camera device, and at this time Wine 220 can allocate a device list for the camera device to store the camera information such as the ID and name. The device list can be a data table or data storage space preset in Wine 220 for returning camera information to the MF application 211. The camera information obtained by the enumerator traversing the video devices in the electronic system device, such as the camera ID and other information, can be stored in the preset device list for providing to the required application, such as the MF application 211.

[0126] In other embodiments, the device enumerator may also sequentially enumerate the camera information corresponding to the 2nd / 3rd / …ith camera device. Correspondingly, Wine 220 may reallocate the size of the device list space according to the sequentially enumerated camera information or other device information. The more device information is enumerated, the larger the device list space allocated by Wine 220 may be, so as to fully store the enumerated device information. Therefore, after Wine 220 completes the step 502 to allocate the device list for the relevant information of the i-th device, it may continue to execute the judgment process of the following step 503 to determine whether there are still unenumerated cameras and other devices in the electronic device 100. For details, please refer to the relevant description below.

[0127] It is understood that, for different application requirements, the above enumeration results may include different video device related information. For example, for the requirement of MF application 211 to open a camera, the above enumeration results may include information such as the ID, name, and model of each camera. For another example, for the requirement of MF application 211 to open a local file, the above enumeration results may include virtual camera information created by the system, etc., which is not limited here.

[0128] 503: Determine whether there is relevant information of the (i+1)th device obtained through enumeration.

[0129] If the result of the determination is yes, it indicates that there are still devices such as cameras that have not been added to the device list. At this time, Wine 220 can continue to perform the following step 504 to reallocate the size of the device list space;

[0130] If the judgment result is no, it indicates that all camera devices in the electronic device 100 have been enumerated. At this time, Wine220 can execute the following step 508 to release the device enumerator.

[0131] Exemplarily, Wine 220 can continue to traverse the enumeration results corresponding to other cameras and other devices in the electronic device 100 based on the above-mentioned device enumerator to determine whether there is relevant information about the i+1th device obtained through enumeration. For the above example, if i=1, Wine 220 performs the judgment process of this step 503, which can be to determine whether the second device related information, such as camera information, is obtained through enumeration. If i=2, Wine 220 performs the judgment process of this step 503, which can be to determine whether the third device related information is obtained through enumeration, which will not be repeated here.

[0132] 504: Increase the list space size for the i+1th camera device.

[0133] Exemplarily, Wine 220 may continue to increase the size of the device list space for the i+1th device obtained through enumeration, so as to store the camera information corresponding to the newly added i+1th camera device.

[0134] For the above example, if i=1, Wine 220 executes this step 504 to increase the size of the device list space for the camera information such as the ID and name of the second camera device obtained by enumeration, which may also be referred to as the list size in the context of the embodiment of the present application. Similarly, if i=2, Wine 220 executes this step 504 to increase the size of the device list space for the camera information such as the ID and name of the third camera device obtained by enumeration, which will not be repeated here.

[0135] 505: Copy the camera information obtained by enumeration to the device list space.

[0136] Exemplarily, Wine 220 can copy the enumerated camera information in the device list space allocated for each camera device obtained by enumeration. The camera information may include but is not limited to the device ID, name, model and other information.

[0137] 506: Determine whether the copying is successful.

[0138] If the judgment result is yes, that is, the copy is successful, Wine 220 may execute the following step 507 to return the camera information to the MF application 211;

[0139] If the judgment result is no, that is, the copying fails, Wine 220 may execute the following step 510 to release the device list.

[0140] 507: Set other properties of the device list.

[0141] Exemplarily, after all the camera information obtained by the device enumerator is copied to the device list, Wine 220 can set other properties of each camera in the device list. For example, Wine 220 can set a name for the camera to be recognized by the MF application 211 and another name for the system to be recognized, etc., according to the requirements of the MF application 211 or the system, and there is no limitation here.

[0142] After executing step 507, Wine 220 may return to step 503 to continue traversing the i+2th device.

[0143] 508: Release the device enumerator.

[0144] Exemplarily, when Wine 220 performs the above step 503, if the result of the judgment is no, that is, all camera devices in the electronic device 100 have been enumerated, the device enumerator can be released. It is worth noting that in the context of the embodiment of the present application, "release" can mean to cancel information, reclaim list space or destroy a program. For example, the above release of the device enumerator can mean to destroy the device enumerator program that has been called or created.

[0145] 509: Returns a list of devices to the application.

[0146] Exemplarily, Wine 220 may send the read path corresponding to the above device list to the required application so that the application can read the device information stored in the device list space. For example, Wine 220 may send the read path corresponding to the camera device list to MF application 211 so that MF application 211 can obtain the enumerated camera information. The camera information may correspond to the camera information traversed and obtained in step 301 above.

[0147] 510: Release the device list.

[0148] For example, referring to the description in the aforementioned step 508, if Wine 220 executes the aforementioned step 506 and determines that the enumerated camera information cannot be copied to the device list, the device list can be released, that is, the storage space provided by the device list can be reclaimed. In the embodiment of the present application, for the device list, "release" can refer to the process of clearing the camera information stored in the device list to reclaim the space in the device list.

[0149] 511: Release the device enumerator.

[0150] The execution process of releasing the device enumerator may refer to the relevant description in the above step 508, which will not be described in detail here.

[0151] 512: Returns an error value.

[0152] Exemplarily, after executing the above steps to release the device list and the device enumerator, Wine 220 may return an error value to the application requesting to open the camera, wherein the application may be, for example, the MF application 211, and the returned error value may be, for example, "NULL".

[0153] Refer to the above Figure 3 As shown, after Wine 220 obtains the camera information through the device enumerator, it can continue to execute the above step 302 and return the traversed camera information to the application. The return process can refer to the detailed description in the above step 509, which will not be repeated here.

[0154] When the MF application 211 receives the camera information returned by Wine 220, a camera compatible with the video function to be provided can be selected as the target camera. At this time, Wine 220 can continue to set the video source acquisition node, etc. according to the selection of the application.

[0155] Specifically, for the specific implementation process of setting the video source acquisition node to the pipeline of the camera source 226 corresponding to the above steps 303 to 305, the following is combined with Figure 6 Provide specific instructions.

[0156] Figure 6 According to an embodiment of the present application, a schematic diagram of an implementation process of Wine 220 setting a video source acquisition node on a camera source pipeline is shown.

[0157] It can be understood that when the Windows application running on the electronic device 100 selects the camera device to be used from the multiple camera information traversed and obtained by Wine 220, Wine 220 can set the video source acquisition node on the pipeline of the camera source 226 according to the camera device selected by the application and the properties adapted to the corresponding settings of the application. The video source acquisition node can obtain the video data stream from the camera selected by the application as the video source, and then the application can smoothly display the corresponding video screen. Taking the conference application as an example, the MF application 211 can select the front camera of the electronic device 100 when providing the video function, and correspondingly Wine 220 can parse the camera device path and call the camera pipeline to complete the setting of the video source acquisition node.

[0158] Specifically, Figure 6 As shown, the implementation process may include the following steps:

[0159] 601: Obtain the attribute information corresponding to the device selected by the application.

[0160] Exemplarily, in response to the request of the MF application 211 to open the camera, after the camera information of the electronic device 100 is traversed by the above-mentioned device enumerator and provided to the MF application 211, the MF application 211 can select one or more cameras according to its own needs. At this time, the MF application 211 can provide the identification information such as the ID and name of the selected camera device as attribute information to Wine 220. Correspondingly, Wine 220 can receive the attribute information fed back by the MF application 211.

[0161] 602: Obtain the device identifier included in the device attribute information.

[0162] Exemplarily, after Wine 220 receives the attribute information corresponding to the camera device selected by the MF application 211, it can parse the device identifier of the camera device from the information, such as the ID, name, model, etc. of the camera device, which can be used to identify the identification information of the corresponding device.

[0163] 603: Resolve the device path based on the device identifier.

[0164] Exemplarily, the device path may be a file access path corresponding to the camera device selected by the MF application 211, for example, " / dev / video0" and " / dev / video1" may correspond to device paths of different cameras. Based on the device identifier obtained in the above steps, Wine 220 may find the file access path corresponding to the target camera as the parsed device path.

[0165] It can be understood that the parsed device path can support the media resource 225 of Wine 220 to access the driver file under the path to open the target camera. The driver file under the device path can be loaded by the video device kernel driver corresponding to the target camera in v4l2.lib provided by the Linux kernel, for example, to drive the target camera to start working. For details, please refer to the relevant description below, which will not be repeated here.

[0166] 604: Call the pipeline initialization interface to construct an initialized camera source pipeline.

[0167] Exemplarily, the media resource 225 in Wine 220 can call the pipeline initialization interface to connect one or more function nodes to be added to construct an initialization pipeline capable of transmitting data. The function node at one end of the initialization pipeline can receive data input, and the function node at the other end can output data processed by each function node of the pipeline. For example, the video source acquisition node connected to the initialization pipeline can receive data stream input from the camera, and at the output end of the pipeline, the node or other function nodes can output the received or processed data stream to the MF application 211 requesting to open the camera.

[0168] It can be understood that the initialized camera source pipeline may include one or more functional nodes, and the specific added functional nodes may be customized according to the requirements of the MF application 211. For example, in some embodiments, the initialized camera source pipeline may only include a video source acquisition node. In other embodiments, the initialized camera source pipeline may include a video source acquisition node and a format conversion node, etc., which are not limited here.

[0169] In some other embodiments, the initialization pipeline constructed above may also include the above Figure 2b The opening local file node and the cropping node, or the desktop screen recording node shown, are not limited here.

[0170] Understandably, Wine 220 is running Figure 6 After completing the steps shown in the constructed camera source pipeline and adding the video source acquisition node, you can also continue to perform the following Figure 7 In the process shown, other functional nodes such as the format conversion node required by the MF application 211 are continuously added to the pipeline. The specific adding process will be described in detail below and will not be repeated here.

[0171] 605: Initialize the video source acquisition node on the camera source pipeline.

[0172] Exemplarily, after executing the above steps to construct the initialized camera source pipeline, Wine 220 has completed the initial addition of some functional nodes, such as the process of adding the video source acquisition node to the camera source pipeline. At this time, Wine 220 can perform functional initialization on the video source acquisition node and other functional nodes on the pipeline, including initializing the interface parameters for the camera as the video source to provide the corresponding data stream. After completing the initialization of each functional node, the camera source pipeline can successfully receive data.

[0173] In other embodiments, if the initialized camera source pipeline includes an open file node or a screen recording node, Wine 220 may initialize the open file node or the screen recording node by executing step 605, and no limitation is made here.

[0174] 606: Encapsulate the initialized camera source pipeline and construct an output end.

[0175] For example, after executing steps 604 to 605 to complete the initialization of the camera source pipeline and each functional node on the pipeline, Wine 220 can execute the relevant code for encapsulating the camera source pipeline and construct an output end for the pipeline to output the processed data stream. As an example, the output end can refer to the above Figure 2b The various "output ports" shown.

[0176] 607: Return to the video source.

[0177] Exemplarily, Wine 220 can return the media resource 225 of the corresponding camera source pipeline to the MF application 211. It can be understood that the camera source pipeline constructed above can be used to implement the functions corresponding to the camera source 226. In the embodiment of the present application, the camera source 226 can be a resource subclass provided by the media resource 225. In other embodiments, the camera source can also be used as another media resource to accept the call of the media session 222 or the source reader 223, etc., and provide a camera data stream based on the physical camera acquisition.

[0178] Refer to the above Figure 3 As shown, after Wine 220 completes adding the video source acquisition node to the corresponding pipeline, it can open the camera and obtain the camera data stream based on the added video source acquisition node. When the video source data format set by the MF application 211 to implement the corresponding function is inconsistent with the video format supported by the target camera acquisition data stream, Wine 220 can also continue to execute the above step 306 to add other function nodes to the pipeline corresponding to the camera source 226, such as adding a format conversion node.

[0179] Specifically, for the specific implementation process of adding a format conversion node on the camera source pipeline corresponding to the above step 306, the following is combined with Figure 7 Provide specific instructions.

[0180] Figure 7 According to an embodiment of the present application, a schematic diagram of an implementation process of Wine 220 adding a format conversion node to a camera source pipeline is shown.

[0181] It is understandable that the format of the image or video data stream corresponding to the video function provided by the Windows application running on the electronic device 100, such as the MF application 211, may be a format not supported by the physical camera of the electronic device 100. For example, when the video format of the video function supported by the MF application 211 is YUV420, and the format supported by the physical camera of the electronic device 100 is YUV422, it is necessary to perform format conversion processing on the video data collected by the physical camera before providing it to the MF application 211, so as to realize the smooth display of the video content collected by the camera in the conference interface of the MF application 211, and avoid garbled video images displayed on the interface.

[0182] Specifically, Figure 7 As shown, the implementation process may include the following steps:

[0183] 701: Returns the list of video formats supported by the target camera to the application.

[0184] Exemplarily, after Wine 220 completes adding the video source acquisition node and initializing the constructed camera source pipeline and function nodes based on the target camera selected by the MF application 211, it can provide the format of the video data stream supported by the target camera to the MF application 211. For example, Wine 220 can return the video format information supported by the target camera to the MF application 211 in the form of a video format list. As an example, the video format list supported by the target camera can include one or more of YUV420, YUV422, YUV444, YUV411, YUV410, etc., which are not limited here. Correspondingly, after receiving the video format list returned by Wine 220, the MF application 211 can determine whether these video formats are adapted to its own functional requirements according to its own needs.

[0185] In other embodiments, Wine 220 may also return hardware configuration information such as the frame rate of the target camera capturing the video data stream to the MF application 211, which is not limited here.

[0186] 702: Read the video format set by the application.

[0187] For example, after receiving the video format list returned by Wine 220, MF application 211 can determine whether the video format supported by the target camera matches the video format required by its own function. If they match, MF application 211 can select a video format from the received video format list and feed it back to Wine 220 as the set video format. If they do not match, MF application 211 can set a video format according to its own functional requirements and feed it back to Wine 220 as the set video format.

[0188] 703: Determine whether the target camera supports the video format set by the application.

[0189] If the result of the determination is yes, that is, the target camera supports the video format set by the application, Wine 220 may execute the following step 708 to determine whether the camera pipeline is ready.

[0190] If the determination result is no, that is, the target camera does not support the video format set by the application, Wine 220 may continue to execute the following steps 704 to 707 to add a format conversion node on the camera source pipeline.

[0191] Exemplarily, Wine 220 may compare the video format set by the MF application 211 with various video formats supported by the target camera to determine whether the target camera has a matching video format, that is, the above-mentioned determination of whether the target camera supports the video format set by the application.

[0192] It can be understood that the video format list returned by Wine 220 to the application (e.g., MF application 211) in step 701 may not include all video formats supported by the target camera, that is, the video format list may have omissions in the corresponding video formats recorded. The judgment process performed by Wine 220 in this step can make up for the video formats that may be recorded in the video format list and determine whether the target camera supports the video format set by the application.

[0193] 704: Pause the data stream transmission process in the camera source pipeline.

[0194] Exemplarily, when Wine 220 performs the above step 703 and determines that the target camera does not support the format set by the MF application 211, the transmission process of the camera data stream in the pipeline can be suspended. Wine 220 can suspend the transmission of the data stream in the pipeline, for example, by closing the video source acquisition node and temporarily not receiving the data stream collected from the camera.

[0195] 705: Add a format conversion node to the camera source pipeline.

[0196] Exemplarily, after Wine 220 suspends the data stream transmission in the pipeline, a preset format conversion node can be added to the camera source pipeline. It can be understood that the added format conversion node can receive the camera data stream output by the video source acquisition node, and after format conversion processing, the processed camera data stream can be output to the media session 222 or the source reader 223 through the output end of the camera source, and finally provided to the MF application 211 by Wine 220.

[0197] 706: Based on the video formats supported by the camera and the formats set by the application, the video formats before and after conversion are determined through negotiation.

[0198] For example, Wine 220 can obtain all video formats supported by the target camera through the video source acquisition node, match the formats supported by the format conversion node, and thus determine a certain video format supported by the target camera as the video format before conversion and notify the video source acquisition node. In this way, the video source acquisition node can set the target camera to collect data streams using the video format.

[0199] Further, Wine 220 may determine the format set by MF application 211 as the target format after conversion. If the formats supported by the format conversion node do not include the video format set by MF application 211, Wine 220 may also request MF application 211 to reset the video format supported by the format conversion node as the target format after conversion. This is not limited here.

[0200] It is understandable that Wine 220 can perform the process of negotiating the video format in this step through the media resource 225, and no limitation is made here.

[0201] 707: Restart the camera source pipeline.

[0202] Exemplarily, Wine 220 may initiate a request to obtain the camera data stream through the media session 222, triggering a restart of the camera source pipeline, so that the pipeline starts transmitting the camera data stream again.

[0203] 708: Determine that the camera source pipeline is ready.

[0204] Exemplarily, after restarting the camera source pipeline, Wine 220 can determine that the pipeline corresponding to the camera source 226 in the media resource 225 is ready. At this time, Wine 220 can perform the following step 709 to receive the request of the MF application 211 to open the camera, obtain the camera data stream, etc.

[0205] 709: Receive a request from an application to read the camera data stream.

[0206] For example, Wine 220 may receive a request from MF application 211 to read the camera data stream in response to the requirement of MF application 211 to enable the video function. Further, Wine 220 may transmit the camera data stream based on the above-prepared camera source pipeline.

[0207] 710: Copy the data stream at the output end of the camera source pipeline.

[0208] 711: Returns data stream to the application.

[0209] Exemplarily, the source reader 223 in Wine 220 can copy the camera data stream transmitted by the camera source pipeline from the output end of the pipeline. Furthermore, the source reader 223 can return the copied camera data stream to the MF application 211.

[0210] In other embodiments, Wine 220 may also copy the camera data stream transmitted by the camera source pipeline from the output end of the pipeline through the media session 222, and return the copied camera data stream to other application programs in need, which is not limited here.

[0211] It is understood that the above data stream may include a plurality of image frames collected and processed in sequence. Therefore, the process of returning the data stream may be, for example, a process of returning processed image data frame by frame, which is not limited here.

[0212] Figure 8 According to an embodiment of the present application, a structural schematic diagram of an electronic device 100 is shown.

[0213] like Figure 8 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, and a temperature sensor 180H.

[0214] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0215] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0216] The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions. In the embodiment of the present application, the processor 110 can control Wine 220 through the controller to execute each step in the implementation process corresponding to the above-mentioned media resource control method provided in the present application, so as to realize that when the MF application is compatible with operating systems such as Linux, it can also realize the function of opening the camera and starting the video, and can also support the opening of local files or sharing desktops quickly and stably through the opening local file node and the desktop recording node on the pipeline, without problems such as size mismatch and frame rate mismatch.

[0217] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0218] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a universal serial bus (USB) interface, etc.

[0219] The USB interface 130 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transmit data between the electronic device 100 and a peripheral device.

[0220] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0221] The charging management module 140 is used to receive charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also power the electronic device through the power management module 141 .

[0222] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.

[0223] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0224] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to electronic device 100. Wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. applied to electronic device 100.

[0225] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0226] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0227] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0228] The electronic device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. In the embodiment of the present application, the shooting function implemented by the electronic device 100 through the above-mentioned ISP, the camera 193 and other devices can support Wine 220 to execute the media resource control method provided in the present application, and respond to the MF application 211 to turn on the video function of the camera.

[0229] Among them, ISP is used to process data fed back by camera 193.

[0230] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0231] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0232] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0233] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.

[0234] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0235] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0236] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0237] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0238] The embodiments of the present application also provide a computer program product for implementing the media resource control methods provided in the above embodiments.

[0239] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program module or module code executed on a programmable system, and the programmable system includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0240] A computer program module or module code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0241] Module code can be implemented with high-level modular language or object-oriented programming language to communicate with the processing system. When necessary, module code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0242] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, a floppy disk, an optical disk, an optical disk, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0243] References to "one embodiment" or "an embodiment" in the specification mean that the specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one exemplary implementation or technology disclosed according to the embodiment of the present application. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily all refer to the same embodiment.

[0244] The disclosure of the embodiment of the present application also relates to an operating device for executing the text. The device can be specially constructed for the required purpose or it can include a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer-readable medium, such as, but not limited to any type of disk, including a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic or optical card, an application-specific integrated circuit (ASIC) or any type of medium suitable for storing electronic instructions, and can be coupled to a computer system bus. In addition, the computer mentioned in the specification may include a single processor or may be an architecture involving multiple processors for increased computing power.

[0245] In addition, the language used in this specification has been primarily selected for readability and instructional purposes and may not be selected to describe or limit the disclosed subject matter. Therefore, the present application embodiment disclosure is intended to illustrate rather than limit the scope of the concepts discussed herein.

Claims

1. A media resource control method, applied to an electronic device, characterized in that: The electronic device is installed with a first operating system and a first application, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and The method comprises: detecting a first request by the first application to open a first media device; In response to the first request, a first calling instruction is transmitted to a first driver of the kernel layer of the first operating system through a first transmission interface, wherein: The first transmission interface is developed based on a first operating system, and the first calling instruction is used to instruct the first driver to open the first media device requested by the first request; The first media resource collected by the first media device is provided to the first application through the first transmission interface.

2. The method according to claim 1, characterized in that The first operating system includes an MF framework running in a compatibility layer, the MF framework is provided by the second operating system, and the MF framework is provided by the second operating system, and The MF framework includes multiple transmission interfaces, and the multiple transmission interfaces include the first transmission interface.

3. The method according to claim 1 or 2, characterized in that: The detecting a first request by the first application to open a first media device comprises: Receiving a user operation on a video control on a first interface, wherein the first interface is an interface displayed corresponding to the electronic device running the first application; A first request for opening a first media device triggered by the user operation is detected.

4. The method according to claim 2, characterized in that: Providing the first media resource collected by the first media device to the first application through the first transmission interface includes: Recognizing that a format of a media resource set by the first application does not match a format of a first media resource collected by the first media device; receiving, through a second transmission interface in the MF framework, a first media resource transmitted through the first transmission interface, wherein the second transmission interface is an interface having a format conversion function; The first media resource is format-converted through the second transmission interface to obtain a second media resource, and the second media resource is provided to the first application through the second transmission interface.

5. The method according to claim 4, characterized in that The output end of the first transmission interface in the MF framework is connected to the input end of the second transmission interface to form a data transmission channel for transmitting the first media resource and the second media resource.

6. The method according to claim 2, characterized in that The MF framework further includes a resource reading module, the first media device is a target device selected from a plurality of media devices acquired by the resource reading module in the MF framework, and The method of acquiring multiple media devices through the resource reading module of the MF framework includes: Accessing the dynamic link library of the first operating system through the resource reading module to obtain a first dynamic link library file; The resource reading module reads a calling interface corresponding to the first device enumerator in the first dynamic link library file, and the first device enumerator is called based on the calling interface to obtain device information of the plurality of media devices.

7. The method according to claim 6, characterized in that The method of acquiring multiple media devices through the resource reading module of the MF framework also includes: The resource reading module reads configuration information instructing to create a second device enumerator in the first dynamic link library file, and creates the second device enumerator based on the configuration information to obtain device information of the plurality of media devices.

8. The method according to claim 6 or 7, characterized in that: The device information of the plurality of media devices includes at least one of the following: an identifier of the media device; the name of the media device; The model of the media device.

9. The method according to claim 2, characterized in that: The method comprises: detecting a second request by the first application to open a locally stored first media file; In response to the second request, transmitting a second call instruction to a second driver of the kernel layer of the first operating system through a third transmission interface in the MF framework, wherein the second call instruction is used to instruct the second driver to open a second media device requested by the second request; The content read by the second media device by calling the storage path of the first media file is provided to the first application through the third transmission interface, wherein the second media device is a virtual device created by virtualization technology.

10. The method according to claim 9, characterized in that The providing the content read by the second media device through the third transmission interface by calling the storage path of the first media file to the first application includes: Recognizing that the screen size required for the second media device to read the content by calling the storage path of the first media file does not match the screen size corresponding to the display of the second interface of the first application; receiving, through a fourth transmission interface in the MF framework, content read by calling the storage path of the first media file transmitted through the third transmission interface, wherein the fourth transmission interface is an interface with a cropping function; The content of the first media file is cropped to correspond to the required screen size through the fourth transmission interface, and the cropped content of the second media file is provided to the first application through the fourth transmission interface.

11. The method according to claim 2, characterized in that The method comprises: detecting a third request for the first application to share desktop display content; In response to the third request, transmitting a third calling instruction to a third driver of the kernel layer of the first operating system through a fifth transmission interface in the MF framework, wherein the third calling instruction instructs the third driver to open a second media device to record desktop display content; The third media resource generated corresponding to the desktop display content recorded by the second media device is provided to the first application through the fifth transmission interface.

12. The method according to any one of claims 1 to 8, characterized in that The first media device includes a camera device.

13. The method according to any one of claims 9 to 11, characterized in that The second media device includes a virtual camera.

14. An electronic device, characterized in that: The electronic device is installed with a first operating system and a first application, wherein the first application is developed based on a second operating system, and the second operating system is of a different type from the first operating system; and The electronic device includes a first transmission interface developed based on the first operating system, wherein: The first transmission interface is used to send a first call instruction to a first driver of the kernel layer of the first operating system, wherein the first call instruction is used to respond to a first request of a first application to open a first media device and instruct the first driver to open the first media device requested by the first request; and The first transmission interface is used to provide the first media resource collected by the first media device to the first application.

15. The electronic device according to claim 14, characterized in that: The first operating system includes an MF framework running in a compatibility layer, the MF framework is provided by the second operating system, and, The MF framework includes multiple transmission interfaces, and the multiple transmission interfaces include the first transmission interface.

16. The electronic device according to claim 15, characterized in that: The plurality of transmission interfaces include a second transmission interface, wherein the second transmission interface is an interface having a format conversion function, and, The second transmission interface is used to receive a first media resource transmitted through the first transmission interface, and, The second transmission interface is used to convert the format of the first media resource to obtain a second media resource, and provide the second media resource to the first application.

17. The electronic device according to claim 15, characterized in that: The plurality of transmission interfaces include a third transmission interface, wherein: The third transmission interface is used to transmit a second calling instruction to a second driver of the kernel layer of the first operating system, wherein the second calling instruction is used to respond to a second request of the first application to open a first media file stored locally, and instruct the second driver to open a second media device requested by the second request; and The third transmission interface is used to provide the content read by the second media device by calling the storage path of the first media file to the first application, wherein the second media device is a virtual device created by virtualization technology.

18. The electronic device according to claim 17, characterized in that: The plurality of transmission interfaces include a fourth transmission interface, wherein the fourth transmission interface is an interface having a clipping function, and, The fourth transmission interface is used to receive the content read by calling the storage path of the first media file transmitted through the third transmission interface, and, The fourth transmission interface is used to crop the content of the first media file to correspond to the required screen size, and provide the cropped content of the second media file to the first application.

19. The electronic device according to claim 15, characterized in that: The plurality of transmission interfaces include a fifth transmission interface, wherein: The fifth transmission interface is used to transmit a third calling instruction to a third driver of the kernel layer of the first operating system, wherein the third calling instruction is used to respond to a third request of the first application to share the desktop display content and instruct the third driver to open the second media device to record the desktop display content; and The fifth transmission interface is used to provide the third media resource generated corresponding to the desktop display content recorded by the second media device to the first application.

20. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the media resource control method described in any one of claims 1 to 13.

21. A computer readable medium, characterized in that The readable medium stores instructions, which, when executed on a computer, enable the computer to execute the media resource control method according to any one of claims 1 to 13.

Images