A data transmission system and data transmission method for electronic equipment

Abstract

The embodiment of the present application provides a data transmission system and a data transmission method applied to an electronic device, and relates to the field of data transmission technology, wherein a data transmission system applied to an electronic device comprises: a first transmission unit and a second transmission unit; the first transmission unit is used to detect that target screen projection data is written in the target memory, and send a data read instruction to the second transmission unit; the second transmission unit is used to read the target screen projection data in the target memory based on the target information carried in the data read instruction, and connect the target screen projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the screen projection data, calls the specified interface of the operating system platform, and obtains the screen projection data required for the processing task for executing the processing task. It can be seen that this solution can reduce the limitations of the processing of screen projection data.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a data transmission system and a data transmission method applied to electronic equipment. Background Art

[0002] Screen projection devices such as wireless screen projectors can project the audio and video data of the screen projector onto any electronic device, and the audio and video data can be output on the electronic device as projection data. Currently, the projection data can be forwarded or used for other purposes with the help of other software running on the electronic device. Exemplarily, the projection device can project the projection data onto a mobile device, and the video conferencing software on the mobile device can transmit the projection data to the conference system, so that other devices participating in the meeting can output the projection data.

[0003] However, in the related technology, the hardware manufacturers of the projection device and the software manufacturers need to cooperate deeply so that the projection device can be adapted to the software, thereby realizing the use of projection data for various purposes; but there are innumerable software manufacturers, and it is obviously not easy to have in-depth cooperation between the hardware manufacturers of the projection device and each software manufacturer, so there are great limitations in the processing of projection data.

[0004] At present, how to reduce the limitations in the processing of screen projection data is an urgent problem to be solved. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a data transmission system and a data transmission method applied to an electronic device to reduce the limitations of processing screen projection data. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a data transmission system applied to an electronic device, including: a first transmission unit and a second transmission unit;

[0007] The first transmission unit is used to detect that target screen projection data is written in the target memory, and send a data read instruction to the second transmission unit; wherein the target memory is a memory into which the target screen projection data is written by the target screen projection server, and the data read instruction carries target information for indicating the address of the target memory; the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is data transmitted by the screen projection device;

[0008] The second transmission unit is used to read the target projection data in the target memory based on the target information carried in the data reading instruction, and connect the target projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for executing the processing task.

[0009] In a second aspect, an embodiment of the present application provides a data transmission method, which is applied to an electronic device; the method includes:

[0010] It is detected that target screen projection data is written in the target memory, and the target screen projection data in the target memory is read; wherein the target memory is a memory into which the target screen projection data is written by a target screen projection server, the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is data transmitted by the screen projection device;

[0011] The target projection data is connected to the operating system platform of the electronic device so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for executing the processing task.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0013] Memory, used to store computer programs;

[0014] The processor is used to implement any of the above data transmission methods when executing the program stored in the memory.

[0015] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned data transmission methods when executed by a processor.

[0016] Beneficial effects of the embodiments of the present application:

[0017] The first transmission unit in the data transmission system applied to the electronic device provided by the embodiment of the present application can send a data reading instruction to the second transmission unit when detecting that the target screen projection data is written in the target memory. The second transmission unit reads the target screen projection data in the target memory based on the target information carried in the data reading instruction, and connects the target screen projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for the subsequent execution of the processing task. It can be seen that when the projection data is used for various purposes in the embodiment of the present application, there is no need for the hardware manufacturer of the projection device and the manufacturer of the software to cooperate deeply to make the projection device and the software compatible (for example, the manufacturer of the projection device opens a private interface for the manufacturer of the software), but the target projection server can write the projection data to the target memory. In this way, the data transmission system connects the projection data written in the target memory to the operating system platform of the electronic device, and the interface of the operating system platform is a public interface, which can be called by various software, so that various software can directly call the specified interface of the operating system platform of the electronic device, thereby obtaining the projection data required for the processing task, so as to use the projection data for various purposes. It can be seen that this solution can reduce the limitations in processing screen projection data.

[0018] Furthermore, in the process of processing the projection data in the embodiment of the present application, the projection data is not copied, and the projection data is only transmitted through the target memory, which also reduces the consumption of storage resources.

[0019] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0021] Figure 1 A schematic diagram of a data transmission system for an electronic device provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the working principle of a data transmission system provided in an embodiment of the present application;

[0023] Figure 3 A flowchart of a data transmission method provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of an all-in-one conference device provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0027] First, some professional terms in the embodiments of the present application are briefly introduced:

[0028] AOSP: Android Open Source Project, which contains the main components and framework of the Android operating system. AOSP is jointly developed and maintained by the Open Handset Alliance, aiming to provide an open, free and customizable operating system platform.

[0029] HAL: Hardware Abstraction Layer, the standard interface that the Android system provides to hardware vendors to implement. Specifically, HAL provides a set of hardware-independent interfaces that hide the specific implementation details of the underlying hardware, so that the Android system does not need to care about the underlying hardware and the implementation of the drivers of various manufacturers; in this way, the upper framework and applications of the operating system can use a unified API (Application Programming Interfac) to interact with the hardware.

[0030] Binder: A core component in the Android system, mainly used to implement inter-process communication; Binder provides an efficient inter-process communication method, allowing data exchange and function calls between components of different applications. The Binder mechanism is based on the client-server communication mode. From the perspective of the underlying communication principle, Binder can be implemented based on shared memory. The Binder driver opens up a memory space in the kernel space for data storage of Binder communication. At the same time, the Binder driver also maintains a node indexed by the process ID to quickly locate the target process. When communicating across processes, the Binder driver will establish a Binder connection between the sender process and the receiver process, and maintain the status of the connection in the Binder driver. Through the Binder connection, the sender process can write data to the shared memory and notify the receiver process to read the data; the receiver process can also send data to the sender process through the Binder connection. Among them, hwbinder is the hardware node specified by Android for the system's AOSP to communicate with HAL, and binder is the hardware node used for communication between services and between services and HAL in the system.

[0031] AIDL: Android Interface Definition Language, a description language used to define cross-process communication interfaces in the Android system. It allows developers to define an interface so that clients and servers in different processes can interact through the IPC mechanism.

[0032] parcelable: The parameter type specified in the AIDL language.

[0033] Wireless screen projector: A device with a USB (Universal Serial Bus) interface and a built-in Windows installation-free software; the built-in software can run after being inserted into Windows, capture the image of the computer screen in real time, and transmit the captured screen data to the all-in-one device through the network connected to the computer.

[0034] All-in-one device: An Android device with a built-in camera, microphone, speaker and other components; it supports standard HDMI (High Definition Multimedia Interface) output; it can support running software of various Android versions.

[0035] V4L2: Short for Video for Linux 2, which is the kernel driver for video in Linux.

[0036] Tiny ALSA: It can be considered as Tiny + ALSA (Advanced Linux Sound Architecture). The current mainstream audio architecture in Linux is ALSA. Tiny ALSA is a simplified version of ALSA. Tiny ALSA simplifies ALSA programming in user space.

[0037] In order to better understand the present application, before introducing the solution provided by the embodiment of the present application, the data transmission solution in the related art is introduced in the scenario of video conferencing:

[0038] In a video conferencing scenario, there are two video streams for the main venue, of which the first video stream is the picture of people captured by the camera in real time; the second video stream is the computer presentation picture projected through a projection device. At this time, there is a demand for video conferencing software: to connect the projection data of the projection device (i.e., the presentation document picture) to the video conferencing system, so that each conference party can also see the projection data. In the related technology, the manufacturer of the projection device needs to cooperate deeply with the manufacturer of the video conferencing software. For example, a set of private interfaces needs to be developed on the projection device so that the video conferencing software can call the private interface to obtain the projection picture; however, the development process is time-consuming and labor-intensive. At the same time, there are countless software manufacturers. It is obviously not easy to cooperate deeply with the hardware manufacturer of the projection device and each software manufacturer. Therefore, there are great limitations on the processing of projection data.

[0039] It should be noted that the first transmission unit or the second transmission unit included in the data transmission system described in the embodiment of the present application does not limit the data transmission system to include these two hardware parts or these two hardware parts in physical form. The term "unit" and "module" in the embodiment of the present application can be a hardware part, a software program, or a software program and the necessary carrier for the operation of the software program. With the evolution of technology, the steps required to execute these "units" or "modules" or the functions undertaken (for example, a certain unit or a certain module is "used" for the steps described later) may be performed in hardware, software, or a combination of hardware and software. As long as it is hardware, software, or a combination of hardware and software that can execute the steps required to be executed by the "unit", it can be considered as a "unit" or "module" in the embodiment of the present application. Based on the problems existing in the relevant technology, the embodiment of the present application provides a data transmission system applied to electronic devices to reduce the limitations of the processing of projection data.

[0040] The following is an introduction to a data transmission system applied to an electronic device provided in an embodiment of the present application.

[0041] Among them, the data transmission system applied to electronic devices provided in the embodiment of the present application can be a software platform, and the electronic devices can be various terminal devices or server devices, such as: all-in-one devices, mobile phone devices and personal computer PC devices, etc. The embodiment of the present application does not specifically limit the form of the electronic device.

[0042] The following is a brief introduction to the application scenarios of the data transmission system of the embodiment of the present application:

[0043] The data transmission system of the embodiment of the present application can be applied to various scenarios based on processing of projection data. For example, through video conferencing software, the projection data can be sent to the video conferencing system, so that all parties in the meeting can see the projection screen; through data recording software, the projection data can be uploaded to the cloud server, so that the projection data can be saved and recorded.

[0044] Of course, the above scenarios are only for illustrative purposes, and the embodiments of the present application do not specifically limit the scenarios in which the data transmission system is applied.

[0045] Among them, a data transmission system applied to an electronic device comprises: a first transmission unit and a second transmission unit;

[0046] The first transmission unit is used to detect that target screen projection data is written in the target memory, and send a data read instruction to the second transmission unit; wherein the target memory is a memory into which the target screen projection data is written by the target screen projection server, and the data read instruction carries target information for indicating the address of the target memory; the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is data transmitted by the screen projection device;

[0047] The second transmission unit is used to read the target projection data in the target memory based on the target information carried in the data reading instruction, and connect the target projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for executing the processing task.

[0048] It can be seen that when the screen projection data is used for various purposes in the embodiments of the present application, there is no need for the hardware manufacturer of the screen projection device to cooperate deeply with the software manufacturer to adapt the screen projection device to the software (for example, the manufacturer of the screen projection device opens a private interface for the software manufacturer). Instead, the target screen projection server can write the screen projection data to the target memory. In this way, the data transmission system connects the screen projection data written in the target memory to the operating system platform of the electronic device. The interface of the operating system platform is a public interface and can be called by various software. In addition, various software can directly call the specified interface of the operating system platform of the electronic device to obtain the screen projection data required for processing tasks, so as to use the screen projection data for various purposes. It can be seen that this solution can reduce the limitations of the processing of screen projection data.

[0049] Furthermore, in the process of processing the projection data in the embodiment of the present application, the projection data is not copied, and the projection data is only transmitted through the target memory, which also reduces the consumption of storage resources.

[0050] The data transmission system applied to an electronic device provided by the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings of the specification.

[0051] like Figure 1 As shown, an embodiment of the present application provides a data transmission system applied to an electronic device, including: a first transmission unit 110 and a second transmission unit 120;

[0052] The first transmission unit 110 is used to detect that target screen projection data is written in the target memory, and send a data read instruction to the second transmission unit 120; wherein the target memory is a memory into which the target screen projection data is written by the target screen projection server, and the data read instruction carries target information for indicating the address of the target memory; the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is data transmitted by the screen projection device;

[0053] The second transmission unit 120 is used to read the target projection data in the target memory based on the target information carried in the data reading instruction, and connect the target projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for executing the processing task.

[0054] It should be emphasized that both the first transmission unit 110 and the second transmission unit 120 can belong to process units, and can also be considered as software units or software modules, and the target screen projection server can be a software unit or a hardware unit, and the present application does not make specific restrictions on this. In addition, the first transmission unit 110 can also be a module embedded in the target screen projection server. The specific content will be introduced in other embodiments and will not be repeated here. In addition, in order to enable the target screen projection server to write the target screen projection data to the target memory, the target screen projection server can establish an adaptation relationship with the data transmission system in advance. The adaptation relationship is to enable the target server to interact with the data transmission system to write the target screen projection data to the target memory; and the adaptation relationship can be set at the time of the target transmission system and the target screen projection server leaving the factory. Of course, the adaptation relationship can also be configured by the user during use, and the embodiments of the present application do not make specific restrictions on this. In addition, it can be understood that the screen projection device and the target screen projection server can be communicatively connected (the target screen projection server itself serves as the server of the screen projection device), that is, the screen projection device and the screen projection server have an adaptation relationship. Then, by establishing an adaptation relationship between the target screen projection server and the data transmission system, it is equivalent to establishing an adaptation relationship between the screen projection device and the data transmission system. This eliminates the need for in-depth cooperation between the screen projection device and various software manufacturers, and can achieve the purpose of reducing the limitations in the processing of screen projection data.

[0055] Among them, the target screen projection data may include image data, audio data, etc., which are not specifically limited in the embodiments of the present application. It can be understood that in the first optional implementation method, the first transmission unit 110 can perform real-time detection on the target memory, and the screen projection server running in the electronic device and communicating with the screen projection device can write the data transmitted by the screen projection device into the target memory after receiving the data transmitted by the screen projection device. At this time, the first transmission unit 110 detects that the target screen projection data is written in the target memory, so that a data read instruction can be sent to the second transmission unit 120, and the data read instruction carries target information for indicating the address of the target memory; and, in the second optional implementation method, the first transmission unit 110 can also detect that the writing of the target screen projection data in the target memory has stopped, and the target screen projection data has been completely written into the target memory. At this time, a data read instruction can be sent to the second transmission unit 120, which is not specifically limited in the embodiments of the present application. For different scenarios, the first optional implementation method or the second optional implementation method mentioned above can be adopted. For example, for scenarios with high real-time requirements, such as video conferencing scenarios, the first optional implementation method mentioned above can be adopted, and for scenarios with low real-time requirements, such as data recording scenarios, the second optional implementation method mentioned above can be adopted. In addition, exemplarily, when there is other software to process the projection data, the data transmission system in the present application is triggered to execute the data transmission process of the present application, which can be triggered by the user through a specified interactive interface, or by the user touching a specific hardware button of the electronic device; similarly, the target projection server is triggered to execute a process adapted to the data transmission process, which can also be triggered by the user through an interface or by the user touching a specific hardware button of the wireless projection device; and the first optional implementation method and the second optional implementation method mentioned above can be selected by the user according to the actual application scenario, for example: select any implementation method through a specified interactive interface, or select any implementation method by touching different hardware buttons.

[0056] In one implementation, the screen projection device can transmit code stream data to a screen projection server running in an electronic device and communicating with the screen projection device through a network. The target screen projection server can decode and process the code stream data to obtain target screen projection data and write the target screen projection data into the target memory. Of course, this implementation is only an exemplary introduction and the embodiments of the present application do not specifically limit this.

[0057] In addition, the positioning of the first transmission unit 110 can be considered as a “bridge”, which can serve as a “bridge” between the target screen projection server and the second transmission unit 120 .

[0058] Exemplarily, in one implementation, the first transmission unit 110 is also used to detect that target screen projection data is written in the target memory, and before sending a data read instruction to the second transmission unit 120, make a memory application to obtain the target memory, and pass the target information of the target memory to the target screen projection server.

[0059] It can be understood that in this implementation, the target memory can be the memory applied for by the first transmission unit 110 to the electronic device. After obtaining the target memory, the target information indicating the address of the target memory can be transmitted to the target screen projection server, so that the target screen projection server responds to the target screen projection data transmitted by the projection device, and writes the target screen projection data to the target memory indicated by the target information. The first transmission unit 110 can detect that the target screen projection data is written in the target memory, thereby executing the subsequent steps. Through this implementation, the target content is applied for immediately by the first transmission unit 110, which can effectively utilize the memory space of the electronic device relative to setting a fixed memory space in advance in the electronic device as the target memory. Of course, in other implementations, the target memory may not be applied for by the first transmission unit 110, but may be a memory applied for in advance by the user for storing projection data. This application does not make specific restrictions on this; and, when the memory space of the electronic device is sufficient, a fixed memory space can also be allocated in the electronic device as the target memory, which is also reasonable.

[0060] It can be understood that after receiving the data reading instruction sent by the first transmission unit 110, the second transmission unit 120 can read the target screen projection data from the target memory based on the target information carried in the data reading instruction, and directly connect the target screen projection data to the operating system platform of the electronic device, wherein the operating system platform can be a variety of platforms, for example: AOSP platform, IOS platform, etc., and this application does not make specific limitations on this; then, the target software responds to the processing task for the projection data and can call the specified interface of the operating system platform to obtain the projection data required for the processing task for executing the processing task; wherein the target software can be software that processes the target projection data, and the processing task is determined according to the scenario. In the video conferencing scenario, the processing task can be a projection data forwarding task, and in the data recording scenario, the processing task can be a projection data upload task. The embodiment of this application does not make specific limitations on the processing task.

[0061] For the sake of clear layout, the process of the second transmission unit 120 connecting the target projection data to the operating system platform of the electronic device and the subsequent target software calling the specified interface of the operating system platform will be introduced in other embodiments and will not be elaborated here.

[0062] It can be seen that when the screen projection data is used for various purposes in the embodiments of the present application, there is no need for the hardware manufacturer of the screen projection device to cooperate deeply with the software manufacturer to adapt the screen projection device to the software (for example, the manufacturer of the screen projection device opens a private interface for the software manufacturer). Instead, the target screen projection server can write the screen projection data to the target memory. In this way, the data transmission system connects the screen projection data written in the target memory to the operating system platform of the electronic device. The interface of the operating system platform is a public interface and can be called by various software. In addition, various software can directly call the specified interface of the operating system platform of the electronic device to obtain the screen projection data required for processing tasks, so as to use the screen projection data for various purposes. It can be seen that this solution can reduce the limitations of the processing of screen projection data.

[0063] Furthermore, in the process of processing the projection data in the embodiment of the present application, the projection data is not copied, and the projection data is only transmitted through the target memory, which also reduces the consumption of storage resources.

[0064] Optionally, in one implementation, the second transmission unit includes: a camera hardware abstraction layer camera HAL process unit and an audio hardware abstraction layer audio HAL process unit, and the target screen projection data includes target image data and / or target audio data;

[0065] The first transmission unit sending a data read instruction to the second transmission unit may include step A1:

[0066] Step A1, the first transmission unit sends a data read instruction to the camera HAL process unit and the audio HAL process unit respectively;

[0067] Correspondingly, the second transmission unit reads the target projection data in the target memory based on the target information carried in the data reading instruction, and connects the target projection data to the operating system platform of the electronic device, which may include steps B1 and / or B2:

[0068] Step B1, the camera HAL process unit reads the target image data in the target memory based on the target information carried in the data reading instruction, and connects the read target image data to the operating system platform of the electronic device;

[0069] and / or,

[0070] Step B2, the audio HAL process unit reads the target audio data in the target memory based on the target information carried in the data reading instruction, and connects the read target audio data to the operating system platform of the electronic device;

[0071] The camera HAL process unit connects the read target image data to the operating system platform of the electronic device, including step B11:

[0072] Step B11, the camera HAL process unit connects the read target image data to the operating system platform in the form of data of a designated camera node; wherein the designated camera node is a virtual camera node in the operating system platform;

[0073] The audio HAL process unit connects the read target audio data to the operating system platform of the electronic device, including step B21:

[0074] Step B21, the audio HAL process unit connects the read target audio data to the operating system platform in the form of data of a specified audio node; wherein the specified audio node is a virtual audio node in the operating system platform. It should be emphasized that the camera HAL process unit and the audio HAL process unit are independent process units and do not interfere with each other, wherein the camera HAL process unit is for image data, and the audio HAL process unit is for audio data; and, in this implementation, the target projection data may include target image data, may include target audio data, or may include both target image data and target audio data.

[0075] It can be understood that, for step A1, since the camera HAL process unit and the audio HAL process unit do not interfere with each other, the first transmission unit can send data reading instructions to the camera HAL process unit and the audio HAL process unit respectively; and, when the target image data does not exist in the target projection data, the camera HAL process unit does not need step B1, and when the target audio data does not exist in the target projection data, the audio HAL process unit does not need step B2.

[0076] It can be understood that for step B1, in this case the target projection data includes target image data. At this time, the camera HAL process unit can read the target image data from the target memory based on the target information carried in the data reading instruction, and connect the read target image data to the operating system platform of the electronic device.

[0077] It can be understood that for step B2, in this case, the target screen projection data includes target audio data. At this time, the audio HAL process unit can read the target audio data from the target memory based on the target information carried in the data reading instruction, and connect the read target audio data to the operating system platform of the electronic device. And, when the target screen projection data includes target image data and target audio data, step B1 and step B2 can be executed at the same time.

[0078] It can be understood that, for step B11, when the operating system platform identifies the image data, it can only identify the data of the designated camera node. Therefore, when the camera HAL process unit connects the read target image data to the operating system platform of the electronic device, it is necessary to connect the read target image data in the form of data of the designated camera node, and the designated camera node is a virtual camera node in the operating system platform; the target image data can be one channel of image data or multiple channels of image data, each channel of image data can correspond to a designated camera node, and the operating system platform has two designated camera nodes by default, the camera of the electronic device can correspond to one of the designated camera nodes, and the target image data can correspond to the other designated camera node, and the designated camera node corresponding to the target image data can be considered as a virtual camera node. When the target image data is one channel of image data, there is no need to add a designated camera node, and when the target image data is multiple channels of image data, the designated camera node can be dynamically added. When the target software ends the processing of the projection data, the dynamically added designated camera node can also be dynamically eliminated. Exemplarily, in the scenario of video conferencing, a designated camera node of the operating system platform can correspond to the image data taken by the camera of the PC device, and another designated camera node of the operating system platform can correspond to the image data that has been projected. For a clear layout, the generation of the designated camera node may be introduced in other embodiments and will not be elaborated on here.

[0079] It can be understood that for step B21, when the operating system platform identifies audio data, it can only identify the data of the specified audio node. Therefore, when the audio HAL process unit connects the read target audio data to the operating system platform of the electronic device, it is necessary to connect the read target audio data in the form of data of the specified audio node, and the specified audio node is a virtual audio node in the operating system platform; the target audio data is consistent with the target image data, which can be one channel of audio data or multiple channels of audio data. Each channel of audio data can correspond to a specified audio node, which is a virtual audio node, and the operating system platform has two specified audio nodes by default. The microphone of the electronic device can correspond to one of the specified audio nodes, and the target audio data can correspond to another specified audio node. When the target audio data is one channel of audio data, there is no need to add a specified audio node, and when the target audio data is multiple channels of audio data, it is necessary to dynamically add a specified audio node. When the target software ends the processing of the projection data, the dynamically added specified audio node can also be dynamically eliminated. Exemplarily, in the scene of video conferencing, a specified audio node of the operating system platform can correspond to the audio data collected by the microphone of the PC device, and another specified audio node of the operating system platform can correspond to the audio data that has been projected. For the sake of clarity, the generation of the specified audio node may be introduced in other embodiments and will not be elaborated on here.

[0080] It should be emphasized that in one implementation, the second transmission unit can also be a Linux driver module, which can also be called a Linux system driver. Then, the first transmission unit can send the target screen projection data directly to the Linux system driver, that is, the Linux driver module, and the camera HAL / audio HAL can also read the target screen projection data from the Linux system driver respectively. This implementation is only an exemplary introduction and does not specifically limit the embodiments of the present application. Among them, from the perspective of system architecture, the Linux driver module is a driver module located below the camera HAL and audio HAL, that is, the camera HAL and audio HAL can be considered to be located in the upper layer of the Linux driver module. It can be seen that the second transmission unit in the embodiment of the present application includes a camera HAL process unit and an audio HAL process unit. The first transmission unit can send data reading instructions to the camera HAL process unit and the audio HAL process unit respectively. If the target projection data includes target image data and target audio data, the camera HAL process unit and the audio HAL process unit can also simultaneously execute the reading of target image data and target audio data from the target memory, and connect the read data to the operating system platform of the electronic device, so as to facilitate the target software to obtain the target projection data from the operating system platform, thereby reducing the limitations of the processing of projection data. In addition, the camera HAL process unit connects the read target image data to the operating system platform in the form of data of the specified camera node, and the audio HAL process unit connects the read target audio data to the operating system platform in the form of data of the specified audio node, thereby realizing the access of the operating system platform to the image data and audio data, thereby facilitating the target software to obtain the target projection data from the operating system platform, thereby reducing the limitations of the processing of projection data.

[0081] Optionally, in another embodiment, the second transmission unit connects the target screen projection data to the operating system platform of the electronic device, including step C1, and / or step C2:

[0082] Step C1, connecting the target image data in the target projection data to the operating system platform in the form of data of a designated camera node; wherein the designated camera node is a virtual camera node in the operating system platform;

[0083] Step C2, connecting the target audio data in the target screen projection data to the operating system platform in the form of data of a designated audio node; wherein the designated audio node is a virtual audio node within the operating system platform.

[0084] It can be understood that step C1 is similar to step B1, and step C2 is similar to step B2. Steps C1 and C2 are the process of introducing the target image data and the target audio data to the operating system platform from the perspective of the second transmission unit;

[0085] Exemplarily, in one implementation, the target image data in the target projection data is connected to the operating system platform in the form of data of a specified camera node, including method C11:

[0086] Mode C11, when there is at least one channel of image data in the read target projection data, if the number of the at least one channel of image data is not greater than the first number of idle designated camera nodes, then for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data of an idle designated camera node; otherwise, for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data about an added designated camera node and one of the idle designated camera nodes; wherein the idle designated camera node is a node other than the designated camera node required for use by the camera of the electronic device among the multiple designated camera nodes that the operating system platform has by default, and the added designated camera node is a node added by the operating system platform after applying for the addition of the designated camera node to the operating system platform, and the number of the added designated camera nodes is the second number, which is the difference between the number of the at least one channel of image data and the first number;

[0087] and / or,

[0088] The method of connecting the target audio data in the target screen projection data to the operating system platform in the form of data of a specified audio node includes method C12:

[0089] Method C12, when there is at least one channel of audio data in the read target projection data, if the number of the at least one channel of audio data is not greater than the third number of idle designated audio nodes, then for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data of an idle designated audio node; otherwise, for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data about an added designated audio node and one of the idle designated audio nodes; wherein the idle designated audio node is a node among the multiple designated audio nodes that the operating system platform has by default, except the designated audio node required by the microphone of the electronic device, and the added designated audio node is added by the operating system platform after applying for the addition of the designated audio node to the operating system platform, and the number of the added designated camera nodes is the fourth number, and the fourth number is the difference between the number of the at least one channel of audio data and the third number.

[0090] It can be understood that, for method C11, there can be at least one channel of image data in the target screen projection data, the operating system platform has multiple designated camera nodes by default, the camera of the electronic device needs to use a designated camera node, and it can also be considered that the camera of the electronic device occupies a designated camera node. The designated camera nodes other than the designated camera nodes used by the camera of the electronic device can be considered to be idle designated camera nodes. There is at least one channel of image data in the target screen projection data that can use the idle designated camera node, and one channel of image data can access an idle designated camera node. Then, when the number of at least one channel of image data is less than or equal to the first number of idle designated camera nodes, for each channel of the at least one channel of image data, the data can be directly connected to the operating system platform in the form of data of an idle designated camera node, without any other operation; conversely, for each channel of data, the data can be connected to the operating system platform in the form of data about an added designated camera node and one of the idle designated camera nodes; wherein the added designated camera node is added by the operating system platform after an application for adding a designated camera node is made to the operating system platform, and the number of the added designated camera nodes is the second number, which is the sum of the number of at least one channel of image data and the first number. The difference in quantity; exemplarily, there is 1 channel of image data in the target projection data, and the first number of idle designated camera nodes is 2. At this time, the number of image data channels is not greater than the number of idle designated camera nodes, so the image data channel can be connected to the operating system platform in the form of data of an idle designated camera node; there are 3 channels of image data in the target projection data, and the first number of idle designated camera nodes is 2. At this time, the number of image data channels is greater than the number of idle designated camera nodes, so the second transmission unit can add a designated camera node to the operating system platform, so that each channel of image data can be connected to the operating system platform in the form of data of the designated camera node.

[0091] It should be noted that, in an optional implementation, while the target screen projection server writes the target screen projection data to the target memory, it can send a first notification message to the first transmission unit or the second transmission unit, and the first notification message is used to indicate the number of at least one image data in the target screen projection data written by the target screen projection server; thus, the first transmission unit or the second transmission unit, in response to receiving the first notification message, can analyze and determine the relationship between the number of at least one image data and the first number of idle designated camera nodes, and when it is analyzed that the number of at least one image data is greater than the first number of idle designated camera nodes, an application for adding designated camera nodes is made to the operating system platform, so that the operating system platform can add a second number of designated camera nodes. In this optional implementation, the method of applying for adding designated camera nodes can be called a pre-application method.

[0092] In another optional implementation, when the second transmission unit connects the target image data in the target projection data to the operating system platform in the form of data of the designated camera node, it can analyze and judge the relationship between the number of the at least one image data and the first number of idle designated camera nodes. When it is analyzed that the number of the at least one image data is greater than the first number of idle designated camera nodes, it applies to the operating system platform for adding designated camera nodes, so that the operating system platform can add a second number of designated camera nodes. In this optional implementation, the application method for adding designated camera nodes can be a method called real-time application.

[0093] In another optional implementation, after the screen projection device establishes a communication connection with the target screen projection server, the target screen projection server can know the number of at least one image data in the target projection data to be written into the target memory by the target screen projection server based on the number of connected screen projection devices and / or the number of screen projection channels applied for by the screen projection device; then, after the screen projection device establishes a communication connection with the target screen projection server, the target screen projection server can send a first notification message to the first transmission unit or the second transmission unit; thereby, the first transmission unit or the second transmission unit, in response to receiving the first notification message, can analyze and determine the relationship between the number of at least one image data and the first number of idle designated camera nodes, and when it is analyzed that the number of at least one image data is greater than the first number of idle designated camera nodes, an application for adding designated camera nodes is made to the operating system platform, so that the operating system platform can add a second number of designated camera nodes. It can be seen that in this optional implementation method, before the target screen projection server writes the target screen projection data to the target memory, an application for adding a specified camera node can be made, which greatly improves the processing efficiency.

[0094] In addition, it should be emphasized that the present application does not limit the specific implementation method of adding designated camera nodes to the operating system platform; any method of adding designated camera nodes can be applied to this solution. It can be understood that for method C12, there can be at least one audio data in the target screen projection data, the operating system platform has multiple designated audio nodes by default, the microphone of the electronic device needs to use a designated audio node, and it can also be considered that the microphone of the electronic device occupies a designated audio node. The designated audio nodes other than the designated audio nodes used by the microphone of the electronic device can be considered to be idle designated audio nodes. There is at least one audio data in the target screen projection data that can use the idle designated audio node, and one audio data can be connected to an idle designated audio node.

[0095] Then, when the number of at least one channel of audio data is less than or equal to the third number of idle designated audio nodes, for each channel of the at least one channel of audio data, the data can be directly connected to the operating system platform in the form of data of an idle designated audio node, without any other operation; conversely, for each channel of data, the data can be connected to the operating system platform in the form of data about the added designated audio node and one of the idle designated audio nodes; wherein the added designated audio node is added by the operating system platform after an application for adding a designated audio node is made to the operating system platform, and the number of the added designated audio nodes is the fourth number, which is the sum of the number of at least one channel of audio data and the third number. The difference between the three quantities; exemplarily, there is 1 channel of audio data in the target screen projection data, and the third number of idle designated audio nodes is 2. At this time, the number of audio data channels is not greater than the number of idle designated audio nodes, so the audio data channel can be connected to the operating system platform in the form of data of an idle designated audio node; there are 3 channels of audio data in the target screen projection data, and the third number of idle designated audio nodes is 2. At this time, the number of audio data channels is greater than the number of idle designated audio nodes, so the second transmission unit can add a designated audio node to the operating system platform, so that each channel of audio data can be connected to the operating system platform in the form of data of a designated audio node.

[0096] It should be noted that, in an optional implementation, while the target screen projection server writes the target screen projection data to the target memory, a second notification message can be sent to the first transmission unit or the second transmission unit, and the second notification message is used to indicate the number of at least one audio data in the target screen projection data written by the target screen projection server; thus, the first transmission unit or the second transmission unit, in response to receiving the second notification message, can analyze and determine the relationship between the number of at least one audio data and the third number of idle designated audio nodes, and when it is analyzed that the number of at least one audio data is greater than the third number of idle designated audio nodes, an application for the addition of designated audio nodes is made to the operating system platform, so that the operating system platform can add a fourth number of designated audio nodes. In this optional implementation, the method of applying for the addition of designated audio nodes may be called a pre-application method.

[0097] In another optional implementation, when the second transmission unit connects the target audio data in the target projection data to the operating system platform in the form of data of a designated audio node, it can analyze and judge the relationship between the number of the at least one audio data channel and the third number of idle designated camera nodes. When it is analyzed that the number of the at least one audio data channel is greater than the third number of idle designated camera nodes, an application for adding designated audio nodes is made to the operating system platform, so that the operating system platform can add a fourth number of designated audio nodes. In this optional implementation, the method of applying for adding designated audio nodes can be a method called real-time application.

[0098] In another optional implementation, after the screen projection device establishes a communication connection with the target screen projection server, the target screen projection server can know the number of at least one audio data in the target screen projection data to be written into the target memory by the target screen projection server based on the number of connected screen projection devices and / or the number of screen projection channels applied for by the screen projection device; then, after the screen projection device establishes a communication connection with the target screen projection server, the target screen projection server can send a second notification message to the first transmission unit or the second transmission unit; thereby, the first transmission unit or the second transmission unit, in response to receiving the second notification message, can analyze and determine the relationship between the number of at least one audio data and the third number of idle designated audio nodes, and when it is analyzed that the number of at least one audio data is greater than the third number of idle designated audio nodes, an application for adding designated audio nodes is made to the operating system platform, so that the operating system platform can add a fourth number of designated audio nodes. It can be seen that in this optional implementation method, before the target screen projection server writes the target screen projection data to the target memory, an application for adding a specified audio node can be made, which greatly improves the processing efficiency.

[0099] In addition, it should be emphasized that the present application does not limit the specific implementation method of adding designated audio nodes to the operating system platform; any method of adding designated audio nodes can be applied to this solution.

[0100] In addition, it can be understood that in an optional implementation method, multiple screen projection devices can be connected to the same target screen projection server, so that one channel of projection data of each screen projection device can be transmitted to the target screen projection server, and the target screen projection data written by the target screen projection server to the target memory can include the projection data transmitted by each screen projection device; since the projection data transmitted by each screen projection device can include one channel of image data and / or one channel of audio data, the target screen projection data read from the target memory can contain at least one channel of image data and / or at least one channel of audio data. For example, in a video conferencing scenario, device 1 is connected to screen projection device a, device 2 is connected to screen projection device b, and device 3 is connected to screen projection device c. Screen projection device a, screen projection device b, and screen projection device c can all communicate with the target screen projection server. The user of device 1, the user of device 2, and the user of device 3 all want to project the audio and video data output on the device. Therefore, the screen projection devices connected to device 1, device 2, and device 3 can all transmit audio and video data to the target screen projection server. The target screen projection server can receive multiple channels of image data and multiple channels of audio data as target screen projection data.

[0101] In another optional implementation, a screen projection device is connected to a target screen projection server, but the screen projection device can also transmit one or more screen projection data to the target screen projection server, and the target screen projection data written by the target screen projection server to the target memory can include each channel of screen projection data transmitted by the screen projection device; since each channel of screen projection data transmitted by the screen projection device can include one channel of image data and / or one channel of audio data, the target screen projection data read from the target memory can contain at least one channel of image data and / or at least one channel of audio data. Exemplarily, in a video conference scenario, there are two windows on the device 1 connected to the screen projection device, one of which is a slide show window and the other is a word document window. At this time, the user of device 1 wants to screen the displayed slides and word documents. The screen projection device connected to device 1 can send the two channels of image data of the displayed slides and documents to the target screen projection server. At this time, the target screen projection server can receive the two channels of image data as target screen projection data.

[0102] Of course, the two implementation methods introduced above are exemplary introductions to the scenario in which the target projection data read from the target memory contains at least one image data and / or at least one audio data. Of course, in the case where a projection device can be connected to multiple laptops and / or mobile phones, a projection device can also transmit multiple projection data, so that there can be at least one image data and / or at least one audio data in the target projection data.

[0103] In addition, exemplarily, in one implementation, the newly added designated camera nodes and / or newly added designated audio nodes may also be deleted. For example, when the operating system platform detects that the newly added designated camera nodes / designated audio nodes have not been used for a predetermined period of time, the operating system platform may actively delete the detected unused designated camera nodes / designated audio nodes. The specific implementation method for deleting the newly added designated camera nodes and / or newly added designated audio nodes is not limited to the above-mentioned method. For example, in other implementations, the second transmission unit in the data transmission system may, after detecting that the transmission of any channel of image data stops, if the designated camera node to which the image data belongs is an newly added node, then the second transmission unit may request the operating system platform to delete the designated camera node to which the image data belongs. Similarly, after detecting that the transmission of any channel of audio data stops, if the designated audio node to which the audio data belongs is an newly added node, then the second transmission unit may request the operating system platform to delete the designated audio node to which the audio data belongs.

[0104] Exemplarily, in one implementation, the correspondence between each channel of image data and the designated camera node to which each channel of image data belongs, and the correspondence between each channel of audio data and the designated audio node to which each channel of audio data belongs can be recorded. The correspondence can be recorded in a data table, so that based on the data table, it is ensured that each channel of image data is continuously connected to the operating system platform in the form of data of the same designated camera node, and each channel of audio data is continuously connected to the operating system platform in the form of data of the same designated audio node.

[0105] In addition, it can be understood that for the image data and audio data in the projection data of the same window of the same projection device, the designated camera nodes and designated audio nodes respectively connected thereto can have a corresponding relationship, so that after the target software reads the data of the designated camera nodes and designated audio nodes with a corresponding relationship, the read data can be output through a window.

[0106] It can be seen that the embodiments of the present application can determine whether it is necessary to add a designated camera node based on the quantitative relationship between the number of at least one channel of image data and the first number of idle designated camera nodes; and can determine whether it is necessary to add a designated audio node based on the quantitative relationship between the number of at least one channel of audio data and the third number of idle designated audio nodes, thereby ensuring that each channel of image data and audio data can be normally connected to the operating system platform, reducing the limitations on the processing of projection data.

[0107] Based on steps C1 and C2 in the above embodiment, the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, obtains the projection data required for the processing task for executing the processing task, and can include steps D1-D3:

[0108] Step D1, the target software requests a node list from the operating system platform in response to a processing task for projection data; wherein the node list includes a camera node and an audio node that have been connected to the operating system platform;

[0109] It can be understood that the target software responds to the processing task for the projection data, that is, when the target software processes the projection data, it can request a node list from the operating system platform. It has been introduced above that the camera HAL process unit connects the read target image data to the operating system platform in the form of data of the specified camera node, and the audio HAL process unit connects the read target audio data to the operating system platform in the form of data of the specified audio node. Therefore, the target image data and the target audio data can be considered to be connected to the operating system platform in the form of node data; the operating system platform can represent the nodes that have been connected to the operating system platform in the form of a node list, and the node list can include camera nodes and audio nodes that have been connected to the operating system platform.

[0110] And, the node list may also include a camera node corresponding to a camera built into the electronic device, and an audio node corresponding to a microphone built into the electronic device. This application does not specifically limit the nodes in the node list. Exemplarily, the video conferencing software may request a node list from the Android open source platform AOSP, and the node list includes camera node 1, camera node 2, audio node 1, and audio node 2, wherein camera node 1 is a designated camera node, the image data of camera node 1 is the image data in the projection data, audio node 1 is a designated audio node, and the audio data of audio node 1 is the audio data in the projection data.

[0111] Step D2, determining the designated camera node and / or the designated audio node from the node list to obtain a target node;

[0112] It is understandable that after obtaining the node list provided by the operating system platform, the target software can select a specified camera node and / or a specified audio node from the node list, that is, select the node corresponding to the projection data; illustratively, the video conferencing software is installed on a mobile device, and the node list includes camera node 1 and camera node 2, wherein the image data of camera node 1 is the image data in the projection data, and the image data of camera node 2 is the image data taken by the camera of the mobile device, then the video conferencing software can determine camera node 1 as the target node from the node list. It should be emphasized that the specified camera node determined in step D2 is the camera node excluding the node corresponding to the camera of the electronic device, and the specified audio node determined is the audio node excluding the node corresponding to the microphone of the electronic device. In one implementation, the target software may also select a node from a node list as a target node based on the requirements of the processing task; when the target software executes the processing task, only the target image data needs to be used, then only the camera node in the node list may be selected as the target node. Similarly, when the target software executes the processing task, only the target audio data needs to be used, then only the audio node in the node list may be selected as the target node. The embodiment of the present application does not make any specific limitation on this.

[0113] Step D3, calling the designated interface of the operating system platform to obtain the data of the target node through the designated interface, and obtain the screen projection data required for the processing task to execute the processing task.

[0114] It should be emphasized that when the target software calls data, it can only call from the designated interface of the operating system platform. The designated interface is the interface provided by the operating system platform for the target software to call data. The target software can directly obtain the data of the target node through the designated interface, thereby obtaining the projection data required for the processing task. The subsequent target software can use the projection data to perform the processing task. Among them, the designated interface can be a standard interface of the operating system platform, for example: in the Android system, the designated interface can be the standard interface of Andorid, that is, the camera API; the embodiment of the present application does not specifically limit the specific form of the designated interface.

[0115] It can be seen that the target software can obtain the projection data required for processing tasks by calling the specified interface of the operating system platform. There is no need for the projection device to establish an adaptation relationship with the target software, which simplifies the process of transmitting the projection data to the target software, thereby reducing the limitations of the processing of projection data.

[0116] In one implementation, the operating system platform is the Android open source platform AOSP;

[0117] The first transmission unit sending a data read instruction to the second transmission unit may include step E1:

[0118] Step E1, sending a data read instruction to the second transmission unit through a binder connection between the first transmission unit and the second transmission unit;

[0119] The method for establishing the binder connection between the first transmission unit and the second transmission unit includes method F1:

[0120] Mode F1: The second transmission unit requests the binder driver to add a binder service between the first transmission unit and the second transmission unit, so that the binder driver establishes a binder connection between the first transmission unit and the second transmission unit.

[0121] It can be understood that, for step E1, when the operating system platform is the Android open source platform AOSP, there is a binder connection between the first transmission unit and the second transmission unit. Through the binder connection, the first transmission unit can send a data read instruction to the second transmission unit. The binder connection is a communication method suitable for the operating system platform being AOSP, which can enable different process units to communicate. When the operating system platform is other platforms, other methods can also be used to send data read instructions. The embodiments of the present application do not specifically limit this. In addition, the binder connection between the first transmission unit and the second transmission unit can be pre-established, and the specific process can be introduced in subsequent embodiments.

[0122] It can be understood that for step F1, the binder driver will establish a binder connection between the sender process unit and the receiver process unit, and manage the state of the binder connection. This process can also be called maintaining the state of the bindre connection. In addition, the second transmission unit can send a new binder service request to the binder driver in advance, requesting to add a binder service between the first transmission unit and the second transmission unit. In response to the request, the binder driver adds a binder connection between the first transmission unit and the second transmission unit in the second transmission unit, and the service node for the binder service between the first transmission unit and the second transmission unit can be dev / binder.

[0123] It can be seen that when the operating system platform is the Android open source platform AOSP, a data read instruction can be sent to the second transmission unit through the binder connection between the first transmission unit and the second transmission unit, so that the first transmission unit and the second transmission unit can communicate. In addition, the second transmission unit can request the binder driver to add a binder service between the first transmission unit and the second transmission unit, so that the binder driver establishes a binder connection between the first transmission unit and the second transmission unit, so that the first transmission unit can subsequently send a data read instruction to the second transmission unit, so that the first transmission unit and the second transmission unit can communicate.

[0124] In one implementation, the first transmission unit transmits the target information in the target memory to the target screen projection server, which may include step G1:

[0125] Step G1, transmitting the target information in the target memory to the target screen projection server through a binder connection between the first transmission unit and the target screen projection server;

[0126] The method for establishing a binder connection between the first transmission unit and the target screen projection server includes method H1:

[0127] Mode H1, the target screen projection server requests the binder driver to add a binder service between the first transmission unit and the target screen projection server, so that the binder driver establishes a binder connection between the first transmission unit and the target screen projection server.

[0128] It can be understood that for step G1, when the operating system platform is the Android open source platform AOSP, there is a binder connection between the first transmission unit and the target screen projection server. Through the binder connection, the first transmission unit can send the target information of the target memory to the target screen projection server. The binder connection is a communication method suitable for the operating system platform being AOSP, which can enable different process units to communicate. When the operating system platform is other platforms, other methods can also be used to send data reading instructions. The embodiments of the present application do not specifically limit this. In addition, the binder connection between the first transmission unit and the target screen projection server can be pre-established, and the specific process can be introduced in subsequent embodiments.

[0129] It should be emphasized that the first transmission unit targeted by step G1 is a process unit. Then, when the specific form of the first transmission unit is a module embedded in the target screen projection server, there is no need to use the binder connection between the first transmission unit and the target screen projection server to realize the transmission of the target information of the target memory to the target screen projection server, and there is no need to verify the first transmission unit, which simplifies the process of sending the target screen projection data. However, in this implementation method, the target screen projection server needs to be adapted to AOSP, and the coupling is relatively large; on the contrary, when the specific form of the first transmission unit is not a module embedded in the target screen projection server, the first transmission unit actively adapts to the target screen projection server, and the first transmission unit and AOSP are also pre-adapted, and the target screen projection server can run in any system, and the coupling is smaller.

[0130] It can be understood that for method H1, the binder driver will establish a Binder connection between the sender process unit and the receiver process unit, and manage the state of the binder connection. This process can also be called maintaining the state of the bindre connection. Also, the target screen projection server can send a new binder service request to the binder driver in advance, requesting to add a binder service between the first transmission unit and the target screen projection server. Then, the first transmission unit can actively connect to the binder service. At this time, the first transmission unit can act as a connection client, so that the target screen projection server can verify the connection client. The specific verification process can be introduced in subsequent embodiments and will not be repeated here; and, the service node for the binder service between the first transmission unit and the target screen projection server can be dev / binder.

[0131] Exemplarily, in another implementation method, if the first transmission unit sends a new binder service request to the binder driver, requesting to add a new binder service between the first transmission unit and the target screen projection server, then the target screen projection server can actively connect to the binder service. At this time, the target screen projection server can act as a connection client. After the connection is successful, it is necessary to use an additional interface to implement verification of the first transmission unit. It can be understood that when the target screen projection server sends a new binder service request to the binder driver, the first transmission unit can actively connect to the binder service, and when the first transmission unit sends a new binder service request to the binder driver, the target screen projection server can actively connect to the binder service. When any unit actively connects to the binder service, the unit can act as a connection client.

[0132] It can be seen that when the operating system platform is the Android open source platform AOSP, the target information of the target memory can be sent to the target screen projection server through the binder connection between the first transmission unit and the target screen projection server, so that the first transmission unit and the target screen projection server can communicate. In addition, the target screen projection server can request the binder driver to add a binder service between the first transmission unit and the target screen projection server, so that the binder driver establishes a binder connection between the first transmission unit and the target screen projection server, which is convenient for the first transmission unit to send the target information of the target memory to the target screen projection server, so that the first transmission unit and the target screen projection server can communicate.

[0133] In one implementation, the first transmission unit is also used to send a signature certificate to the target screen projection server before transmitting the target information in the target memory to the target screen projection server, so that the target screen projection server performs signature verification based on the signature certificate, and after the signature verification passes, feedbacks result information representing the verification pass to the first transmission unit; after receiving the result information representing the verification pass, the target information in the target memory is transmitted to the target screen projection server.

[0134] It can be understood that, based on security considerations, the first transmission unit transmits the target information of the target memory to the target screen projection server, and can send a signed certificate to the target screen projection server. The signed certificate can indicate that the first transmission unit has the authority to call the target screen projection data. The target screen projection server can verify the signed certificate. After the signature verification is passed, it indicates that the target screen projection data can be written to the target memory. At this time, the result information representing the verification passed can be fed back to the first transmission unit; and, after receiving the result information representing the verification passed, the first transmission unit can send the target information of the target memory to the target screen projection server, so that the target screen projection server responds to the target information of the target memory sent by the first transmission unit and writes the target screen projection data into the target memory.

[0135] In addition, the signature certificate of the first transmission unit may be granted in advance by the screen projection device to the first transmission unit, so as to prevent other process units from calling the target screen projection data. Of course, it may also be granted by the electronic device to the first transmission unit, and the embodiment of the present application does not make specific limitations on this; and, the signature certificate sent by the first transmission unit to the target screen projection server may be in the form of a key or in other forms, and the embodiment of the present application does not make specific limitations on this.

[0136] It can be seen that before the first transmission unit transmits the target information of the target memory to the target screen projection server, it can send a signature certificate to the target screen projection server. The target screen projection server can verify the signature certificate. After the signature verification is passed, it will feedback the result information indicating the verification is passed to the first transmission unit. Then, the target screen projection server ensures that the first transmission unit has the authority to call the target screen projection data before it can write the target screen projection data into the target memory, thereby ensuring the security of the target screen projection data transmission.

[0137] In one implementation, the data reading instruction carries a signature certificate of the first transmission unit;

[0138] Correspondingly, the second transmission unit is also used to verify the signature certificate carried by the data reading instruction before reading the target projection data in the target memory based on the target information carried in the data reading instruction. If the verification is successful, the step of reading the target projection data in the target memory based on the target information carried in the data reading instruction is executed.

[0139] It can be understood that the above embodiment is that the target screen projection server verifies the signature certificate sent by the first transmission unit, while this embodiment is that the second transmission unit verifies the signature certificate of the first transmission unit. The signature certificate of the first transmission unit is carried in the data reading instruction, and the signature certificate can indicate that the first transmission unit has the authority to call the target screen projection data. Before the second transmission unit reads the target screen projection data in the target memory, the signature certificate carried by the data reading instruction can be verified. After the signature verification is passed, it indicates that the target screen projection data can be read from the target memory, and the step of reading the target screen projection data in the target memory based on the target information carried in the data reading instruction can be executed.

[0140] In addition, the signature certificate of the first transmission unit may be granted in advance by the projection device to the first transmission unit, so as to prevent other process units from calling the target projection data. Of course, it may also be granted by the electronic device to the first transmission unit, and the embodiment of the present application does not make specific limitations on this; and the specific form of the signature certificate may be a key or other forms, and the embodiment of the present application does not make specific limitations on this.

[0141] It can be seen that the second transmission unit can verify the signature certificate carried by the data reading instruction before reading the target screen projection data in the target memory based on the target information carried in the data reading instruction. Only after the signature verification is passed can the step of reading the target screen projection data in the target memory be executed; then, the second transmission unit can read the target screen projection data from the target memory only when it is ensured that the first transmission unit has the authority to call the target screen projection data, thereby ensuring the security of the target screen projection data transmission.

[0142] In order to better understand the data transmission system used in electronic devices, the working principle of the data transmission system is introduced below. Figure 2 As shown:

[0143] The electronic device may include a target screen projection server, a first transmission unit, a second transmission unit, an operating system platform and target software, wherein the second transmission unit may include a camera HAL and a voice HAL, the camera HAL is the camera HAL process unit in the above embodiment, and the voice HAL is the audio HAL process unit in the above embodiment; the screen projection device may send target screen projection data to the target screen projection server through the network. In addition, the first transmission unit and the second transmission unit are both process units in the data transmission system, the camera HAL may include a HAL video server, that is, a HAL video server, and the voice HAL may include a HAL audio server, that is, a HAL voice server.

[0144] After receiving the target screen projection data, the target screen projection server can write the target screen projection data in the target memory. After detecting that the target screen projection data is written in the target memory, the first transmission unit can send data reading instructions to the camera HAL and voice HAL in the second transmission unit respectively. The camera HAL reads the target image data in the target memory based on the target information carried in the data reading instruction, and connects the read target image data to the operating system platform of the electronic device. The voice HAL reads the target audio data in the target memory based on the target information carried in the data reading instruction, and connects the read target audio data to the operating system platform of the electronic device. The subsequent target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task, that is, the target image data and the target disk audio data, for executing the processing task.

[0145] It can be seen that when the screen projection data is used for various purposes in the embodiments of the present application, there is no need for the hardware manufacturer of the screen projection device to cooperate deeply with the software manufacturer to adapt the screen projection device to the software (for example, the manufacturer of the screen projection device opens a private interface for the software manufacturer). Instead, the target screen projection server can write the screen projection data to the target memory. In this way, the data transmission system connects the screen projection data written in the target memory to the operating system platform of the electronic device. The interface of the operating system platform is a public interface and can be called by various software. In addition, various software can directly call the specified interface of the operating system platform of the electronic device to obtain the screen projection data required for processing tasks, so as to use the screen projection data for various purposes. It can be seen that this solution can reduce the limitations of the processing of screen projection data.

[0146] Furthermore, in the process of processing the projection data in the embodiment of the present application, the projection data is not copied, and the projection data is only transmitted through the target memory, which also reduces the consumption of storage resources.

[0147] A data transmission method provided by an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0148] like Figure 3 As shown, a data transmission method provided in an embodiment of the present application is applied to an electronic device; the method includes:

[0149] S301, detecting that target screen projection data is written into a target memory, and reading the target screen projection data in the target memory;

[0150] Wherein, the target memory is a memory into which the target screen projection data is written by the target screen projection server, the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is the data transmitted by the screen projection device;

[0151] S302, connecting the target screen projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the screen projection data, calls the specified interface of the operating system platform, and obtains the screen projection data required for the processing task for executing the processing task.

[0152] Optionally, before detecting that target screen projection data is written into the target memory and reading the target screen projection data in the target memory, the method further includes:

[0153] Apply for target memory;

[0154] The target information of the target memory is transmitted to the target screen projection server; wherein the target address information is used to indicate the address of the target memory;

[0155] The target information in the target memory is transmitted to the target screen projection server, including:

[0156] By establishing the obtained binder connection, the target information in the target memory is transmitted to the target screen projection server; wherein the method of establishing the obtained binder connection includes: requesting a binder driver to add a binder service to establish the connection through the binder driver;

[0157] The method further comprises:

[0158] Before the target information in the target memory is passed to the target screen projection server, a signature certificate is sent to the target screen projection server so that the target screen projection server performs signature verification based on the signature certificate, and after the signature verification passes, feedback is given indicating that the verification passed; after receiving the result information indicating that the verification passed, the target information in the target memory is passed to the target screen projection server.

[0159] Optionally, the target screen projection data includes target image data and / or target audio data;

[0160] Reading the target screen projection data in the target memory includes:

[0161] Sending a data read instruction carrying target information to a camera hardware abstraction layer camera HAL process unit and an audio hardware abstraction layer audio HAL process unit, so that the camera HAL process unit reads the target image data in the target memory based on the target information carried in the data read instruction, and the audioHAL process unit reads the target audio data in the target memory based on the target information carried in the data read instruction; the target information is used to indicate the address of the target memory;

[0162] The step of connecting the target screen projection data to the operating system platform of the electronic device includes:

[0163] Accessing the read target image data to the operating system platform through the camera HAL process unit;

[0164] The read target audio data is connected to the operating system platform through the audio HAL process unit.

[0165] Optionally, the step of connecting the target screen projection data to an operating system platform of the electronic device includes:

[0166] Connecting the target image data in the target screen projection data to the operating system platform in the form of data of a designated camera node; wherein the designated camera node is a virtual camera node in the operating system platform;

[0167] and / or,

[0168] The target audio data in the target screen projection data is connected to the operating system platform in the form of data of a designated audio node; wherein the designated audio node is a virtual audio node within the operating system platform.

[0169] Optionally, the target image data in the target projection data is connected to the operating system platform in the form of data of a specified camera node, including:

[0170] In the case where there is at least one channel of image data in the read target screen projection data, if the number of the at least one channel of image data is not greater than the first number of idle designated camera nodes, then for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data of an idle designated camera node; otherwise, for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data about an added designated camera node and one of the idle designated camera nodes; wherein the idle designated camera node is a node other than the designated camera node required for use by the camera of the electronic device among the multiple designated camera nodes that the operating system platform has by default, and the added designated camera node is a node that is added by the operating system platform after applying for the addition of the designated camera node to the operating system platform, and the number of the added designated camera nodes is a second number, which is the difference between the number of the at least one channel of image data and the first number;

[0171] and / or,

[0172] The method of connecting the target audio data in the target screen projection data to the operating system platform in the form of data of a specified audio node includes:

[0173] In the case where there is at least one channel of audio data in the read target screen projection data, if the number of the at least one channel of audio data is not greater than the third number of idle designated audio nodes, then for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data of an idle designated audio node; otherwise, for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data about an added designated audio node and one of the idle designated audio nodes; wherein the idle designated audio node is a node among the multiple designated audio nodes that the operating system platform has by default, except for the designated audio node required to be used by the microphone of the electronic device, and the added designated audio node is added by the operating system platform after applying for the addition of the designated audio node to the operating system platform, and the number of the added designated camera nodes is a fourth number, and the fourth number is the difference between the number of the at least one channel of audio data and the third number;

[0174] and / or,

[0175] The target software, in response to a processing task for projection data, calls a specified interface of the operating system platform to obtain the projection data required for the processing task for executing the processing task, including:

[0176] The target software requests a node list from the operating system platform in response to a processing task for projection data; wherein the node list includes a camera node and an audio node that have been connected to the operating system platform;

[0177] Determine, from the node list, a designated camera node having image data in the target projection data and / or a designated audio node having image data in the target projection data, and obtain a target node;

[0178] The designated interface of the operating system platform is called to obtain the data of the target node through the designated interface, and obtain the screen projection data required for the processing task to execute the processing task.

[0179] Optionally, the operating system platform is the Android open source platform AOSP; the data reading instruction is sent by establishing a binder connection; the established binder connection is established by requesting a binder driver to add a binder service and establishing the connection through the binder driver;

[0180] and / or,

[0181] The data read instruction carries a signature certificate. Accordingly, the camera HAL process unit verifies the signature certificate carried by the data read instruction before reading the target image data in the target memory based on the target information carried in the data read instruction. If the verification succeeds, the step of reading the target image data in the target memory based on the target information carried in the data read instruction is executed. The audio HAL process unit verifies the signature certificate carried by the data read instruction before reading the target audio data in the target memory based on the target information carried in the data read instruction. If the verification succeeds, the step of reading the target audio data in the target memory based on the target information carried in the data read instruction is executed.

[0182] It can be seen that when the screen projection data is used for various purposes, the embodiment of the present application does not require the hardware manufacturer of the screen projection device to cooperate deeply with the software manufacturer to adapt the screen projection device to the software (for example, the manufacturer of the screen projection device opens a private interface for the software manufacturer), and the screen projection data written in the target memory is connected to the operating system platform of the electronic device. The interface of the operating system platform is a public interface and can be called by various software. Then, various software can directly call the specified interface of the operating system platform of the electronic device to obtain the screen projection data required for processing tasks, so as to use the screen projection data for various purposes. It can be seen that this solution can reduce the limitations of the processing of screen projection data.

[0183] Furthermore, in the process of processing the projection data in the embodiment of the present application, the projection data is not copied, and the projection data is only transmitted through the target memory, which also reduces the consumption of storage resources.

[0184] Optionally, in another embodiment, the present application also provides a conference all-in-one device, such as Figure 4 As shown, the above-mentioned data transmission system is provided in the all-in-one conference device.

[0185] It can be understood that the data transmission system runs on an all-in-one conference device, and the all-in-one conference device is suitable for an Android environment. The data transmission system can specifically include a first transmission unit and a second transmission unit.

[0186] Among them, the wireless screen projection device can be the screen projection device in the above embodiment, and the screen projection server can be the target screen projection server in the above embodiment; and, the screen projection server includes a wireless screen projection stream receiving device and a wireless screen projection stream decoding device, and the first transmission unit includes a wireless screen projection stream transfer device. The screen projection server can be written in Java language and can also run in an Android virtual machine environment. The first transmission unit can also be written in Java language and can also run in an Android virtual machine environment. The second transmission unit is written in C / C++ language and can run in a Linux environment.

[0187] The wireless screen projection device can send a data stream to the screen projection server through the network, the wireless screen projection stream receiving device in the screen projection server can receive the data stream, and the wireless screen projection stream decoding device can decode the data stream into the target screen projection data. At this time, the target screen projection data can be directly sent to AOSP to achieve local display; the target screen projection data can also be written to the target memory, and the wireless screen projection stream transfer device in the first transmission unit detects that the target screen projection data is written in the target memory, and can send data reading instructions to the camera HAL and voice HAL in the second transmission unit respectively; wherein, the screen projection server and the first transmission unit use binder for efficient communication, and the first transmission unit and the second transmission unit use binder for efficient communication. The binder driver can add a binder service to the camera HAL and voice HAL of the second transmission unit, and the service node can be dev / binder. In addition, the target memory can also be called a surface, and the screen projection data can be transferred between the screen projection server and the first transmission unit through the surface. The surface can also be created by the first transmission unit for efficient transmission between process units, for example: 4K image data can be transmitted.

[0188] Furthermore, the camera HAL reads the target image data in the target memory based on the target information carried in the data read instruction, and connects the read target image data to AOSP. The voice HAL reads the target audio data in the target memory based on the target information carried in the data read instruction, and connects the read target audio data to AOSP. Subsequently, various types of conference software respond to the forwarding task for the projection data, call the specified interface of AOSP, and obtain the projection data required for the forwarding task to perform the forwarding task for the projection data. It should be emphasized that the video kernel driver in the Linux kernel can be used to drive the camera HAL to read the target image data from the target memory, and the voice kernel driver can be used to drive the voice HAL to read the target audio data from the target memory; wherein, the video kernel driver can also be called V4L2, and the voice kernel driver can also be called Tiny ALSA.

[0189] It can be seen that when the screen projection data is used for various purposes in the embodiments of the present application, there is no need for the hardware manufacturer of the screen projection device to cooperate deeply with the software manufacturer to adapt the screen projection device to the software (for example, the manufacturer of the screen projection device opens a private interface for the software manufacturer). Instead, the target screen projection server can write the screen projection data to the target memory. In this way, the data transmission system connects the screen projection data written in the target memory to the operating system platform of the electronic device. The interface of the operating system platform is a public interface and can be called by various software. In addition, various software can directly call the specified interface of the operating system platform of the electronic device to obtain the screen projection data required for processing tasks, so as to use the screen projection data for various purposes. It can be seen that this solution can reduce the limitations of the processing of screen projection data.

[0190] Furthermore, in the process of processing the projection data in the embodiment of the present application, the projection data is not copied, and the projection data is only transmitted through the target memory, which also reduces the consumption of storage resources.

[0191] The present application also provides an electronic device, such as Figure 5 As shown, including:

[0192] Memory 501, used for storing computer programs;

[0193] The processor 502 is used to implement any of the above-mentioned data transmission methods when executing the program stored in the memory 501.

[0194] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 502, the communication interface, and the memory 501 communicate with each other via the communication bus.

[0195] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0196] The communication interface is used for communication between the above electronic device and other devices.

[0197] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0198] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0199] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned data transmission methods is implemented.

[0200] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the data transmission methods in the above embodiments.

[0201] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0202] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0203] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0204] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A data transmission system applied to electronic equipment, characterized in that: include: a first transmission unit and a second transmission unit; The first transmission unit is used to detect that target screen projection data is written in the target memory, and send a data read instruction to the second transmission unit; wherein the target memory is a memory into which the target screen projection data is written by a target screen projection server, and the data read instruction carries target information for indicating the address of the target memory; the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is data transmitted by the screen projection device, and the target memory is the memory applied for by the first transmission unit; The second transmission unit is used to read the target projection data in the target memory based on the target information carried in the data reading instruction, and connect the target projection data to the operating system platform of the electronic device, so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for executing the processing task.

2. The data transmission system according to claim 1, characterized in that: The first transmission unit is also used to detect that target screen projection data is written in the target memory, and before sending a data read instruction to the second transmission unit, make a memory application to obtain the target memory, and pass the target information of the target memory to the target screen projection server.

3. The data transmission system according to claim 1, characterized in that: The second transmission unit includes: a camera hardware abstraction layer process unit and an audio hardware abstraction layer process unit, and the target screen projection data includes target image data and / or target audio data; The first transmission unit sends a data read instruction to the second transmission unit, including: The first transmission unit sends a data reading instruction to the camera hardware abstraction layer process unit and the audio hardware abstraction layer process unit respectively; The second transmission unit reads the target screen projection data in the target memory based on the target information carried in the data reading instruction, and connects the target screen projection data to the operating system platform of the electronic device, including: The camera hardware abstraction layer process unit reads the target image data in the target memory based on the target information carried in the data reading instruction, and connects the read target image data to the operating system platform of the electronic device; and / or, The audio hardware abstraction layer process unit reads the target audio data in the target memory based on the target information carried in the data reading instruction, and connects the read target audio data to the operating system platform of the electronic device.

4. The data transmission system according to any one of claims 1 to 3, characterized in that: The second transmission unit connects the target screen projection data to the operating system platform of the electronic device, including: Connecting the target image data in the target projection data to the operating system platform in the form of data of a designated camera node; wherein the designated camera node is a virtual camera node in the operating system platform; and / or, The target audio data in the target screen projection data is connected to the operating system platform in the form of data of a designated audio node; wherein the designated audio node is a virtual audio node in the operating system platform.

5. The system according to claim 4, characterized in that The method of connecting the target image data in the target projection data to the operating system platform in the form of data of a specified camera node includes: In the case where there is at least one channel of image data in the read target screen projection data, if the number of the at least one channel of image data is not greater than the first number of idle designated camera nodes, then for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data of an idle designated camera node; otherwise, for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data about an added designated camera node and one of the idle designated camera nodes; wherein the idle designated camera node is a node other than the designated camera node required for use by the camera of the electronic device among the multiple designated camera nodes that the operating system platform has by default, and the added designated camera node is a node that is added by the operating system platform after applying for the addition of the designated camera node to the operating system platform, and the number of the added designated camera nodes is a second number, which is the difference between the number of the at least one channel of image data and the first number; and / or, The method of connecting the target audio data in the target screen projection data to the operating system platform in the form of data of a specified audio node includes: When there is at least one channel of audio data in the read target screen projection data, if the number of the at least one channel of audio data is not greater than the third number of idle designated audio nodes, then for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data of an idle designated audio node; otherwise, for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data about an added designated audio node and one of the idle designated audio nodes; wherein the idle designated audio node is a node among the multiple designated audio nodes that the operating system platform has by default, except the designated audio node required by the microphone of the electronic device, and the added designated audio node is added by the operating system platform after applying for the addition of the designated audio node to the operating system platform, and the number of the added designated camera nodes is the fourth number, and the fourth number is the difference between the number of the at least one channel of audio data and the third number.

6. The data transmission system according to claim 4, characterized in that: The target software responds to the processing task for the projection data by calling the specified interface of the operating system platform to obtain the projection data required for the processing task for executing the processing task, including: The target software requests a node list from the operating system platform in response to a processing task for projection data; wherein the node list includes a camera node and an audio node that have been connected to the operating system platform; Determine the designated camera node and / or designated audio node from the node list to obtain a target node; The designated interface of the operating system platform is called to obtain the data of the target node through the designated interface, and obtain the screen projection data required for the processing task to execute the processing task.

7. The data transmission system according to any one of claims 1 to 3, characterized in that: The operating system platform is an Android open source platform; The first transmission unit sends a data read instruction to the second transmission unit, including: Sending a data read instruction to the second transmission unit through a binder connection between the first transmission unit and the second transmission unit; The method for establishing a binder connection between the first transmission unit and the second transmission unit includes: The second transmission unit requests the binder driver to add a binder service between the first transmission unit and the second transmission unit, so that the binder driver establishes a binder connection between the first transmission unit and the second transmission unit.

8. The data transmission system according to claim 2, characterized in that: The first transmission unit transmits the target information in the target memory to the target screen projection server, including: The target information in the target memory is transmitted to the target screen projection server through a binder connection between the first transmission unit and the target screen projection server; Among them, the method for establishing the binder connection between the first transmission unit and the target screen projection server includes: The target screen projection server requests the binder driver to add a binder service between the first transmission unit and the target screen projection server, so that the binder driver establishes a binder connection between the first transmission unit and the target screen projection server.

9. The data transmission system according to claim 2, characterized in that: The first transmission unit is also used to send a signature certificate to the target screen projection server before transmitting the target information in the target memory to the target screen projection server, so that the target screen projection server performs signature verification based on the signature certificate, and after the signature verification passes, feeds back result information indicating the verification passes to the first transmission unit; After receiving the result information indicating that the verification has passed, the target information in the target memory is transmitted to the target screen projection server.

10. The data transmission system according to claim 1, characterized in that: The data reading instruction carries a signature certificate of the first transmission unit; The second transmission unit is also used to verify the signature certificate carried by the data reading instruction before reading the target projection data in the target memory based on the target information carried in the data reading instruction. If the verification is successful, the step of reading the target projection data in the target memory based on the target information carried in the data reading instruction is executed.

11. A data transmission method, characterized in that: Applied to electronic equipment; the method comprises: It is detected that target screen projection data is written in the target memory, and the target screen projection data in the target memory is read; wherein, the target memory is a memory into which the target screen projection data is written by a target screen projection server, the target screen projection server is a screen projection server running in the electronic device and communicating with the screen projection device, and the target screen projection data written to the target memory is data transmitted by the screen projection device, the target memory is a memory applied for by a first transmission unit, and the first transmission unit is a unit in the electronic device for detecting whether the target screen projection data is written in the target memory; The target projection data is connected to the operating system platform of the electronic device so that the target software responds to the processing task for the projection data, calls the specified interface of the operating system platform, and obtains the projection data required for the processing task for executing the processing task.

12. The method according to claim 11, characterized in that Before detecting that target screen projection data is written into the target memory and reading the target screen projection data in the target memory, the method further includes: Apply for target memory; The target information of the target memory is transmitted to the target screen projection server; wherein the target information is used to indicate the address of the target memory; The target information in the target memory is transmitted to the target screen projection server, including: By establishing the obtained binder connection, the target information in the target memory is transmitted to the target screen projection server; wherein the method of establishing the obtained binder connection includes: requesting a binder driver to add a binder service to establish the connection through the binder driver; The method further comprises: Before the target information in the target memory is passed to the target screen projection server, a signature certificate is sent to the target screen projection server so that the target screen projection server performs signature verification based on the signature certificate, and after the signature verification passes, feedback is given indicating that the verification passed; after receiving the result information indicating that the verification passed, the target information in the target memory is passed to the target screen projection server.

13. The method according to claim 11, characterized in that The target screen projection data includes target image data and / or target audio data; Reading the target screen projection data in the target memory includes: Sending a data read instruction carrying target information to a camera hardware abstraction layer process unit and an audio hardware abstraction layer process unit, so that the camera hardware abstraction layer process unit reads the target image data in the target memory based on the target information carried in the data read instruction, and the audio hardware abstraction layer process unit reads the target audio data in the target memory based on the target information carried in the data read instruction; the target information is used to indicate the address of the target memory; The step of connecting the target screen projection data to the operating system platform of the electronic device includes: The read target image data is connected to the operating system platform through the camera hardware abstraction layer process unit; the read target audio data is connected to the operating system platform through the audio hardware abstraction layer process unit.

14. The method according to any one of claims 11 to 13, characterized in that: The step of connecting the target screen projection data to the operating system platform of the electronic device includes: Connecting the target image data in the target projection data to the operating system platform in the form of data of a designated camera node; wherein the designated camera node is a virtual camera node in the operating system platform; and / or, The target audio data in the target screen projection data is connected to the operating system platform in the form of data of a designated audio node; wherein the designated audio node is a virtual audio node in the operating system platform.

15. The method according to claim 14, characterized in that The method of connecting the target image data in the target projection data to the operating system platform in the form of data of a specified camera node includes: In the case where there is at least one channel of image data in the read target screen projection data, if the number of the at least one channel of image data is not greater than the first number of idle designated camera nodes, then for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data of an idle designated camera node; otherwise, for each channel of the at least one channel of image data, the channel of data is connected to the operating system platform in the form of data about an added designated camera node and one of the idle designated camera nodes; wherein the idle designated camera node is a node other than the designated camera node required for use by the camera of the electronic device among the multiple designated camera nodes that the operating system platform has by default, and the added designated camera node is a node that is added by the operating system platform after applying for the addition of the designated camera node to the operating system platform, and the number of the added designated camera nodes is a second number, which is the difference between the number of the at least one channel of image data and the first number; and / or, The method of connecting the target audio data in the target screen projection data to the operating system platform in the form of data of a specified audio node includes: In the case where there is at least one channel of audio data in the read target screen projection data, if the number of the at least one channel of audio data is not greater than the third number of idle designated audio nodes, then for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data of an idle designated audio node; otherwise, for each channel of the at least one channel of audio data, the channel of data is connected to the operating system platform in the form of data about an added designated audio node and one of the idle designated audio nodes; wherein the idle designated audio node is a node among the multiple designated audio nodes that the operating system platform has by default, except for the designated audio node required to be used by the microphone of the electronic device, and the added designated audio node is added by the operating system platform after applying for the addition of the designated audio node to the operating system platform, and the number of the added designated camera nodes is a fourth number, and the fourth number is the difference between the number of the at least one channel of audio data and the third number; and / or, The target software responds to the processing task for the projection data by calling the specified interface of the operating system platform to obtain the projection data required for the processing task for executing the processing task, including: The target software requests a node list from the operating system platform in response to a processing task for projection data; wherein the node list includes a camera node and an audio node that have been connected to the operating system platform; Determine, from the node list, a designated camera node having image data in the target screen projection data and / or a designated audio node having audio data in the target screen projection data, and obtain a target node; The designated interface of the operating system platform is called to obtain the data of the target node through the designated interface, and obtain the screen projection data required for the processing task to execute the processing task.

16. The method according to claim 13, characterized in that The operating system platform is an Android open source platform; the data reading instruction is sent by establishing a binder connection; the established binder connection is established by requesting a binder driver to add a binder service and establishing the connection through the binder driver; and / or, The data read instruction carries a signature certificate. Accordingly, the camera hardware abstraction layer process unit verifies the signature certificate carried by the data read instruction before reading the target image data in the target memory based on the target information carried in the data read instruction. If the verification is successful, the step of reading the target image data in the target memory based on the target information carried in the data read instruction is executed; Before reading the target audio data in the target memory based on the target information carried in the data read instruction, the audio hardware abstraction layer process unit verifies the signature certificate carried by the data read instruction, and if the verification is successful, executes the step of reading the target audio data in the target memory based on the target information carried in the data read instruction.

17. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 11-16 when executing a program stored in a memory.

18. A computer program product, characterized in that The method comprises a computer program which, when executed by a processor, implements the method according to any one of claims 11 to 16.

Images