A method, apparatus, and system for transmitting data in near field

Abstract

The application provides a method, device and system for transmitting data in a near field. In response to a request of a first service on a first device, the first device establishes a first connection with a second device to transmit data of the first service to the second device, and in the process of establishing the first connection, the first device transmits part of the data of the first service using a second connection already established between the first device and the second device. In this way, without changing the process of establishing the first connection, the data of the service is transmitted using another connection in the process, so that the transmission of the data can be started more quickly to shorten the response time of the second device to the first service.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to methods, devices and systems for transmitting data in a near-field wireless network communication environment. Background Technology

[0002] Currently, users own an increasing number of terminal devices, and the scenarios for business collaboration and communication between these devices are becoming increasingly diverse. In the near field, terminals can interact, such as casting a video from a video application running on a mobile phone to a TV. This can be done directly to the TV or via a module or device connected to the TV via a cable to a TV without an operating system. Another example is projecting the screen of one terminal (e.g., a mobile phone) onto another terminal (e.g., a tablet or smart TV), allowing the projected terminal to display the screen of the projected terminal. Yet another example is transferring files stored on one terminal to another. In a near-field wireless network communication environment, the positional relationships between terminal devices may change as the user moves, and stable communication quality is crucial for ensuring a good user experience.

[0003] Near-field, short for near-field wireless network communication environment, refers to a communication environment where the distance between devices is about ten centimeters to ten meters. In daily life, a common near-field wireless network communication environment is a scenario where multiple terminal devices are deployed in the same room or adjacent rooms. For example, a user uses a mobile phone on the sofa in the living room to operate the smart TV in the living room, or a user uses a tablet computer at a seat in a conference room to operate a display screen with an operating system.

[0004] In the near field, two devices can communicate using various communication technologies and architectures. Using different technologies and / or architectures will impact transmission performance. Currently, the commonly used peer-to-peer (P2P) architecture based on Wireless Fidelity (Wi-Fi) technology in near-field environments has its own limitations and cannot meet the performance requirements of data transmission in some scenarios, thus affecting user experience. Summary of the Invention

[0005] This application provides a method, apparatus, and system for transmitting data in the near field. Compared with commonly used transmission methods, it can better meet the performance requirements of data transmission in some scenarios and improve transmission performance.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a method for transmitting data in the near field, wherein a first device, in response to a request of a first service, establishes a first connection between the first device and a second device to send data of the first service to the second device, and, during the establishment of the first connection, the first device uses a second connection already established between the first device and the second device to send part of the data of the first service.

[0008] The first connection and the second connection can use different wireless networks.

[0009] The first service is a service provided by an application or component running on the first device. Requests for this first service can be triggered by the user interacting with the first device, or by the application or component itself.

[0010] The first service is one that requires sending data to another device. The data for the first service consists of the data that needs to be processed (e.g., sent) to complete that first service; it can also be considered data belonging to that first service. Instructions are also considered a type of data. For example, if the first service is a screen mirroring service, then the data for the first service includes the screen content being mirrored. If the first service is a multimedia playback service, then the data for the first service includes the bitstream of the multimedia or the network address of the multimedia. If the first service is a service used to control another device, then the data for the first service includes control instructions. If the first service is a file transfer service, then the data for the first service includes the file being transferred. A file is a collection of data with a certain format that can be recognized or processed by an application. Files can be images, videos, audio, compressed files, documents of various formats or types, mini-programs, or web links, etc.

[0011] The request for the first service instructs the processing of that first service and may be issued by an application or component of the first device to the operating system of the first device. The request itself may or may not include instructions to transmit data. However, to complete the first service, at least the transmission of the data for that first service to another device is required. The identifier of the second device may be carried in the request for the first service, but more often, the second device is discovered during the establishment of the first connection through a device discovery process, within a certain range of the first device via scanning or broadcasting.

[0012] The first and second connections use different networks, and this difference includes at least one of the following: the networks use different communication technologies, or the networks have different network architectures. For example, Wi-Fi, 5G, Bluetooth, and infrared technologies are different communication technologies, and directly connected networks and access point-based networks have different network architectures. Different networks use different interfaces on the network interface cards (NICs) or different NICs themselves. Different networks can be indicated by different names.

[0013] The process of establishing a first connection includes a preparatory process before establishing the first connection, and a process of establishing a connection based on an already accessed network. The preparatory process before establishing the first connection includes at least one of the networks used to establish the first connection and the networks used to access the first connection. How the process of establishing the first connection is specifically implemented depends on the wireless communication technology used and the near-field communication architecture used (usually a P2P architecture or an access point-based architecture). Of course, regardless of the technology used, the first device and the second device need to exchange messages during this process. Each wireless communication technology and each near-field communication architecture has its own protocols and rules, which this application does not modify. Therefore, as long as the communication technology and communication architecture used for the first connection are clearly defined, the process of establishing the first connection can follow the provisions in existing documents. The communication technology and communication architecture used for the first connection correspond to the communication technology and architecture that the first and second devices are preset or default configured to use for communication with other devices in the near field. For example, configured to use a peer-to-peer (P2P) architecture based on wireless fidelity (Wi-Fi) technology.

[0014] The second connection is established before the first connection is established. Therefore, data can be transmitted using the second connection while the first connection is being established. In one scenario, the second connection is established before the first connection is established. In another scenario, the second connection is established after the first connection is established but before it is completed.

[0015] In the above method, in response to the request of the first service, the first device establishes a first connection to transmit the data of the first service. That is, the data of the first service should ideally be transmitted through the first connection. However, establishing the first connection takes time. Therefore, during the establishment of the first connection, a portion of the data of the first service is transmitted using the already established second connection between the first and second devices. In this way, while the first connection is still being established, the transmission of the first service data is carried out using the already established second connection, allowing for faster commencement of service data transmission without waiting for the first connection to complete. This shortens the waiting time before service data transmission, speeds up the response to service requests, and thus improves transmission performance.

[0016] Based on the above description of the first aspect, in some implementations, the first service can be one or more specific services. That is, not all services may require improvements to the existing method as described in this application; some services may still utilize existing technologies. For example, the first service is screen mirroring. This is because screen mirroring has high latency requirements for display; that is, after initiating a screen mirroring request, it is desirable to display the mirrored content on the target device as quickly as possible. The above method can initiate service data transmission more quickly, thereby improving transmission performance.

[0017] Based on the above description of the first aspect, in one implementation, after the first connection is established, the first device switches to using the first connection to send the data of the first service to the second device. In this way, the first connection ultimately transmits the data of the first service, without wasting the resources required to establish and maintain the first connection, and also better transmitting the data of the first service. This is because, typically, the default communication technology and architecture used by the first connection have more stable transmission performance, which is more conducive to data transmission.

[0018] Once the first connection is established, the first device is aware of it. For example, the first device could receive a message from the second device indicating that the connection has been established. How the first device determines that the first connection has been established depends on the communication technology used for the first connection; specific details can be found in the existing regulations of communication technologies.

[0019] In addition, the first connection can transmit data for the first service, which may be data that was not successfully transmitted in the second connection, or it may be data for the first service that is retransmitted.

[0020] Optionally, after switching to use the first connection to send the data for the first service, the first device may dismantle or terminate the second connection in order to save transmission resources.

[0021] In conjunction with the above description of the first aspect, the method of the first aspect further includes: based on the request of the first service, the first device initiates the establishment of a second connection.

[0022] In other words, the second connection is established after the request for the first service is generated. It can also be understood that the second connection is established during the process of establishing the first connection, but the second connection is established faster than the first connection.

[0023] On the other hand, "initiating the establishment of a second connection based on the request of the first service" means that the request of the first service does not need to be modified and can still use the existing service request format, that is, it does not reflect the need to initiate a second connection. However, the first device modifies the data transmission process based on the request of the first service and establishes an additional connection. That is, after receiving the request issued by the first service, the program code of the first device not only needs to establish the first connection that should be used to transmit the data of the first service, but also needs to establish a second connection that takes less time than the establishment process of the first connection. In this way, the second connection will be established before the first connection and will start transmitting the data of the first service first. Thus, during the establishment process of the first connection, at least some time can be used to transmit the data of the first service through another connection, so that the second device at the other end can start receiving the data of the first service faster and reduce the response time to the first service. Moreover, there is no need to prepare the second connection in advance, and the second connection is only established when there is a need for near-field data service quality.

[0024] Based on the above description of the first aspect, in one implementation, a third connection is used to send data of a second service to a second device, the third connection using the same wireless network as the first connection; during the transmission of the data of the second service, in response to a request from the third service, a fourth connection is used to send the data of the third service; or in response to a request from the third service, a fifth connection is used to send the data of the third service, and the data of the second service is switched to be sent using the fourth connection, wherein the fourth connection uses the same wireless network as the second connection, and the fifth connection uses the same wireless network as the first connection.

[0025] Using different wireless networks means using different network technologies or corresponding network topologies. Since the two connections are carried by different wireless networks, their transmission performance is relatively easy to compare. The evaluation of transmission performance can refer to existing standards. For example, transmission performance is indicated by at least one of the following parameters: bandwidth, latency, and packet loss rate. Bandwidth or latency is the most commonly used. Generally, the transmission performance of the first connection is better than that of the second connection. This could be because the bandwidth (or throughput) of the network corresponding to the first connection is better than that of the network corresponding to the second connection, or because the stability of the network corresponding to the first connection is better than that of the second connection. Stability refers to the smaller fluctuations in the parameters indicating transmission performance of the network.

[0026] In other words, while the first device is sending service data using the first connection, another service needs to be executed—that is, when it needs to send data for another service—the first device still has service data in transmission that has not yet been completed. In response to the request of this service (i.e., the second service), the first device can use another network to send the new service data, or use the network it is currently using to send the new service data, and it also performs an additional action: adjusting the network used by the currently being sent service data. The data transmission of these two services overlaps in time, meaning that at least part of the process requires simultaneous use of network resources, thus the two services will affect each other. This will manifest on the second device, such as one service experiencing lag or another service becoming unresponsive to operations on the second device. Existing technologies either use a single network to transmit the data for both services or transmit the data for each service on two separate networks, rather than transmitting the data for the two services on separate networks as described above, thus decoupling the data transmission of the two services. The above implementation utilizes the resources of two different networks to transmit the data for the two services separately, making full use of resources and avoiding resource contention when transmitting the data for the two services simultaneously.

[0027] Additionally, it should be noted that in the scheme of switching the data of the third service to be sent using the fourth connection, the first device can first send the data of the second service, and then switch the data of the third service to the fourth connection for transmission; or it can first switch the data of the third service to the fourth connection for transmission, and then send the data of the second service; or even, it may be possible to process these two services simultaneously, that is, to trigger the transmission of the data of the third service and switch the connection for the transmission of the second service at the same time.

[0028] Based on the above description of the first aspect, in one implementation, the total bandwidth requirement of the first and second services is greater than the bandwidth of the third connection, and the bandwidth of the third connection is greater than the bandwidth of the fourth connection. The bandwidth of the third and fourth connections can be represented by the bandwidth of the connections themselves, or by the bandwidth of the networks used by the connections.

[0029] The scenario described here involves two services that require simultaneous transmission of at least some data, and the bandwidth needed is insufficient to meet the network resources of the third connection alone. In this case, utilizing the resources of another network can better guarantee the transmission performance of the data for both services. Furthermore, because the bandwidth of the third connection is greater than that of the fourth connection, the data for the second service is transmitted first using the third connection, meaning that the network with better transmission performance is prioritized.

[0030] Based on the above description of the first aspect, in one implementation, if the bandwidth requirement of the second service is greater than that of the third service, then the fourth connection is used to send the data for the third service. For example, the second service is a screen mirroring service, and the third service is a service for controlling a second device; or the second service is a file transfer service, and the third service is either a screen mirroring service or a service for controlling a second device. If the bandwidth requirement of the second service is less than that of the third service, the fifth connection is used to send the data for the third service, and the data for the second service is switched to be sent using the fourth connection. In other words, services with high bandwidth requirements will use the network defaulted to the first connection to send data, while services with low bandwidth requirements will use the network defaulted to the second connection. For example, the second service is a service for controlling a second device, and the third service is a screen mirroring service; or the second service is either a screen mirroring service or a service for controlling a second device, and the third service is a file transfer service.

[0031] Based on the above description of the third aspect, in one implementation, if the priority of the second service is higher than that of the third service, then the fourth connection is used to send the data of the third service; if the priority of the second service is lower than that of the third service, then the fifth connection is used to send the data of the third service, and the data of the second service is switched to be sent using the fourth connection. In other words, services with higher priority use the network defaulted to the first connection to send data, while services with lower priority are switched to use the network used by the second connection. For example, the service used to control the second device has a higher priority.

[0032] Secondly, embodiments of this application provide yet another method for transmitting data in the near field. This method includes: based on a first instruction received from a first device, a second device assisting the first device in establishing a first connection to receive data from a first service of the first device; during the process of assisting in establishing the first connection, the second device uses a second connection already established between the first device and the second device to receive a portion of the data from the first service. The first connection and the second connection may use different wireless networks.

[0033] As can be seen, the second aspect is a method corresponding to the first and even the third aspect, described from the perspective of the second device. This is because the establishment of the connection and the successful transmission of service data both require the cooperation of the two devices. Therefore, the various descriptions and implementation methods of the first and third aspects also apply to the second aspect, and will not be discussed further here. Based on the description of the first aspect, the following implementation methods are briefly listed as supplementary.

[0034] In conjunction with the above description of the second aspect, in one implementation, the method of the second aspect further includes: after the first connection is established, the second device receives data of the first service from the first device from the first connection.

[0035] In conjunction with the above description of the second aspect, the method of the second aspect further includes: assisting the first device in establishing a second connection based on a second instruction received from the first device. That is, the second device is unaware of the request from the first service; however, the first device initiates the process of establishing the second connection by sending an instruction to the second device for establishing the second connection. Based on this instruction, the second device can begin a corresponding instruction interaction to establish the connection.

[0036] Thirdly, embodiments of this application provide yet another method for transmitting data in the near field, the method comprising:

[0037] The device uses a first network to send data for a first service to a second device; during the sending of the data for the first service, in response to a request from a second service, it uses the second network to send data for the second service; or in response to a request from a second service, it uses the first network to send data for the second service, and also switches the data for the first service to be sent using the second network.

[0038] The first network and the second network are different networks, meaning they use different network technologies and / or correspond to different network topologies. In some implementations, at least one of the first and second networks is a wireless network. This makes it relatively easy to compare the transmission performance of the two networks, and the superiority or inferiority of transmission performance can be judged by referring to existing standards. For example, transmission performance is indicated by at least one of the following parameters: bandwidth, latency, and packet loss rate. Bandwidth or latency is the most commonly used. Generally, the transmission performance of the first network is better than that of the second network, either because the bandwidth of the first network (which can also be characterized by throughput) is better than that of the second network, or because the stability of the first network is better than that of the second network. Stability refers to the smaller fluctuations in the parameters indicating the network's transmission performance.

[0039] On the other hand, the connection to the second network may be established in response to a request from the second service, or the connection to the second network may have already been established but not used when only data from the first service is being transmitted. The first device can always access the second network, which makes establishing a connection much faster.

[0040] Based on the above description of the third aspect, in one implementation, the total bandwidth requirement of the first service and the second service is greater than the bandwidth of the first network, and the bandwidth of the first network is greater than the bandwidth of the second network.

[0041] Based on the above description of the third aspect, in one implementation, if the bandwidth requirement of the first service is greater than the bandwidth requirement of the second service, the first device uses the second network to send the data of the second service; if the bandwidth requirement of the first service is less than the bandwidth requirement of the second service, the first device uses the first network to send the data of the second service, and also switches the data of the first service to be sent using the second network.

[0042] Based on the above description of the third aspect, in one implementation, the transmission performance of the first network is better than that of the second network. If the priority of the first service is higher than that of the second service, the first device uses the second network to send the data of the second service. If the priority of the first service is lower than that of the third service, the first device uses the first network to send the data of the second service, and also switches the data of the first service to be sent using the second network.

[0043] Based on the above description of the third aspect, in one implementation, the first device uses a first network and a second network based on multipath MP technology.

[0044] As can be seen, the method in the third aspect corresponds to the data transmission phase, while the first aspect fully describes both the connection establishment phase and the data transmission phase. Therefore, some implementation methods in the first aspect also correspond to this phase. The content of the third aspect is similar to some implementation methods in the first aspect, except that the description changes from the perspective of connection switching to the perspective of network switching. Therefore, the description of the implementation methods in the first aspect that correspond to the content of the third aspect also applies to the third aspect. Furthermore, the first and third aspects are described from the perspective of the first device, i.e., the sender of the service data. This method is a data transmission method, which also involves the receiver of the service data, i.e., the second device. The method corresponding to the second device is the method in the second aspect. Therefore, the explanations of the first and third aspects also apply to the corresponding parts of the second aspect, and will not be repeated here.

[0045] Furthermore, the "first business" in the third aspect can be the same as or different from the "first business" in the first aspect; the use of the same name here is merely for consistency with the summary. The "first business" and "second business" in the third aspect are different businesses. Therefore, the various interpretations and descriptions of the first aspect also apply to the third aspect, and will not be repeated here.

[0046] Fourthly, embodiments of this application provide a system for transmitting data in the near field. The system includes a first device and a second device. The first device is configured to establish a first connection between itself and the second device in response to a request from a first service, so as to send data of the first service to the second device. The first device is also configured to send partial data of the first service using an existing second connection between itself and the second device during the establishment of the first connection. The second device is configured to cooperate with the first device in establishing the first and second connections, and to receive data of the first service. The first and second connections use different networks.

[0047] The fourth aspect is a system corresponding to the methods of the first aspect (the third aspect can be used as a supplement to the first aspect) and the second aspect. Therefore, for a specific description of the system and an explanation of its technical effects, please refer to the descriptions in the methods of the foregoing aspects.

[0048] Fifthly, embodiments of this application provide an apparatus for transmitting data in the near field. The apparatus includes a first connection establishment module, configured to establish a first connection between a first device and a second device in response to a request of a first service, so as to transmit data of the first service to the second device; and a transmission module, configured to transmit part of the data of the first service using a second connection already established between the first device and the second device during the establishment of the first connection.

[0049] In some implementations, the first connection and the second connection use different wireless networks.

[0050] In conjunction with the description of the fifth aspect, in one implementation, the sending module is also used to send the data of the first service to the second device using the first connection after the first connection is established.

[0051] In conjunction with the above description of the fifth aspect, in one implementation, the connection establishment module is also used to initiate the establishment of a second connection based on the request of the first service.

[0052] In conjunction with the above description of the fifth aspect, in one implementation, the sending module is further configured to send data of the second service to the second device using a third connection, wherein the third connection uses the same wireless network as the first connection; during the sending of the data of the second service, in response to a request from the third service, the module uses a fourth connection to send the data of the third service; or in response to a request from the third service, the module uses a fifth connection to send the data of the third service, and the module also switches the sending of the data of the second service to use the fourth connection, wherein the fourth connection uses the same wireless network as the second connection, and the fifth connection uses the same wireless network as the first connection.

[0053] In conjunction with the above description of the fifth aspect, in one implementation, the sending module is further configured to: if the bandwidth requirement of the second service is greater than the bandwidth requirement of the third service, then use the fourth connection to send the data of the third service; if the bandwidth requirement of the second service is less than the bandwidth requirement of the third service, then use the fifth connection to send the data of the third service; and to convert the data of the second service to be sent using the fourth connection.

[0054] In conjunction with the above description of the fifth aspect, in one implementation, the sending module is further configured to: if the priority of the second service is higher than the priority of the third service, then use the fourth connection to send the data of the third service; if the priority of the second service is lower than the priority of the third service, then use the fifth connection to send the data of the third service; and to switch the data of the second service to be sent using the fourth connection.

[0055] The fifth aspect is the apparatus corresponding to the first aspect. For a description of the apparatus and its technical effects, please refer to the first aspect.

[0056] In a sixth aspect, embodiments of this application provide an apparatus for receiving data in the near field. The apparatus includes a connection establishment module for assisting the first device in establishing a first connection based on a first instruction received from the first device, so as to receive data from the first service of the first device; and a receiving module for receiving part of the data of the first service using a second connection already established between the first device and the second device during the process of assisting in establishing the first connection.

[0057] In some implementations, the first connection and the second connection use different wireless networks.

[0058] In conjunction with the above description of the sixth aspect, in one implementation, the connection establishment module is further configured to assist the first device in establishing a second connection based on a second instruction received from the first device.

[0059] In conjunction with the above description of the sixth aspect, in one implementation, the receiving module is further configured to receive data of the first service from the first device after the first connection is established.

[0060] The sixth aspect is the apparatus corresponding to the second aspect, and for a description of the apparatus and its technical effects, please refer to the first and third aspects.

[0061] In a seventh aspect, embodiments of this application provide an apparatus for transmitting data in the near field. The apparatus includes a transmission module for transmitting data of a first service to a second device using a first wireless network; during the transmission of data of the first service, in response to a request of a second service, transmitting data of a second service using a second wireless network; or in response to a request of a second service, transmitting data of the second service using the first wireless network, and converting data of the first service to be transmitted using the second wireless network.

[0062] The seventh aspect corresponds to the method in the third aspect. For an explanation of this part of the device, a description of various implementation methods and technical effects, please refer to the first and third aspects.

[0063] To save space, the following descriptions of the commonalities and implementation methods from the first to the seventh aspects will be presented. These descriptions are applicable to all of the above aspects.

[0064] In one implementation, the first device communicates with the second device using multipath MP technology. That is, the first and second connections are sub-connections of the same multipath connection. In this implementation, both the first and second devices support multipath technology. This allows for more convenient management of the first and second connections under a single multipath connection. Switching between transmitting service data from the second connection and the first connection is smoother, and data transmission is uninterrupted. There is also no need to reuse data transmitted on a previous connection for transmission on a different connection.

[0065] In one implementation, the first connection uses a directly connected wireless network, and the second connection uses an access point-based wireless network. Alternatively, the first network mentioned earlier is a directly connected wireless network, and the second network is an access point-based wireless network. A directly connected wireless network allows two devices to communicate directly, which is a P2P architecture network. This allows two devices to exclusively use the P2P network resources, resulting in stable transmission performance. An access point can be understood as a device used to access a network. Devices accessing the network through this access point also need to go through the access point to transmit data; therefore, the access point also functions as a forwarding device. Although the term "access point" is commonly used in Wi-Fi wireless LANs, it doesn't actually limit the type of network technology. Any device used to access a network and with forwarding capabilities can be called an access point.

[0066] In this implementation, data transmission can begin more quickly via an access point-based network because the network is already established; simply accessing it and establishing a connection is sufficient, shortening the connection preparation process compared to using a directly connected wireless network. During the creation of a connection between two devices via the access point, data transmission can begin more rapidly, reducing response time. The processes of establishing the first access point-based connection and the second directly connected connection can utilize existing technologies.

[0067] In one implementation, both the first and second connections use Wi-Fi communication technology. This means that establishing the first connection creates a Wi-Fi network between the first and second devices, and the access point is also a Wi-Fi-based device, such as a router. Of course, the communication technology used for the first and second connections is not limited, as long as establishing the first connection takes longer than establishing the second connection.

[0068] In one implementation, the first service is screen mirroring. Thus, using the method described in this application, the screen of the first device can be mirrored to the second device as quickly as possible, shortening response time and user waiting time.

[0069] In one implementation, the screen area of ​​the first device is smaller than that of the second device. For example, the first device is a smartphone, and the second device is a tablet or smart TV. Another example is that the first device is a laptop or tablet, and the second device is a smart TV.

[0070] Eighthly, this application provides a system for transmitting data in the near field. The system includes a first device and a second device. The first device is configured to establish a first connection between itself and the second device in response to a request from a first service, so as to send data of the first service to the second device. The first device is also configured to send part of the data of the first service using a second connection already established between the first device and the second device during the establishment of the first connection. The first connection and the second connection use different wireless networks. The second device is configured to cooperate with the first device to establish the first connection and the second connection, and to receive data of the first service.

[0071] For the descriptions of the first and second devices in aspect eight, please refer to the descriptions of aspect one, aspect two, and commonalities above. They will not be repeated here.

[0072] In a ninth aspect, this application also provides a method for controlling a device for a second device whose screen is projected by a first device, the method comprising: the second device displaying screen content of the first device; the second device receiving a file from the first device; during the process of the second device receiving the file, the screen of the second device maintaining the display of the screen content of the first device; and in response to an operation by a user on the second device, changing at least a portion of the screen content of the first device displayed on the second device.

[0073] In some implementations, maintaining the display of the screen content on the first device may include scaling down the screen content or changing the display format of the screen content, such as its orientation or color.

[0074] User interactions with the second device essentially involve manipulating controls or areas displayed on the first device's screen. These interactions can include single-clicking, double-clicking, long-pressing, knuckle tapping, or scrolling.

[0075] A file is a collection of data with a specific format that can be recognized or processed by an application. Files can be images, videos, audio files, compressed files, documents of various formats or types, mini-programs, web links, and so on.

[0076] During the process of the second device receiving a file from the first device, screen mirroring did not stop, and the second device did not fail to respond to user controls on the first device's screen due to receiving the file. This reduces service lag and improves user experience. This service lag is actually caused by multiple services needing to transmit data, leading to contention for network resources, and some service data failing to be transmitted to the second device in a timely or smooth manner. The ninth aspect corresponds to the scenario mentioned above where data from two services is simultaneously sent to the second device.

[0077] In a tenth aspect, this application provides an apparatus having the function of implementing the method of any one of the above aspects. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0078] Eleventhly, an apparatus is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, wherein when the communication device is in operation, the processor executes the computer execution instructions stored in the memory to cause the communication device to perform a transmission rate adjustment method as described in any of the preceding aspects.

[0079] In a twelfth aspect, a communication device is provided, comprising: a memory and a processor; the memory is coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, perform a method as described in any of the preceding aspects.

[0080] In a thirteenth aspect, embodiments of this application provide a chip system including processing circuitry and a storage medium storing instructions. When the instructions are executed by the processing circuitry, they are used to implement the functions of the methods described in any of the preceding aspects. The chip system may be composed of chips or may include chips and other discrete devices.

[0081] Fourteenth aspect, a communication device is provided, which may be a circuit system including a processing circuit configured to perform the method as described in any of the preceding aspects.

[0082] In a fifteenth aspect, embodiments of this application also provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0083] In a sixteenth aspect, this application also provides a computer program product including instructions that, when run on a computer, cause the computer to perform the methods described above.

[0084] The methods of the foregoing aspects are referenced in aspects 10 through 16. When referencing any aspect, the descriptions of the technical effects in each aspect are referred to accordingly. Attached Figure Description

[0085] Figure 1 A communication diagram of a smartphone, tablet device, and router provided for embodiments of this application;

[0086] Figure 2A This is a schematic diagram illustrating the waiting process when a smartphone projects its screen onto a smart TV, as provided in an embodiment of this application.

[0087] Figure 2B and Figure 2C A schematic diagram illustrating multi-service interaction between a smartphone and a smart TV, provided in an embodiment of this application;

[0088] Figure 3 A schematic diagram of a system architecture provided for an embodiment of this application;

[0089] Figure 4 This is a schematic diagram of the structure of the first device and the second device provided in the embodiments of this application;

[0090] Figure 5 This is a schematic diagram of protocol stack encapsulation and data transmission provided in an embodiment of this application;

[0091] Figure 6A A flowchart illustrating a data transmission method provided in an embodiment of this application;

[0092] Figure 6B A flowchart illustrating another data transmission method provided in an embodiment of this application;

[0093] Figure 7 A schematic diagram of three parts of a protocol stack supporting multipath protocols provided for embodiments of this application;

[0094] Figure 8 This is a schematic diagram of data processing for a protocol stack supporting multipath protocols, provided in an embodiment of this application.

[0095] Figure 9 This is a schematic diagram of the architecture of a device provided in an embodiment of this application.

[0096] Figure 10 This is a flowchart illustrating a data transmission method provided in an embodiment of this application.

[0097] Figure 11 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application.

[0098] Figure 12This is a schematic diagram of the structure of the data sending end (corresponding to the first device) provided in the embodiments of this application.

[0099] Figure 13 This is a schematic diagram of the structure of the data receiving end (corresponding to the second device) provided in the embodiments of this application. Detailed Implementation

[0100] First, some expressions that may appear in this application will be explained.

[0101] Unless otherwise specified, words like "first" and "second" are used to distinguish different objects or to differentiate different treatments of the same object, rather than to describe a specific order of objects. However, when specified, "first" and "second" can refer to the same object.

[0102] "At least one" means one or more. "More than one" means two or more.

[0103] "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone. A and B can be singular or plural.

[0104] The character " / " generally indicates that the objects before and after it are in an "or" relationship. For example, A / B can mean A or B.

[0105] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0106] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0107] In the specification and drawings of this application, the terms "corresponding" and "relevant" may sometimes be used interchangeably. It should be noted that, without emphasizing the difference, they have the same meaning.

[0108] On the other hand, let me briefly describe some of the technical terms that appear in this application.

[0109] Network Interface Card (NIC): Also known as Network Interface Controller (NIC) or Network Adapter, simply called a network card. This article refers to both physical network cards and virtual network cards created from physical network cards. A physical network card is a piece of hardware designed to allow a physical machine or other device to access a network and communicate with other network elements. For example, a Subscriber Identity Module (SIM), i.e., a SIM card, is also a type of network card.

[0110] A connection is used to transmit data between a sender and a receiver. Data is considered a special type of data. Commands have control functions. A connection uses one or more paths in the network topology to transmit data. A path is a link consisting of the sender and receiver end devices, and the network devices between them. In this paper, the path is also referred to as the transmission path.

[0111] Multipath (MP) connection: A connection that allows communication between two hosts via multiple paths over an application. A multipath connection typically includes two or more sub-connections, each using one path. The connection usually described is a shortened term for a single-path connection, which uses only one path to transmit data. In this application, "connection" can refer to either a single-path connection or a multipath connection.

[0112] Path: A link between the sender and receiver. A link can forward data through one or more network nodes (such as routers, gateways, and other forwarding devices).

[0113] End devices: The devices located at both ends of a path; the sender and receiver of data are both end devices. End devices differ from forwarding devices, whose function is to transfer data from one end device to another along a path. Forwarding devices are also referred to as access points in this application. With the development of communication technology today, network structures are diverse and complex, such as end-to-end communication, end-to-cloud communication (e.g., cloud devices, servers), and cloud-to-cloud communication. This application addresses problems in near-field wireless network communication environments; therefore, the devices described in this application are primarily end devices, such as mobile phones, smart TVs, and tablets.

[0114] Sub-connections: In a multipath connection, a connection operates on each individual path. In multipath transport technologies, sub-connections can use various common single-stream transport protocols, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Quick UDP Internet Connections, Stream Control Transmission Protocol (STCP), and even some proprietary protocols. Different sub-connections within the same multipath connection can use different transport protocols.

[0115] Identifier: Used to distinguish one thing from other things of the same or different categories. Examples include business identifiers, network identifiers, and network interface card (NIC) identifiers. Identifiers can be names, numbers, or identifiers based on a distinctive characteristic, such as category identifiers. This application does not limit the implementation of various identifiers, as long as they can achieve differentiation.

[0116] A service refers to a function that can be initiated and even completed by an application on a device. Examples include screen mirroring, file transfer, device control, and control of displayed content (such as applications). Service data refers to data processed by the device to complete a specific service, or other data describing the service (such as service logs). Service data also includes instructions, as some services are control-oriented. The data for these services consists of instructions for the function of that service, which can instruct the completion of at least one action, including editing, transmission, and execution. Service data is issued by applications on the device. Furthermore, service requirements refer to the service's requirements for one or more Quality of Service (QoS) metrics. For example, the time required to complete a service, the continuity of the service (whether the transmission of service data or instructions will be interrupted), the bandwidth for sending service data, and the priority of the network used by the service are all QoS indicators. Different services may have different QoS requirements, or they may have different requirements for the same QoS metric.

[0117] The previous section briefly defined a connection; the following section will further explain connections. Connections can be identified using a four-tuple or a five-tuple. A four-tuple includes the source IP address, source port number, destination IP address, and destination port number. A five-tuple, in addition to the four-tuple, also includes the name of the transport layer protocol. An application on one device can simultaneously transmit data with an application on another device through multiple connections. This is because an application includes multiple functional modules, each with its own business logic, and each module can initiate one or more connections. Typically, applications include a module for displaying the application's user interface, whose main business is image display and interface control; this module can initiate a connection. If the application also includes a module for wireless screen mirroring, a connection can be initiated to achieve wireless screen mirroring. If the application can download files, for example, an application called an "App Store," it includes a module for downloading other applications; its main business is file (application installation package) transfer, and a separate connection can be initiated. Of course, if multiple applications need to be downloaded simultaneously, each download task can initiate a connection. Thus, an application may need to use multiple connections simultaneously. Of course, establishing a connection also requires the support of the protocol stack below the application layer and the corresponding hardware.

[0118] In addition, a connection goes through a connection establishment phase, a data transmission phase, and a connection termination phase. This application considers the connection establishment preparation phase as part of the connection establishment phase. The establishment preparation phase may include network establishment, device discovery, etc., as mentioned below. The connection establishment phase is when two devices establish a connection and begin data transmission through command interaction. The data transmission performance in this phase varies considerably and is unstable. For example, the bandwidth and congestion window used for data transmission gradually increase during this phase. In the data transmission phase, the transmission performance is relatively stable; most of the data to be transmitted in a connection is transmitted in this phase. For example, the average bandwidth in the data transmission phase is comparable to the maximum bandwidth in the connection establishment phase. Therefore, congestion control and flow control in the connection establishment and data transmission phases typically use different algorithms. The connection termination phase is the process by which two devices terminate the use of a connection and release the resources used by that connection through command interaction.

[0119] Near-field wireless network communication environment refers to a communication environment where the data sender and receiver communicate directly through a wireless network (also known as P2P architecture), or through a single-hop network device (also known as AP architecture). This network device has the functions of accessing the network and forwarding data; that is, providing a network for the data sender and receiver, and forwarding data from the sender to the receiver. Therefore, this network device is referred to as an access point or forwarding device in this application. An access point, abbreviated as AP, is not limited to the AP in Wi-Fi technology. Typically, the physical form of this network device is a single device. In some implementations, the network device can be deployed in a distributed manner. For example, the network device can be a group of modules or a parent-child device, where some modules or child devices can be placed at intervals to increase the signal strength coverage area of ​​the network device. For example, these modules may include modules for signal relay to facilitate signal penetration through walls or coverage of a larger area. Figure 1 It describes two possible transmission paths that smartphones, tablets, and routers can use as access points: transmission path 1 is a P2P architecture, and transmission path 2 is an AP architecture.

[0120] This application does not limit the communication technologies supported by the network equipment. For example, it can be a device that supports Wi-Fi technology, such as a router, or it can be a device that supports non-3rd generation partnership projects (3GPP), such as a 4th generation (4G) mobile communication technology base station or a 5th generation (5G) mobile communication technology base station.

[0121] On the other hand, various wireless communication technologies can be used when using a P2P architecture, which will not be exhaustive in this application. For example, available wireless communication technologies include Wi-Fi, cellular networks, Bluetooth, ZigBee, and even Near Field Communication (NFC) in some scenarios. When using cellular networks, the two devices no longer communicate via a base station (BS) relay, but directly connect and communicate. This type of P2P architecture is also known as Device to Device (D2D) technology. In some scenarios, implementing D2D technology also requires one or more mature wireless communication technologies such as Wi-Fi, Bluetooth, and ZigBee. In general, the connection establishment process based on a P2P architecture is relatively complex and time-consuming for most wireless communication technologies.

[0122] In contrast, establishing a P2P connection using NFC technology is much simpler, taking only about 0.1 seconds, which is several times or even tens of times longer than most current wireless communication technologies. However, NFC is a short-range wireless communication technology, used for communication within 10cm, with an effective communication distance of 4cm. This is insufficient for common wireless communication scenarios where devices are tens of centimeters to tens of meters apart. Therefore, NFC requires two devices to be very close and is mostly used for instantaneous communication, commonly found in mobile payments, electronic tags, and access control security scenarios. Compared to other wireless communication technologies, the application scenarios for the method described in this application are very limited and differ from those described in the background section of this application.

[0123] This application does not involve improvements to existing wireless communication technologies and communication architectures. In other words, this application does not improve the process of establishing a connection using a wireless communication technology in a certain communication architecture. The method of this application aims to cleverly transmit a portion of the service data by leveraging an already established connection using another network during the process of establishing a connection using a network.

[0124] For example, the specific implementation and detailed specifications of Wi-Fi-based P2P architecture networks (including the network establishment process and data transmission methods) can be found in publicly available technical documents, such as the Wi-Fi Peer-to-Peer Technical Specification, version 1.7. Some implementations of D2D technology based on wireless cellular networks require licensed cellular network frequency bands, which are currently not standardized. Other implementations rely on one or more wireless communication technologies such as Wi-Fi, Bluetooth, and ZigBee, and should adhere to the corresponding communication standards. For yet another example, the specific implementation and detailed specifications of using Wi-Fi technology for AP communication can be found in publicly available documents, such as the IEEE 802.11 wireless LAN communication standard family, which includes 802.11a / b / g / n / p / i, etc.

[0125] The technical solution described in this application mainly relates to transmitting service data from one device to another in a near-field environment. Many networks can be used in a near-field environment; some establish connections quickly, while others establish connections slowly. However, networks that establish connections quickly may have lower data transmission capabilities or network stability than those that establish connections slowly. For example, using the same communication technology, establishing a connection based on a P2P architecture is slower than establishing a connection based on an AP architecture. The following uses P2P and AP architectures as examples to illustrate situations where existing solutions cannot meet the performance requirements of data transmission between service pairs. In the following text, a network based on a P2P architecture is also simply referred to as a P2P network. A connection based on a P2P network, or a connection using a P2P network, is also simply referred to as a P2P connection. A network based on an AP architecture is also simply referred to as an AP network, and a connection based on an AP network is also simply referred to as an AP connection. In multi-path connection scenarios, a sub-connection based on a P2P network is also simply referred to as a P2P sub-connection. A sub-connection based on an AP network is also simply referred to as an AP sub-connection.

[0126] P2P architecture is a direct connection between devices, where two devices communicate directly without a third node. This allows each device to have its own dedicated network, resulting in more stable data transmission. However, when using Wireless Fidelity (Wi-Fi) or other wireless communication technologies for direct communication, the time required for the two devices to negotiate and establish the network is relatively long, leading to high energy consumption in maintaining such a network. In contrast, communication via an Access Point (AP) involves the two devices connecting to the network through the AP, with data transmission relayed through the AP – essentially utilizing the existing communication network. Since an AP can connect to other devices besides the two directly connected, its communication resources may be contested by more devices, resulting in poor performance control and fluctuations in data transmission when the two devices transmit data through the AP. In near-field transmission, the bandwidth of a P2P architecture is often superior to that of an AP.

[0127] Therefore, in existing technologies, even if two terminals can communicate via an access point (AP) in the near field, if services requiring large data transfer volumes, such as wireless screen mirroring or file transfer, are needed, the terminals will still use a P2P architecture and common Wi-Fi networks for network construction and transmission. Using a P2P architecture for networking can still encounter situations where service requirements cannot be met, impacting user experience.

[0128] To facilitate understanding, the following examples illustrate two specific scenarios where business requirements cannot be met using Wi-Fi technology and P2P network communication. Of course, the situations where business requirements cannot be met when using a P2P architecture are not limited to the scenarios described below.

[0129] Scenario 1 corresponds to the connection establishment phase described earlier. Two terminals using a Wi-Fi-based P2P architecture need to establish a network before communicating. This process involves device discovery, service discovery, and peer-to-peer invitations, among other sub-connection procedures. Furthermore, there is no directly usable communication channel between the two terminals, resulting in a lengthy signaling interaction process (typically 4-6 seconds). Data transmission is impossible during network establishment. Additionally, the success rate of establishing a network using a P2P architecture is low; often, the first attempt fails, requiring retry, sometimes even multiple attempts. This can lead to delays exceeding 10 seconds in some cases. For example, Table 1 shows the results of six tests on the waiting time for screen mirroring from a mobile phone to test device 1 using a P2P architecture, and six tests on the waiting time for screen mirroring from a mobile phone to test device 2 using a P2P architecture. The duration in the table represents the time from when the mobile phone triggers the screen mirroring command (e.g., the user clicks the screen mirroring indicator on the screen) to when the device receiving the mirrored screen displays the mobile phone's screen, in seconds (s).

[0130] Table 1

[0131] 1 2 3 4 5 6 Test equipment 1 8.4 10.2 4.56 12.22 4.93 12.56 Test equipment 2 5.51 4.52 20.3 4.6 4.55 4.4

[0132] In practical applications, such time consumption might manifest as a long wait after one device sends a screen mirroring instruction to another before the other device can display its screen. Alternatively, it could mean that transferring a file between two devices requires several seconds of preparation before the transfer begins, resulting in long waiting times, unsmooth service initiation, and a poor user experience. Figure 2A This illustrates how, when a connection is established between a smartphone and a smart TV via a P2P architecture, indicator boxes on both devices show that a connection is in progress. This process begins when the smartphone initiates screen mirroring and continues until the smart TV displays the smartphone's screen content, indicating that screen mirroring has officially started. Imagine a user in an office setting, preparing to present to a client or colleague, and needing to project materials from their laptop onto a large electronic device (such as a smart screen) in the meeting room; for the user, this waiting time would be extremely tedious.

[0133] In existing technologies, there are considerations to improve the network establishment process using a P2P architecture to shorten the network establishment time. For example, during the network establishment process between two devices using a P2P architecture, the existing network assists in the signaling transmission during the connection establishment process through the access points already connected to the two devices, thereby speeding up the connection establishment process. However, since the network establishment process needs to comply with common documentation (such as wireless network access point technical specifications), the interaction flow cannot be arbitrarily reduced or changed. The above improvements do not simplify the connection establishment process based on P2P networks, and there are still issues where the connection cannot be established successfully on the first attempt. Therefore, the improvement effect is limited. Although it can shorten the connection establishment time to some extent, it still cannot meet the requirements of screen projection services, and the improvement to the user experience is not significant.

[0134] To address the technical problem in Scenario 1, this application describes a method for transmitting data in the near field. In response to a service request, two devices establish a first connection. During the establishment of this first connection, the two devices use an already established second connection to transmit the service's data. Once the first connection is successfully established, the data transmission switches to using that first connection. In other words, the first connection should ideally be used to transmit the service's data. However, establishing the first connection takes a considerable amount of time. Using the already established second connection to transmit data first allows for faster data transmission, shortening the response time. For the user, this translates to a faster response to user input commands (such as screen mirroring), improving the user experience. In Scenario 1, the network used for the first connection is based on a P2P architecture, while the network used for the second connection is based on an AP architecture. That is, unlike existing technologies that attempt to shorten the connection process by optimizing the P2P network connection process, the method described in this application uses another network to transmit data before establishing the connection. Without altering the P2P connection establishment process, the use of another connection further reduces the service's waiting time and improves transmission performance.

[0135] Furthermore, multipathing technology can be used in this process, allowing two sub-connections of a multipath connection to use two different networks. Once the sub-connection using the AP network is established, data is transmitted through it. The sub-connection using the P2P network takes longer to establish, but once established, the service transmitted via the AP network sub-connection is switched to the P2P network sub-connection for better transmission performance. The AP network sub-connection can be disconnected. In this way, switching data transmission from one connection to another becomes switching from one sub-connection of a connection to another sub-connection within that connection. This not only shortens waiting time and improves transmission performance but also ensures smooth data transmission and execution, resulting in a better user experience.

[0136] On the other hand, this application also discusses solutions for scenario two.

[0137] Scenario 2: Even when two devices are interacting collaboratively, issues may arise that fail to meet business requirements. This is because collaborative interaction between two devices often involves the simultaneous transmission of data for more than one service, leading to contention for network resources. Simultaneous transmission means that data for more than one service is being transmitted within a certain time period. In other words, as long as there exists a time period during which more than one service is transmitting data, it is considered simultaneous transmission. Simultaneous transmission does not require that the data for these services start or stop transmission at the same time.

[0138] When the screen of one device (a small-screen device, such as a mobile phone) is projected onto another device (a large-screen device, such as a tablet or smart TV), the large-screen device can be used as the screen for the small-screen device, and its screen is larger than the small-screen device's. Users can control the small-screen device by operating the large-screen device. For example, if the large-screen device is also a touchscreen, users can open files on the small-screen device by touching controls on its screen. Of course, the files on the small-screen device are still stored locally and need to be transferred to the large-screen device; this transfer process can be understood as a sub-process of opening the file. At this point, because the network between the two devices needs to transmit both the image from the small-screen device's screen and the file, if the network resources cannot fully meet the data transmission needs of these two services, the data from these two services will compete for network resources. As another example, if the small-screen device is also a touchscreen, users can directly trigger file transfer from the small-screen device to the large-screen device using gestures (such as dragging a finger across the screen) while the small-screen device is projecting its screen onto the large-screen device. During file transfer, other services requiring data transfer can be triggered on the large-screen device. For example, clicking on an application displayed on a small-screen device on the large-screen device initiates the application opening process. The application opening command needs to be transmitted to the small-screen device, meaning multiple services are simultaneously transferring data between the two devices, leading to competition for direct network resources. This competition can cause slow file transfers, longer waiting times, or delays in transmitting other service commands, resulting in unstable or even malfunctioning control of the small-screen device from the large-screen device. Tests revealed that in some cases, transfer-related services like file transfer exclusively utilize the direct network bandwidth to ensure rapid file transfer, while control-related services such as screen mirroring and application opening are interrupted. Regardless, the service quality of some services is significantly affected, or even prevents normal operation, and this impact is readily perceptible to the user.

[0139] To address the technical issues in Scenario 2, this application provides a near-field data transmission method. This method corresponds to the data transmission phase. During the data transmission of services using the first network, if multiple services need to be transmitted, at least one of these services is transferred to the second network for transmission. Alternatively, only one service's data can be retained for transmission on the first network, while the data of the remaining services is switched to the second network. This allows the resources of the first network to be exclusively used by a single service, enabling faster completion of data transmission for services still on the first network. For example, services still transmitted on the first network often have high performance requirements, such as higher throughput or sufficient bandwidth for rapid completion, or are large-volume data services, such as file transfer. Furthermore, services transferred to the AP network can be control-related services, such as screen projection or device operation. Alternatively, services still transmitted on the first network can have higher priority.

[0140] Of course, if the data for a service exclusively using the first network has been transmitted, but the data for a service using the second network has not yet been transmitted, the service being transmitted on the second network can be switched back to use the second network. For example, the first network could be a P2P network and the second network could be an AP network, since in near-field communication scenarios, the transmission performance of a P2P network is more stable and superior.

[0141] Furthermore, multipath technology can be used in this process. Multipath technology establishes a multipath connection between two devices, which can use two different networks, such as an AP network and a P2P network. When transmitting data for multiple services using a P2P-based sub-connection, data for at least one of the services is switched to the AP-based sub-connection. Optionally, the data switched to the AP network can be switched back to the P2P-based sub-connection. This results in smoother data transmission path switching, uninterrupted data transmission for switched services, and a better user experience.

[0142] In addition, if the transmission performance of a network cannot meet the needs of a service when transmitting data for a single service, the data for that service can be switched to another network for transmission. Once the transmission performance of the previously used network is stable, the data for the service can be switched back to the previously used network for transmission. This process can also use MP technology.

[0143] On the other hand, for scenario two, this application also describes a method for controlling a device, which is described from the perspective of the interface interaction of the device being projected. This method can be considered as the manifestation of the aforementioned method from the perspective of interface interaction.

[0144] The method includes: the device being projected displays the screen content of the projecting device; the device being projected receives a file from the projecting device; during the process of the device being projected receiving the file, the screen of the device being projected maintains the display of the screen content of the projecting device; and in response to the user's operation on the device being projected, at least a portion of the screen content of the projecting device displayed on the device being projected is changed.

[0145] Of course, it is also possible for the device being projected to display the screen content of the projecting device, and for the device being projected to receive a file from the projecting device; during the process of the device being projected to receive the file, the screen of the device being projected to maintain the display of the screen content of the projecting device, and in response to the user's operation on the projecting device, change at least a portion of the screen content of the projecting device displayed on the device being projected.

[0146] User operations on the device being projected to are essentially operations on controls or areas within the screen content displayed on the projected device. These operations can include single-clicking, double-clicking, long-pressing, knuckle tapping, or scrolling, etc., and this application does not impose any restrictions.

[0147] In other words, this operation can be performed by the user on the device being projected onto. For example, operating the screen of the projected device on a smart TV, or by the user operating the screen of the projected device itself—as long as it involves a data transmission service. This service can occur on either of the two devices. After this operation occurs, the projected device will respond promptly to the operation, rather than experiencing lag. This demonstrates that multiple data services are transmitting without interfering with each other, and the quality of service for each service is met.

[0148] In some implementations, maintaining the display of the screen content on the first device may include scaling down the screen content or changing the display format of the screen content, such as its orientation or color.

[0149] A file is a collection of data with a specific format that can be recognized or processed by an application. Files can be images, videos, audio files, compressed files, documents of various formats or types, mini-programs, web links, and so on.

[0150] During the process of the device receiving the file from the screen mirroring device, the screen mirroring does not stop, and the device receiving the file does not become unresponsive to user controls on the screen of the screen mirroring device. This reduces service lag and improves user experience. Service lag is caused by multiple services needing to transmit data, leading to contention for network resources, and some services' data failing to be transmitted to the screen mirroring device in a timely or smooth manner.

[0151] Figure 2B and Figure 2C An example of a method for controlling the device is described. Figure 2B In the image, the smartphone on the left projects the movie playing on the smart TV on the right. The movie images displayed on both the smartphone and the smart TV are identical, and both displays control components for controlling the movie playback below the movie images. The graphic displayed by these control components indicates that the movie is playing, and is shown in the image as two short vertical lines.

[0152] Figure 2C In the diagram, while the smartphone on the left is casting the movie, it also sends a file, for example, a photo stored on the smartphone, to the smart TV. Because a movie is currently playing, the photo transfer is not indicated by an icon on either device's screen. During the photo transfer, the user taps the control component on the smartphone or smart TV screen used to control the movie's playback, pausing the movie. Using this method, when the user taps the control component on the smartphone or smart TV, the movie playback on both devices pauses, and the graphics displayed on the control components on both devices simultaneously change to a paused movie icon, illustrated in the diagram as a triangle pointing to the right. Therefore, the transmission of the image data does not interrupt the casting between the two devices, and the data for the second service initiated during this process—controlling the movie pause—can also be transmitted promptly, meeting the service quality (i.e., real-time) requirements of this pause-of-movie service.

[0153] The system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solutions of this application, and do not constitute the only limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0154] The following is based on Figure 3 This describes a system architecture to which the technical solution of this application is applicable. Figure 3This system includes two devices and one forwarding device. The two devices can communicate directly (transmission path 1 in the diagram) or through the forwarding device (transmission path 2 in the diagram). The two devices can be referred to as the projection device and the second device. Both devices can send and receive data. The transmitted data includes, but is not limited to, various files such as documents, videos, audio, cloned mobile data, or images. The forwarding device can be a non-3GPP (3rd Generation Partnership Project) device, such as a Wi-Fi device. Alternatively, the forwarding device can be a 3GPP device, such as a 4G or 5G mobile communication technology base station.

[0155] The projection device and the second device can be communication devices with wireless communication capabilities, typically terminals. Terminals can include desktop, laptop, handheld, and vehicle-mounted user equipment (UE) devices. Examples include smartphones, cellular phones, smartwatches, desktop computers, tablets, smart TVs, smart TV boxes, ultra-mobile personal computers (UMPCs), laptops, personal digital assistants (PDAs), portable multimedia players (PMPs), dedicated media players, consumer communication devices, wearable devices, and AR (augmented reality) / VR (virtual reality) devices.

[0156] The aforementioned forwarding device can take the form of a complete device, a chip or chip system that can be installed in the device, or a component with the functions of the forwarding device described above. The forwarding device can be one of the following examples: an access point (AP) in a wireless fidelity (WiFi) system, such as a home gateway, router, server, switch, or bridge; an evolved Node B (eNB), radio network controller (RNC), Node B (NB), home base station (e.g., home evolved Node B, or home Node B, HNB), baseband unit (BBU), wireless relay node, wireless backhaul node, transmission and reception point (TRP or transmission point, TP), 5G, such as a gNB in ​​a new radio (NR) system, or a transmission point (TRP or TP), or one or a group of antenna panels (including multiple antenna panels) of a base station in a 5G system.

[0157] Figure 4 An implementation of the main components of two devices applicable to embodiments of the present invention is shown. The projection device 10 only illustrates its applications and protocol stack. The second device 20 illustrates the hardware it includes, specifically a processor 201, a memory 202, and a network interface card (NIC) 203. It should be understood that these hardware components in the second device 20 are also present in the projection device 10, although the specific models may differ. The network interface card 203 includes a transmitting circuit 231 and a receiving circuit 232. The memory 202 stores an executable program 21, which includes applications, a protocol stack, and an operating system. The processor 201 can run the executable program 21 in the memory 202 to implement various functions. The projection device and the second device can transmit service data using a network 30 in a near-field wireless communication environment. Both the projection device and the second device include one or more physical NICs. Each physical NIC can provide one or more network interfaces for use by the upper layers of the protocol stack, and each network interface can use one type of network. That is, the solution of this application requires at least two network interfaces. These two network interfaces can be either virtual network interface cards (NICs) or physical NICs. This application does not limit the implementation method of the network interfaces, as long as different network interfaces can be used to transmit data separately. For example, these two network interfaces belong to different NICs.

[0158] It should be understood that, Figure 4 The example shown is merely an illustration, and in practical applications, communication devices can have more than [a specific feature]. Figure 4 The more or fewer components shown can be combined into two or more components, or they can have different component configurations.

[0159] For devices 10 and 20 to establish a connection and transmit data based on that connection, multiple network protocols must be used. These protocols are layered together to form the Internet Protocol Suite (IPS), and the components that implement the functions of the IPS are called the protocol stack. For example, the data of the service generated by the second device 20 is processed by the protocol stack, encapsulated into a message, and sent to the terminal 10 through the transmit circuit 231 of the network card 203. The projection device 10 receives the message sent by the second device 20 from the network 30, processes it through the protocol stack 11, parses out the service data carried in the message, and reports it to the application 12. The protocol stack can be implemented by an appropriate combination of software, hardware, and / or firmware. The executable program 21 includes the software part of the protocol stack.

[0160] In one implementation, the protocol stack, for example Figure 4 The illustrated protocol stack 11 is a Transmission Control Protocol / Internet Protocol (TCP / IP) protocol stack. The TCP / IP protocol stack refers to a protocol stack implemented with reference to the protocol family defined by the TCP / IP reference model. This protocol family includes two core protocols: TCP (Transmission Control Protocol) and IP (Internet Protocol). According to the definition of the TCP / IP reference model, the protocols included in the protocol family are categorized into five abstract "layers": physical layer, link layer, network layer, transport layer, and application layer. The definitions of each layer are existing technologies and will not be elaborated further in this embodiment. For example, application 12 interacts with the transport layer through the Standard Socket Application Programming Interface (Standard Socket API) of the application layer. The Standard Socket Application Programming Interface is also commonly referred to as a socket, application interface, or API in the industry. APIs can have many different designs, typically including functions, constants, variables, and data structures in various implementations. The protocol stack in this embodiment is not limited to the TCP / IP protocol stack. Other protocol stacks, such as those based on the Open System Interconnection Reference Model (OSI), are also applicable to the methods described in this embodiment of the invention. Figure 5 As shown, if protocol stack 204 is a TCP / IP protocol stack, then for downlink messages from the sending end, the sending application sends data to the protocol stack, and the protocol stack adds headers and / or trailers to the data according to the protocol specifications of each layer of the TCP / IP protocol to generate the downlink message. For uplink messages from the receiving end (sent from the sending end to the receiving end), the protocol stack removes the headers and / or trailers from the uplink message according to the protocol specifications of each layer to parse the data and report it to the application. Figure 5 In the message, the data carried is the application-generated data "Hello".

[0161] This application focuses on the processing logic of the application layer and transport layer in the protocol stack. For the transport layer, it receives data from the application layer, decides on the network to use for transmitting the data, and distributes it to the corresponding network interface. Subsequent processing of this data is then performed by the protocol stack and hardware below the transport layer; specific implementation details can be found in existing data transmission processes.

[0162] The implementation of this application's method is described below using two networks, one based on a P2P architecture and the other on an AP architecture. Service data is sent down from the application layer. Specifically, this could be a plugin for the operating system, a dedicated screen mirroring application, or a functional module within a non-screen mirroring application. For example, some video, audio, and image applications already include screen mirroring functionality. The forwarding device is a Wi-Fi network access point (AP), and the screen mirroring device and the second device are two terminals. For example, the access point could be a router, the screen mirroring device a smartphone, and the second device a smart display device with a built-in or external operating system, whose screen is larger than the smartphone's.

[0163] For scenario one, the proposed data transmission method includes the following steps, and the corresponding schematic diagram is as follows. Figure 6A :

[0164] Step 601A: In response to the service request, establish a first connection based on a P2P network between the projection device and the second device to transmit the service data.

[0165] P2P networks are basically built on demand because maintaining such an architecture consumes a lot of power, and since it is a direct connection between two devices, the network cannot be established if it is uncertain which two devices will communicate.

[0166] The following uses a Wi-Fi network as an example to briefly describe the network establishment process involved in establishing a connection based on Wi-Fi. Establishing a P2P network based on Wi-Fi includes device discovery and formation. Device discovery includes the Scan Phase and the Find Phase. Formation includes the Negotiation Phase and the Provisioning Phase.

[0167] During the scanning phase, both devices send probe request frames on all supported frequency bands (e.g., bands 1, 6, and 11 of 2.4 GHz) to quickly discover nearby groups. This is called the Search State, and the frequency bands used to send probe request frames are called Social Channels. During the scanning phase, neither device processes probe request frames received from other devices. In the search phase, the two devices switch between Search State and Listen State, each controlling its own switching rhythm without synchronization. In Listen State, the device randomly selects a frequency band (e.g., one of bands 1, 6, or 11) to listen for probe request frames and replies with a probe response frame. The selected frequency band is called the Listen Channel. Once the Listen Channel is selected, it cannot be changed during the entire device discovery phase. In a Wi-Fi-based P2P architecture, the two devices are defined as a Group Owner (GO) and a Client. The Group Owner can establish P2P connections with one or more Clients. During the GO negotiation phase, the two devices negotiate who will act as the GO and negotiate some intra-group communication information, such as the operating channel that the group can use and the duration the group can maintain. During the provisioning phase, the two devices exchange security configuration information based on the results of the negotiation during device discovery. This allows clients to associate with the GO by arranging the configuration information, thereby establishing a Wi-Fi-based P2P connection.

[0168] During the scanning phase, it's impossible to respond to other devices' request frames, while the search phase often requires multiple state transitions to complete device discovery. Furthermore, if a device sends a GO negotiation request frame and doesn't receive an acknowledgment frame within 100ms, the negotiation is considered failed, and the GO negotiation request frame must be resent. Therefore, establishing a P2P network and the process of establishing a connection based on that network are time-consuming.

[0169] For example, the connection establishment process of D2D technology based on wireless cellular networks also includes device discovery, which is also time-consuming. In D2D technology, device discovery can involve the base station. In one implementation, the base station first instructs the user equipment (UE) wanting to establish a D2D network to send a discovery frame for device discovery, and then designates some UEs near that UE to receive the discovery frame. In another implementation, the base station periodically broadcasts information for establishing a D2D network to guide the two UEs. The UE wanting to establish a D2D connection sends and receives discovery frames based on the information broadcast by the base station, and also sends a response frame. Alternatively, in D2D technology, the two UEs can also perform device discovery themselves.

[0170] How to establish a P2P connection based on a network communication technology is existing technology. The technical solution of this application does not involve any improvement to this process, and will not be elaborated further here.

[0171] For the two devices, establishing a P2P network means that each device is assigned a network card and network interface corresponding to the P2P network. The identifier of the network interface corresponding to the P2P network is also registered in the operating system and is perceptible to the protocol stack. In this application, the identifier of the network interface corresponding to the P2P network is schematically designated as p2p-0. The protocol stack then uses the registered network interface to transmit data through the P2P network.

[0172] Step 602A: During the establishment of the P2P connection, the data of the service is transmitted using another established second connection. The network used by the second connection is different from that of the P2P connection.

[0173] In this embodiment, the second connection uses a Wi-Fi-based access point (AP) network. This AP network is provided by a Wi-Fi access point.

[0174] Before transmitting data using an AP network, both devices should be connected to the same AP network based on their access points, for example, the same Wi-Fi LAN. The process of connecting to the same AP network and transmitting data using the AP network is relatively quick. Because the AP network already exists, and both devices are configured with network cards and corresponding network interfaces for accessing the AP network, binding or assignment only needs to be done based on the actual name of the AP network being used. In this application, the network interface of the AP network is schematically identified as wlan-0. From another perspective, the implementation of these two steps overlaps in time; establishing a P2P connection and transmitting data using the AP network can be considered parallel processes.

[0175] The implementation of using a Wi-Fi-based access point (AP) network for data transmission can refer to existing technologies. Simply put, two devices associate with the AP device to access the local area network (LAN) where the AP is located. The process of accessing the LAN requires the associated devices to search for surrounding Wi-Fi signals and send a connection request through the AP. When using WEP or WAP protocols, both devices must enter a password to join the LAN. The AP authenticates the two devices, using Dynamic Host Configuration Protocol (DHCP) to assign Dynamic Internet Protocol (IP) addresses to them, thus completing the LAN access process. Afterwards, before establishing a connection based on the accessed AP network, device discovery is also required based on the corresponding protocol. However, because the two devices are already connected to the same AP, the pre-connection preparation instructions (such as device discovery instructions and authentication instructions) are transmitted through the AP, allowing for a rapid completion of this preparation phase. The device also uses this AP to send a request to another device to establish a connection. After receiving the request, the other device returns a response to the device that needs to transmit data through this AP, thus completing the connection.

[0176] In the above process, establishing a P2P connection and establishing an AP connection require the use of different network interface cards.

[0177] Additionally, during data transmission using an AP network, it's necessary to monitor whether the P2P network has been established or is ready. This can be achieved using existing technologies, which will not be elaborated here. For example, the following method can be used: a thread or process can monitor the status of the network interface corresponding to the P2P network or receive notifications from the operating system, and then send the acquired information to the protocol stack or application.

[0178] The second connection based on the AP network is already established compared to the first connection being established. The establishment process of this second connection can begin after the service command is issued, or it can begin at an earlier time.

[0179] Step 602A is the key point of this technical solution. First, data is transmitted using the AP network, and during this process, a P2P connection is established. This prevents data transmission from being delayed due to the slow establishment of the P2P connection.

[0180] Step 603A: After the first connection based on the P2P network is successfully established, switch to using the first connection to transmit the data of this service.

[0181] In one implementation, after the P2P network is established, the corresponding network interface is registered in the operating system. The operating system can then issue a notification to the protocol stack or application. The protocol stack can then establish a first connection between the applications running on the two devices based on the network interface allocated to the P2P network and the instructions issued by the application. Once the first connection is established, data is transmitted using the first connection. For example, the data transmitted on the second connection in step 602A can be switched to be transmitted using the first connection.

[0182] As described above, the step numbers do not indicate the order in which the steps are executed. For example, step 602A can be completed during the execution of step 601A, or the process of establishing the second connection mentioned above can be started first, followed by the process of establishing the first connection mentioned above. The key is that during the time-consuming process of establishing the first connection, the second connection is used to transmit data first.

[0183] Therefore, to address the drawback of long connection establishment times on P2P networks, the above method does not directly optimize the P2P network connection establishment process. Instead, it proposes that during the process of two devices establishing a P2P network connection, they also establish a connection based on the existing AP network and use the AP network to transmit data. Once the P2P network connection is successfully established, the method switches to using the P2P network for data transmission. This way, without changing the P2P network establishment and connection establishment process, data is transmitted using the AP network first, and both the AP network and the P2P network are used collaboratively throughout the process. This method can leverage the transmission performance advantages of P2P networks while avoiding excessively long waiting times during the P2P network connection establishment process. It can respond more quickly to user input commands (such as screen mirroring), start data transmission as soon as possible (e.g., display the screen on a device with a larger screen quickly), and shorten waiting time.

[0184] The above-described method flow represents a novel data transmission approach, which can be stored as code in the protocol stack for later invocation. In practical applications, the application can instruct the invocation of the corresponding data transmission method described in this application, thus avoiding the need to wait for the establishment of the first connection according to the existing process. The application can issue instructions to the protocol stack to initiate the process of establishing a P2P connection. The application issues instructions to the protocol stack through the aforementioned application interface.

[0185] In one implementation, a thread or process in the application layer specifies to the transport layer via an application interface that a certain connection should use the data transmission method described in this application. That is, during the establishment of a P2P connection, the establishment of an AP connection is also initiated, and data is transmitted using the previously established AP connection. This functionality can be implemented using a dedicated socket or by carrying a parameter in a general socket; this application does not impose any restrictions. For example, it can be indicated by an assignment of a general socket type or by setting the `setsocketopt` parameter. Through the aforementioned dedicated or general socket, the connection establishment instruction and the identifier of the connection to be established are passed to the transport layer. Based on the aforementioned socket, the transport layer can establish the corresponding connection and transmit data using the methods described above. In one implementation, the connection identifier can be indicated using the four-tuple of the connection; in another implementation, the connection identifier is indicated by the socket's file descriptor (FD).

[0186] In some implementations, the device can run a program to obtain the current state of the application that needs to be cast (hereinafter referred to as the casting application). For example, the state may be that the casting application is started or that it is preparing to establish a connection in a near-field environment. These states can be considered as triggering the casting request. In some implementations, the device can also run a program to determine whether the data of the service to be executed by the application is transmitted using a P2P network or a dedicated P2P network. Obtaining the state and determining the network to be used for the transmission of service data both help manage the above process, enabling the application and the operating system to clearly understand when there is a need to use the above methods. In other words, it enables the application and the operating system to clearly understand when a "casting" request should be generated.

[0187] In addition to determining whether a P2P network has been established and whether a P2P connection is ready, the device in the above method can also determine which network is more suitable for transmitting the service based on the Quality of Service (QoS) of the service to be transmitted and the transmission performance of the two networks (including at least one of bandwidth, signal strength, and latency). Of course, in some scenarios, it can be assumed that various services should use a P2P network whenever possible. These can be achieved using existing technologies, which will not be described in detail in this application.

[0188] On the other hand, for scenario two, the proposed data transmission method includes the following steps, and the corresponding schematic diagram is as follows: Figure 6B .

[0189] Step 601B: Transmit data from multiple services using a P2P-based network.

[0190] Step 602B: Switch the data of at least one of the multiple services to AP-based network transmission.

[0191] In some implementations, the method may also include step 603B: the data of the service that has not been switched to the P2P network has been transmitted. If the data of the service that has been switched to the AP network has not been transmitted, the service that has not been transmitted will be switched back to the P2P network for transmission. If the data of the service that has been switched to the AP network has been transmitted, no action is required.

[0192] The above method addresses situations where multiple services need to transmit data to another device relatively quickly, but some services require even better transmission performance. Using a P2P network for transmission across multiple services can lead to resource contention. The method described above switches at least one of these services to the AP network for transmission. Alternatively, only one service's data can be transmitted via the P2P network, while the data for the remaining services is switched to the AP network. This frees up some P2P network resources, allowing services still transmitting via the P2P network to complete the transmission more quickly and ensuring their transmission performance.

[0193] For example, services retained for transmission over a P2P network often have high requirements for certain transmission performance aspects. For instance, some scenarios require higher throughput or more bandwidth, or involve large data volumes. Another example is file transfer. Yet another example is control-related services transferred to the AP network, such as screen mirroring or device operation.

[0194] In one implementation, the device can clearly identify which services require priority for P2P network transmission or have higher requirements for certain transmission performance aspects. Specifically, a portion of the code in the application or operating system within the device can maintain the relationship between services and transmission performance; for example, the correspondence between service identifiers and the transmission networks that service should use; or it can store service priorities, which can be used to distinguish the transmission performance requirements of services. This application does not limit how the application or operating system distinguishes or indicates the need for priority for P2P network transmission or higher requirements for certain transmission performance aspects.

[0195] The business identifier is used to distinguish different businesses and can indicate the name, function, or type of the business; of course, it can also be a simple number or code. This application does not restrict the implementation method of the business identifier.

[0196] In the correspondence, the two networks can be directly indicated by "P2P" and "AP", or by the network identifier, the identifier of the corresponding network connection, the identifier of the corresponding network card or network interface for accessing the two networks, or by other parameters that correspond to the network. This application does not restrict how to indicate the transmission network.

[0197] In addition, the device can store service quality requirements metrics for various services, including at least one of bandwidth, throughput, round-trip latency, packet loss rate, and jitter.

[0198] Applications use application-layer APIs to indicate to transport layer protocol stack services with the service identifier Proc 1 that they require continuous P2P network transmission. For example, this service identifier can be carried through a dedicated API.

[0199] The transport layer determines whether the quality of service (QoS) requirements of other services (identified as Procs) currently being transmitted besides service Proc 1 can be met by the currently available access point (AP) network. These other services can be one or more. Specifically, the protocol stack can obtain performance metrics of the AP network collected by the device hardware through an interface or one or more threads or processes. These performance metrics can be at least one of signal strength, bandwidth, throughput, round-trip time, packet loss rate, and jitter. The protocol stack can calculate whether the AP network meets the QoS requirements of these other services based on the collected performance metrics. The information indicating the QoS requirements of these other services is provided by the application.

[0200] The protocol stack accesses the AP network through the network card corresponding to the AP network, and switches the data of a portion of the services currently available on the AP network to be transmitted on the AP network, while the data of service Proc 1 continues to be transmitted through the P2P network. This portion of the services can be all or part of other services.

[0201] Once the application detects that the data for service Proc 1 has been completely transmitted, it can instruct the protocol stack to switch the data of services that have switched to the AP network but have not yet completed transmission back to the P2P network for continued transmission. Specifically, "complete transmission" here can mean that the data has been sent out of the device, the data has entered the corresponding transport layer transmission queue from the application's transmission buffer, or the device has received feedback from another device indicating that the transmitted data has been received after sending the service data.

[0202] Furthermore, the process of switching data for a specific service from one network to another, as mentioned in the above method, involves using different networks. Different networks correspond to different network interface cards (NICs), as well as different send and receive queues. The protocol stack only needs to distribute the service data received from the application to the corresponding send queue of the AP network or P2P network, and then it can send the data through the appropriate network interface and NIC. Therefore, the protocol stack can switch the network used to transmit service data by switching the send queue of the service data and controlling the hardware to perform further actions. Of course, since the switched send queue corresponds to a new connection, the service data needs to be retransmitted.

[0203] In practical applications, multiple services within an application may need to start transmitting data simultaneously, but a certain portion of these services may prioritize using the P2P network for data transmission. Alternatively, during the transmission of one service, the application may trigger other services that also need to transmit data. Therefore, multiple services may need to transmit data simultaneously within a certain period of time. These services have different requirements for transmission performance, and some, as mentioned earlier, require more network resources to ensure transmission.

[0204] Although the methods described above for Scenario 1 and Scenario 2 are separate and for different scenarios, these two methods can also be used in combination. That is, the method for Scenario 1 can be used during the connection establishment phase, and the method for Scenario 2 can be used during the data transmission phase after the connection is established. In this way, the problem of low transmission performance of P2P networks in near-field transmission, which cannot meet business needs, can be solved more effectively.

[0205] In some implementations, the above method can use multipath protocol technology to allow data transmitted between two networks to be transmitted through a single multipath connection. That is, the multipath connection includes two sub-connections, each using a different network. This way, switching between the two networks will not interrupt data transmission, minimizing the impact on data delivery and resulting in smoother service implementation. Of course, in this case, both devices need to support multipath technology; for example, both devices' protocol stacks should contain code for implementing the multipath protocol.

[0206] For Scenario 1, the connection established based on the P2P network and the connection established based on the AP network are sub-connections of the same multipath connection. MP technology manages these two sub-connections uniformly at the transport layer. First, one sub-connection of the multipath connection is established using the AP network, and service data is transmitted on this sub-connection. After the other sub-connection is established using the P2P network, the service data is switched from the sub-connection using the AP network to this new sub-connection for transmission. The sub-connection based on the AP network can be disconnected or its usage priority reduced. This switching is controlled by the protocol stack supporting MP technology.

[0207] For Scenario 2, data is initially transmitted using sub-connections based on the P2P network. When a service needs to switch to AP network transmission (e.g., a new service requiring data transmission arises), data from one or more services continues to be transmitted using sub-connections based on the AP network. Once the data transmission for the service using the P2P network is complete, the data from that service or service is then transmitted again using sub-connections based on the P2P network. The protocol stack can distinguish between data from different services and can place data from different services into different transmission queues for each sub-connection.

[0208] With MP (P2P) technology, devices can switch the network used for data transmission more flexibly and conveniently. For example, one sub-connection can be interrupted, or the sub-connection used by a specific service can be changed directly, or different priorities can be set for the network interfaces corresponding to the two sub-connections, making management more convenient. In the above scenario, the network interface corresponding to the P2P sub-connection can be set to take precedence over the network interface corresponding to the AP sub-connection. Thus, when both sub-connections are available, the device can prioritize using the P2P sub-connection for data transmission. The protocol stack can also adjust the priority of sub-connections based on events or conditions. These events or conditions can be obtained from applications, underlying hardware, or other modules in the operating system.

[0209] With MP technology, the functions of the transport layer in the protocol stack are performed by the multipath transport layer, enabling the establishment and management of multipath connections. Specifically, this can be understood as the transport layer being replaced by the multipath transport layer, or as an MP layer being added to the transport layer and called the multipath transport layer. The application layer and the multipath transport layer also interact through the Standard Socket API. The multipath transport layer includes the MP layer mentioned above, which controls the entire multipath connection, and sub-connection layers, which control sub-connections. Sub-connection layers can use one or more of the aforementioned single-path transport layer protocols, such as TCP or UDP. The MP layer uses MP technology to manage the send and receive queues of the entire multipath connection, while the sub-connection layers manage the send and receive queues of multiple sub-connections. The MP layer uses a scheduler to decide how to schedule the sub-connection layers. The scheduler can access the state of each sub-connection, such as the congestion window (CWND) and round-trip time (RTT). The scheduler also distributes the data in the MP layer's send queue to the send queues of each sub-connection, and aggregates the data received by each sub-connection to the MP layer's receive queue.

[0210] From the operating system's perspective, the application layer of the protocol stack resides in the operating system's application mode; the multipath transport layer is more complex, and can be entirely in kernel mode or application mode, or partly in application mode and partly in kernel mode. The protocol stack below the multipath transport layer resides entirely in kernel mode. Figure 7 Three implementations of the multipath transport layer are illustrated. Each implementation includes an MP layer and a sub-connection layer. The sub-connection layer in each implementation is schematically shown, illustrating the two sub-connections managed by that sub-connection process. Figure 7 The diagram also illustrates the application layer and network layer adjacent to the multipath transport layer in the protocol stack. Figures 1 to 4 The illustrated device may include any of the three multipath transport layers. For simplicity, Figure 7 The application layer and network layer are illustrated uniformly across the three implementation methods. The left side represents the kernel-mode multipath transport layer, managing sub-connections using TCP or SCTP protocols; the middle side represents the user-mode MP layer, managing sub-connections using the QUIC protocol while the kernel uses the UDP protocol; and the right side represents the user-mode multipath transport layer, managing sub-connections using TCP or SCTP protocols.

[0211] The most common MP technology is the Multipath Transmission Control Protocol (MPTCP). The name of this protocol indicates that it uses MP technology and that sub-connections use the TCP protocol. Figure 8 Taking the MPTCP protocol as an example, this diagram illustrates the application layer, standard socket application programming interface, multipath transport layer (still referred to as the transport layer in the standard), scheduler, and network layer of the protocol stack. In the multipath transport layer, the MP layer's send queue includes five data fragments (labeled 1, 2, 3, 4, 5). The MP layer, through the MPTCP scheduler, distributes these five data fragments to two sub-connections respectively. Figure 8 The diagram illustrates that TCP subconnection 1 is assigned data with identifiers 1, 2, and 3, while TCP subconnection 2 is assigned data with identifiers 4 and 5.

[0212] Existing standards (such as IETF standardization document RFC.6824) contain more information about MPTCP. The description of MP technology in this application can be found in that document.

[0213] MP technology is primarily used in far-field wireless communication. In far-field environments, two devices are far apart. A link originating from one device must pass through multiple forwarding devices to reach another, and the network environment is complex and varied. In contrast, in the near-field environment described in this paper, two devices can communicate directly or through a single forwarding device. Furthermore, in far-field environments, the networks used for sub-connection data transmission, such as Wi-Fi and various cellular networks, are already deployed; devices only need to access the network via their network interface cards (NICs), without prior network setup. Therefore, existing MP technology usage scenarios do not involve the long service latency caused by a service request being sent but the network for data transmission not yet being ready, as described in this application's near-field environment. Moreover, it does not address the issue in near-field P2P architectures where a portion of the data transmitted by multiple services monopolizes transmission resources, leading to interruptions of other services or compromised service quality.

[0214] The following describes the architecture of a device to which this application applies when using MP technology, and a schematic diagram is shown. Figure 9 . Figure 9 The device 900 illustrates an application, a multi-channel management module, a multipath protocol module, a network creation module, and two network interfaces (network interface 1 and network interface 2). The multipath protocol module corresponds to the multipath transport layer described above. This application does not limit the specific implementation of the above modules. The multi-channel management module and the network creation module are management modules in the device's operating system (OS) and are typically not part of the protocol stack. Figure 9The diagram only illustrates the instruction transmission between the multi-channel management module and the network creation module and other parts, without illustrating the data transmission between the application, the multi-protocol module, and the network interface. The functions of the aforementioned application and modules can be implemented by the processor running code stored in memory, for example, using multiple threads or multiple processes. Some or all of the multiple threads can belong to the same process. The network interface requires hardware support from the network card. In one implementation, the multi-channel management module resides in the application mode of the operating system, and the network creation module resides in the kernel mode of the operating system. Communication between multiple modules can utilize various existing inter-process communication technologies. For example, communication between the multi-channel management module and the multipath protocol module can use an API, which is not described in detail in this application. The two network interfaces can be used to establish and access any of the networks mentioned above that conform to the scenario described in this application. For example, both network interfaces may correspond to the device's Wi-Fi network card, or one may correspond to the device's Wi-Fi network card, and the other to the device's cellular network card, etc.

[0215] The module division described above is merely an example. Other implementation methods will be listed below. This application does not limit the form of module division. For example, the functions of the multi-channel management module and the multipath protocol module can be implemented by a single module. Furthermore, the functions of the multipath protocol module can also be implemented in the application layer.

[0216] In one implementation, the multi-channel management module can be used to obtain event indication information from applications, which can also be understood as service requests. For example, the event indication information might indicate the need for a service that requires data transmission, such as screen mirroring or file transfer. In other words, the event indication information can trigger the process of establishing a connection. In another implementation, due to the near-field environment and relatively certain network conditions, the multi-channel management module can also estimate the performance indicators of the connection to be established. These performance indicators may include one or more of the following: signal strength of the AP access point, bandwidth, throughput, and latency. Thus, the multi-channel management module can initiate the establishment of a network connection, provided that preparatory work, such as device discovery and service discovery, is performed first. The event indication information can be issued by the application or by a module in user space related to managing the application's network connections. For example, the application network control module mentioned below is typically a process or thread. Figure 9 The text only illustrates the application.

[0217] In one implementation, the multi-channel management module can also be used to obtain network interface information from the multipath protocol module. This network interface information can be actively collected by the multipath protocol module, or it can be collected by the network interface card, driver, or module and then passed to the multipath protocol module. The network interface information can include one or more of the following: signal strength, bandwidth, throughput, and latency of the AP access point. The multi-channel management module can also obtain the quality of service (QoS) of the service; for example, QoS can be indicated by at least one of bandwidth, latency, throughput, and resolution. Thus, based on the network interface information and the QoS requirements of the service, or based on the network interface information and the service priority, the multi-channel management module can identify the service that needs to be switched to the AP-based sub-connection and pass the information of the service to be switched (e.g., service identifier) ​​to the multipath protocol module.

[0218] In one implementation, the multi-channel management module can also communicate with the network creation module to initiate the establishment of the P2P network and activate the network interface corresponding to the AP network. Of course, after the P2P network is no longer in use, to save power, the multi-channel management module can communicate with the network creation module to shut down the P2P network.

[0219] The multipath protocol module manages data transmission across multiple paths, including congestion control, data transmission control, and sub-connection priority control. In the scenario-based approach, the multipath protocol module receives instructions from the multi-channel management module, first using the AP-based sub-connection to transmit data, and then switching to the P2P-based sub-connection once it is successfully established. Similarly, in the scenario-based approach, the multipath protocol module prioritizes using the P2P-based sub-connection to transmit data for multiple services, then redirects at least one service or a newly added service to the AP-based sub-connection for transmission.

[0220] The network creation module controls the network interface to enable access to the AP network and establish a P2P network. This module can also report the status of the AP and P2P networks, primarily indicating network availability. For example, the AP network status could indicate whether AP network access is normal, and the P2P network status could indicate whether the P2P network has been established.

[0221] To enhance understanding, an embodiment of the method corresponding to Scenario 1 is described below. The process for this embodiment can be found in the following reference. Figure 10This method is applied to the device sending the data for screen mirroring (referred to as screen mirroring data), which is the source end, and the device receiving the screen mirroring is called the sink end. Furthermore, the source end and the sink end are connected to the same access point (AP). This embodiment should be read in conjunction with the preceding descriptions; details already described will not be repeated here. This embodiment includes the following steps:

[0222] S1001: Send a first instruction to the multi-channel management module. The first instruction is used to request screen projection. This instruction to request screen projection is sent by the application or the application network control module.

[0223] The first instruction carries a number or identifier corresponding to the screen mirroring service. In one implementation, the first instruction also carries an indicator of the service quality of the screen mirroring service, such as at least one of bandwidth, latency, throughput, and resolution.

[0224] Sending the first command signifies the start of the screen mirroring process. The next step involves following the prescribed procedures for device discovery and connection establishment.

[0225] S1002: Based on the first instruction, the multi-channel management module executes the device discovery process before accessing the AP network to discover the aggregation terminals connected to the same AP.

[0226] The multi-channel management module needs to call other modules or hardware at the source end to send information through the AP in order to discover the aggregation end. These other modules or hardware are not present in... Figure 10 As shown in the diagram, the information exchange process at the aggregation point can utilize the Constrained Application Protocol (CoAP).

[0227] In some implementations, capability negotiation between two devices is required based on a service request initiated by the source (e.g., a "command for screen mirroring"). This is necessary to find a suitable aggregation point for the business scenario. For example, for the screen mirroring service in this embodiment, the aggregation point needs a large screen, or even a high resolution. An AP can connect to many devices, but only devices like smart TVs may be qualified aggregation points. Capability negotiation is also completed by the multi-channel management module calling other modules or hardware on the source side to communicate with other devices connected to the same AP.

[0228] S1003: The multi-channel management module returns the identifier of the discovered aggregation point to the application or application network control module.

[0229] The identifier of the aggregation end is used to indicate an aggregation end, which can be the device identity (ID), device name, or device type of the aggregation end.

[0230] Optionally, the system can also return an identifier indicating the network accessed, as well as the network type. For example, the network type can be related to the communication technology used by the network, such as Wi-Fi, Long Term Evolution (LTE), etc.; it can also be related to the network topology, such as AP, P2P, etc.; and for example, a local area network (LAN) can also be considered a network type.

[0231] Optionally, the returned aggregator identifier can be presented to the user through the application's interface. For example, the multi-channel management module can return one or more identifiers, displayed as controls on the application interface for the source user to confirm or select, or displayed as images or text for the source user to know.

[0232] S1004: The multi-channel management module receives a second instruction, which is used to specify the aggregation end.

[0233] The second instruction comes from the application or the application network control module.

[0234] The second instruction includes an identifier of a selected aggregation point. Optionally, the second instruction may also include identifiers of one or more connections used to transmit data for the screen projection service. In one implementation, the application or application network control module may also send identifiers of the transport protocols used by these connections (e.g., TCP or UDP) to the multichannel management module.

[0235] In addition, the application or application management module also calls other modules on the source end to complete device authentication with the designated aggregation end based on the AP network.

[0236] Steps S1001-S1004 are the preparation work for AP connection, which can be implemented by existing technology.

[0237] S1005: The multi-channel management module instructs the multipath protocol module to establish a sub-connection based on the AP network so that the multipath protocol module can transmit screen projection service data based on the AP network.

[0238] The multi-channel management module can instruct the multipath protocol module to establish sub-connections based on the AP network via API.

[0239] After the AP-based sub-connection is established, the multipath protocol module uses the AP network to transmit the screen projection data without waiting for the P2P-based sub-connection to be established before starting data transmission. This allows the source screen to be displayed more quickly at the aggregation end, reducing user waiting time.

[0240] The multi-channel management module can indicate the transport layer protocol used by the sub-connection to be established, such as TCP or UDP, so that the multipath protocol module can use the corresponding protocol to establish the sub-connection.

[0241] S1006: The multi-channel management module instructs the network creation module to establish a P2P network.

[0242] The network creation module can access the software and hardware resources of the source end and communicate with the aggregation end to establish a P2P network between the source and aggregation ends. Once the P2P network is established, the network creation module can manage the network interfaces corresponding to that P2P network.

[0243] Optionally, the multi-channel management module can report to the application or application management module that the P2P network has begun to be established.

[0244] S1007: After the P2P network between the source and the aggregation ends is successfully established, the network creation module reports to the multi-channel management module that the P2P network has been established.

[0245] S1008: After receiving information indicating that a P2P network has been established, the multi-channel management module instructs the multipath protocol module to establish a sub-connection based on the P2P network.

[0246] In one implementation, the multi-channel module adds the identifier of the sub-connection corresponding to the successfully created P2P network to the list of available sub-connections of the multipath protocol module, instructing the multipath protocol module to establish a sub-connection based on the P2P network.

[0247] S1009: Once the P2P sub-connection is established, the multipath protocol module will switch the data of the screen projection service from the AP sub-connection to the P2P sub-connection for transmission.

[0248] Since the two sub-connections belong to a multipath connection of the same transport layer, the data transmission of the screen projection service from the AP sub-connection to the P2P sub-connection will not cause service interruption.

[0249] Optionally, after data transmission has changed to a different sub-connection, the multipath protocol module can reduce the priority of the AP sub-connection or interrupt the AP sub-connection.

[0250] In the above process, the actions performed by the application or the application management module can be performed entirely by the application, entirely by the application management module, or the application or the application management module can cooperate to perform a part of the action. This application does not limit the specific implementation method.

[0251] The process in this embodiment first describes the preparatory work before establishing the AP connection, and in terms of the step sequence, it first describes the initiation of AP connection creation, and then the initiation of P2P network creation. It should be understood that in other implementations, P2P network creation can be initiated first, and AP connection preparation and establishment can begin during the P2P network creation process, or AP connection creation and P2P network creation can be performed simultaneously. On the other hand, the two sub-processes of establishing the P2P network and establishing a sub-connection based on the AP network can also be performed simultaneously; or the P2P network can be established first, and then the multipath protocol module can be instructed to establish a sub-connection based on the AP network; or the multipath protocol module can be instructed to establish a sub-connection based on the AP network first, and then the P2P network can be established. This application does not impose limitations.

[0252] In summary, as long as the established AP sub-connection can be used to transmit screen-projected data during the establishment of the P2P network and P2P sub-connection (i.e., the AP sub-connection is established and put into use before the P2P sub-connection (i.e., transmitting screen-projected data), and the screen-projected data is switched to the P2P sub-connection for transmission after the P2P sub-connection is successfully established), the aggregation end can display the screen content of the source end before the P2P connection is successfully established, thereby shortening the waiting time and improving transmission performance. This application does not strictly limit the order of steps in the above embodiments. For example, using the process of the above embodiments, the aggregation end can display the screen of the source end within 1 second from the time the screen-projection command is issued from the source end. This is much faster than the prior art listed in Table 1, which requires at least 3-4 seconds to display the screen, and the user can clearly feel this difference. On the other hand, since MP technology is used, the connection will not be broken during the transmission switching process, which can also ensure the smooth transmission of business data and the smooth execution of business, resulting in a better user experience.

[0253] The following is a supplementary description of an embodiment corresponding to scenario two. The method described in this embodiment is applied to a device that sends service data. For example, the service may be a video casting service, which is to project a video stored on one device onto another device. This service involves a large amount of data and requires a long transmission time.

[0254] One device uses a P2P-based subconnection to send video to be projected to another device. Furthermore, the device's multi-channel management module evaluates the transmission performance of the P2P network in real time. Of course, other service data can be transmitted simultaneously using the P2P network for video projection.

[0255] As mentioned earlier, the multi-channel management module obtains information about the network interface corresponding to the P2P network, as well as the identifier and quality of service (QoS) of the video casting service. For example, the QoS indicator states that the video casting service requires a bandwidth of 2 Mbps or greater.

[0256] Based on this information, the multi-channel management module can calculate whether the current transmission performance of the P2P network can meet the service quality requirements of the video projection service being transmitted.

[0257] When the multi-channel management module detects that the P2P network cannot meet the quality of service (QoS) requirements of the video projection service, it switches the video projection service to sub-connection transmission based on the AP network. For example, device movement or strong interference with the P2P network can degrade its transmission performance, thus failing to meet the QoS requirements of the video projection service. This allows for flexible switching of the data transmission network, ensuring data transmission performance and service execution as much as possible under existing network conditions.

[0258] In one implementation, when the multi-channel management module detects that the transmission performance of the P2P network has recovered, the multi-channel management module instructs the multi-path module to switch the data of the video projection service back to using the P2P network for transmission.

[0259] On the other hand, if during the transmission of video projection service data, the multi-channel management module does not detect any fluctuations in the P2P network's transmission performance, but other service data also needs to be transmitted during this process, such as a new service request being generated, the multi-channel management module can notify the multi-path module to switch the new service to sub-connection transmission based on the AP network. In this case, the bandwidth required by the new service and the bandwidth required by the video projection service exceed the bandwidth available from the P2P network, and the video projection service requires significantly more bandwidth than the new service. Alternatively, the video projection service may have a higher priority than the new service. In this way, the video projection service can better utilize the P2P network's transmission resources, while other service data can still be transmitted normally, effectively utilizing the transmission resources of both networks.

[0260] The descriptions related to Scenario 2 mentioned above also apply to this embodiment, and will not be repeated here.

[0261] In one implementation, both the projection device and the second device in this application embodiment can be equipped with... Figure 4 The communication device that implements the described structure has been roughly described above, and some supplementary information is provided below. Figure 4 The diagram shown is a schematic representation of the hardware structure of the device provided in an embodiment of this application. Figure 4 The network interface card 203 illustrated is one type of communication interface. A communication interface is used to communicate with other devices. The projection device and the second device may also include other possible components of the communication interface, or the functionality of the network interface card may be implemented by other types of communication interfaces.

[0262] The processor 201 may consist of one or more processing units, which may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0263] There are communication lines between the above components for transmitting information between them.

[0264] A communication interface is used for communicating with other devices. In this embodiment, the communication interface can be a module, circuit, interface, or other device capable of communication functions, used for communicating with other devices. Optionally, the communication interface can be a separately configured transmitter used to send information to other devices, or it can be a separately configured receiver used to receive information from other devices. The communication interface can also be a component that integrates sending and receiving information functions. This embodiment does not limit the specific implementation of the communication interface.

[0265] The memory 202 can be a read-only memory (ROM) or other type of storage module capable of storing static information and instructions, random access memory (RAM) or other type of storage module capable of dynamically storing information and instructions, or it can be an electrically erasable programmable read-only memory (EEPROM), optical disk, magnetic disk, or other magnetic storage device. The memory can exist independently and be connected to the processor via communication lines. Alternatively, the memory can be integrated with the processor.

[0266] The memory 202 stores the executable program 21, or computer execution instructions. The applications, protocol stacks, various modules, and operating systems mentioned above can be implemented by the executable program 21. The executable program 21 can be invoked by one or more processing units in the processor 401 to execute the corresponding steps in the various implementations of the methods described above.

[0267] The executable program in this application embodiment may also be referred to as an instruction, computer program or other name, and this application embodiment does not specifically limit it.

[0268] As one embodiment, both the projection device and the second device can include multiple processors. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0269] As one embodiment, both the projection device and the second device may further include output devices and input devices. The output device communicates with the processor 201 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 201 and can receive user input in various ways. For example, the input device may be a mouse, keyboard, touchscreen device, or sensing device, etc.

[0270] In most real-world scenarios, both the screen mirroring device and the second device are terminals. Below, we will take... Figure 11 Taking this as an example, the structure of the terminal is described. The terminal structure described below applies to screen projection devices and second devices.

[0271] The following description uses terminal 100 as an example to illustrate the embodiment. It should be understood that... Figure 11 The terminal 100 shown is merely an example, and terminal 100 can have more than... Figure 11 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0272] Terminal 100 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include one or more of the following: pressure sensor 180A, gyroscope sensor 180B, barometric pressure sensor 180C, magnetic sensor 180D, accelerometer sensor 180E, distance sensor 180F, proximity sensor 180G, fingerprint sensor 180H, temperature sensor 180J, touch sensor 180K, ambient light sensor 180L, and bone conduction sensor 180M.

[0273] It is understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the terminal 100. In other embodiments of this application, the terminal 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0274] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0275] The controller can serve as the central nervous system and command center of the terminal 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0276] The processor 110 may also include a memory for storing the instructions and data described above. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0277] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0278] The charging management module 140 is used to receive charging input from the charger.

[0279] The wireless communication function of terminal 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. These can all be regarded as the communication interfaces described above. Some functions of the transmitting and receiving modules described below also require the cooperation of one or more of these devices.

[0280] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0281] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the terminal 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device. A network card can be considered as a form of the wireless communication module 160.

[0282] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0283] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. A network interface card (NIC) can be considered a form of the wireless communication module 160.

[0284] In some embodiments, antenna 1 of terminal 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0285] Terminal 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0286] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0287] Terminal 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0288] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the terminal 100 (such as audio data, phonebook, etc.). Furthermore, the internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0289] Terminal 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor. The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Terminal 100 can listen to music or make hands-free calls through the speaker 170A. The receiver 170B, also called a "handpiece," is used to convert audio electrical signals into sound signals. The microphone 170C, also called a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones. The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be located on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of terminal 100. A barometric pressure sensor 180C is used to measure air pressure. A magnetic sensor 180D includes a Hall effect sensor. An accelerometer 180E detects the magnitude of acceleration of the terminal 100 in various directions (typically three axes). A distance sensor 180F is used to measure distance. The terminal 100 can measure distance via infrared or laser. A proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. An ambient light sensor 180L is used to sense ambient light intensity. A fingerprint sensor 180H is used to collect fingerprints. A temperature sensor 180J is used to detect temperature. A touch sensor 180K, also called a "touch panel," can be located on the display screen 194. The touch sensor 180K and the display screen 194 together form a touchscreen, also called a "touch screen." The touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be located on the surface of the terminal 100, in a different position than the display screen 194. The bone conduction sensor 180M can acquire vibration signals. Buttons 190 include a power button, volume buttons, etc. A motor 191 can generate vibration alerts. An indicator 192 may be an indicator light, used to indicate charging status, battery level changes, and also to indicate messages, missed calls, notifications, etc. A SIM card interface 195 is used to connect a SIM card.

[0290] It is understood that, in order to achieve the aforementioned functions, the device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0291] In this application embodiment, the screen projection device and the second device described above can be divided into functional modules according to the above method example. For example, each function can be divided into a functional module, or two or more functions can be integrated into a processing module. Figure 9 This illustration represents one possible division method. The integrated modules described above can be implemented either in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Figure 12 Another method of module division for the data sending end is described. It should be understood that the data sending end is the end that sends data for a service within a connection. That is, a device can be both a data sending end and a data receiving end simultaneously, or it can be a data sending end for a certain period and a data receiving end for another period. This data sending end can be the aforementioned screen projection device or a component within the screen projection device (such as a chip system) or other components that support screen projection device functionality. The data sending end 1200 includes: a connection establishment module 1210 and a sending module 1220.

[0292] The connection establishment module 1210 is used to support the data sending end in executing S601A, S603A in the above method embodiments, and / or in the connection establishment-related processes described herein. The sending module 1120 is used to support the communication device in executing S602A, S601B, S602B in the above method embodiments, and / or in the data transmission-related processes described herein.

[0293] The connection establishment module 1210 is also used to support the communication device in performing the connection establishment-related steps in S1002-S1008 of the above method embodiments.

[0294] The sending module 1220 is also used to support the communication device in performing the actions related to the data of the sending service in S1005 and S1009 of the above method embodiments.

[0295] In other words, some of the functions of the multi-channel management module and network creation module described above can be implemented by the connection establishment module 1210, while some of the functions of the application, multipath protocol module and network interface described above can be implemented by the sending module 1220.

[0296] Optionally, the data sender may also include a storage module (not shown in the figure) to store code and business data.

[0297] Correspondingly, Figure 13 This paper describes a method for dividing the modules of a data receiving end. A device can simultaneously act as both a data sender and receiver, or it can act as a data sender for a certain period and a data receiver for another period. The data receiver can be the aforementioned second device, a component of the second device (such as a chip system), or other components supporting the functions of the second device. The data sender 1300 includes: a connection establishment module 1310 and a sending module 1320.

[0298] The connection establishment module 1310 is used to support the data receiving end in executing S601A, S603A in the above method embodiments, and / or in the connection establishment-related processes described herein. This is because the connection establishment process is a process that requires interaction and cooperation between both ends.

[0299] The receiving module 1320 is used to support the communication device in performing the steps of receiving data for various services in S602A, S601B, and S602B of the above method embodiments and / or the process related to the data of the transmission service described herein.

[0300] The data receiver may also include a storage module (not shown in the diagram) to store code and business data.

[0301] This application focuses on describing the data sending end, and the data receiving end cooperates with the data sending end. Therefore, those skilled in the art can understand the role of the data receiving end in this application. Moreover, this application also describes a data transmission system, which can better aid in understanding.

[0302] When using integrated units, the modules mentioned above, excluding the transmitting and receiving modules, can be integrated into a single processing module. This processing module can be a processor or controller, such as a CPU, general-purpose processor, DSP, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0303] The aforementioned receiving module and transmitting module can be the communication interface of a device (such as a mobile phone), specifically one or more of the following: network card, RF circuit, WiFi module, and Bluetooth module.

[0304] For example, when the connection establishment module mentioned above is Figure 4 The processor 201 shown has a storage module. Figure 4 The memory 202 shown has a transmitting / receiving module as follows: Figure 4 When the network interface card shown is used, the communication device provided in this application can be Figure 4 The second device 20 is shown. The processor, communication interface, and memory described above can be coupled together via a bus. For example, the transmitting / receiving module is... Figure 11 The antenna 1 shown is coupled to the mobile communication module 150, or the antenna 2 is coupled to the wireless communication module 160.

[0305] This application also provides a chip system applied to the aforementioned communication device. The chip system includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected via lines. The interface circuits are used to receive signals from the memory of the communication device and send the signals to the processor. The signals include computer instructions stored in the memory. When the processor executes the computer instructions, the communication device executes the method executed by the transmitting device in the above method embodiments, or executes the method executed by the forwarding device in the above method embodiments, or executes the method executed by the receiving device in the above method embodiments.

[0306] This application embodiment also provides a computer storage medium including computer instructions. When the computer instructions are executed on a communication device, the communication device performs the method executed by the transmitting device in the above method embodiment, or performs the method executed by the forwarding device in the above method embodiment, or performs the method executed by the receiving device in the above method embodiment.

[0307] This application also provides a computer program product that, when run on a computer, causes the computer to execute the method executed by the sending device in the above method embodiments, or execute the method executed by the forwarding device in the above method embodiments, or execute the method executed by the receiving device in the above method embodiments.

[0308] The communication equipment, chip system, computer storage medium or computer program product provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0309] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0310] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0311] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0312] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0313] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0314] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for transmitting data in the near field, characterized in that, The method is used in a first device, and the method includes: In response to a request from a first service, a first connection is established between the first device and the second device to send data of the first service to the second device; the request from the first service is used to instruct the processing of the first service. During the establishment of the first connection, part of the data of the first service is sent using the second connection already established between the first device and the second device. The first connection uses a directly connected wireless network, and the second connection uses an access point-based wireless network.

2. The method according to claim 1, characterized in that, The method further includes: After the first connection is established, the system switches to using the first connection to send the data of the first service to the second device.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the request from the first service, the establishment of the second connection is initiated.

4. The method according to claim 1 or 2, characterized in that, The first device uses multipath (MP) technology to communicate with the second device, and the first connection and the second connection are sub-connections of the same multipath connection.

5. The method according to claim 2, characterized in that, The method further includes: The third connection is used to send data for the second service to the second device, and the third connection uses the same wireless network as the first connection. During the transmission of data for the second service, in response to a request from the third service, the data for the third service is transmitted using a fourth connection; or in response to a request from the third service, the data for the third service is transmitted using a fifth connection, and the data for the second service is switched to be transmitted using the fourth connection, wherein the fourth connection uses the same wireless network as the second connection, and the fifth connection uses the same wireless network as the first connection.

6. The method according to claim 5, characterized in that, The combined bandwidth requirements of the second and third services are greater than the bandwidth of the third connection, and the bandwidth of the third connection is greater than the bandwidth of the fourth connection.

7. The method according to claim 5 or 6, characterized in that, If the bandwidth requirement of the second service is greater than that of the third service, the fourth connection is used to send the data of the third service; if the bandwidth requirement of the second service is less than that of the third service, the fifth connection is used to send the data of the third service, and the data of the second service is switched to be sent using the fourth connection.

8. The method according to claim 5, characterized in that, If the priority of the second service is higher than that of the third service, the data of the third service is sent using the fourth connection; if the priority of the second service is lower than that of the third service, the data of the third service is sent using the fifth connection, and the data of the second service is switched to be sent using the fourth connection.

9. The method according to any one of claims 1, 2, 5, 6, or 8, characterized in that, The first service is a screen mirroring service, and the screen area of ​​the first device is smaller than that of the second device.

10. A method for receiving data in the near field, characterized in that, The method is used in a second device, and the method includes: Based on the first instruction received from the first device, assist the first device in establishing a first connection to receive data from the first service of the first device; During the process of assisting in establishing the first connection, a portion of the data of the first service is received using the second connection already established between the first device and the second device. The first connection uses a directly connected wireless network, and the second connection uses an access point-based wireless network.

11. The method according to claim 10, characterized in that, The method further includes: After the first connection is established, the second device receives the data of the first service from the first device through the first connection.

12. The method according to claim 10 or 11, characterized in that, The method further includes: Based on the second instruction received from the first device, assist the first device in establishing the second connection.

13. The method according to claim 10 or 11, characterized in that, The communication between the second device and the first device uses multipath (MP) technology, and the first connection and the second connection are sub-connections belonging to the same multipath connection.

14. A system for transmitting data in the near field, characterized in that, The system includes a first device and a second device. The first device is used to establish a first connection between the first device and the second device in response to a request from the first service, so as to send the data of the first service to the second device; The first device is also used to send part of the data of the first service using the second connection already established between the first device and the second device during the establishment of the first connection; the request for the first service is used to instruct the processing of the first service; The second device is used to cooperate with the first device to establish the first connection and the second connection, and to receive data from the first service; The first connection uses a directly connected wireless network, and the second connection uses an access point-based wireless network.

15. The system according to claim 14, characterized in that, The first device is also used to switch to using the first connection to send the data of the first service to the second device after the first connection is established.

16. The system according to claim 14 or 15, characterized in that, The first device is also used to initiate the establishment of the second connection based on the request of the first service.

17. The system according to claim 14 or 15, characterized in that, The communication between the first device and the second device uses multipath (MP) technology, and the first connection and the second connection are sub-connections belonging to the same multipath connection.

18. The system according to claim 15, characterized in that, The first device is also used to send data of the second service to the second device using a third connection, wherein the third connection uses the same wireless network as the first connection; During the transmission of data for the second service, the first device may also respond to a request for the third service by using a fourth connection to transmit data for the third service. Alternatively, in response to a request from a third service, the fifth connection may be used to send the data for the third service, and the data for the second service may be switched to be sent using a fourth connection, wherein the fourth connection uses the same wireless network as the second connection, and the fifth connection uses the same wireless network as the first connection.

19. The system according to claim 18, characterized in that, If the bandwidth requirement of the second service is greater than that of the third service, the first device is used to send the data of the third service using the fourth connection; if the bandwidth requirement of the second service is less than that of the third service, the first device is used to send the data of the third service using the fifth connection, and to switch the data of the second service to be sent using the fourth connection.

20. The system according to claim 18, characterized in that, If the priority of the second service is higher than the priority of the third service, the first device is used to send the data of the third service using the fourth connection; if the priority of the second service is lower than the priority of the third service, the first device is used to send the data of the third service using the fifth connection, and to switch the data of the second service to be sent using the fourth connection.

21. A device, characterized in that, The device includes: a memory and a processor; the memory is coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the device to perform the method as described in any one of claims 1-9.

22. A device, characterized in that, The device includes: a memory and a processor; the memory is coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the device to perform the method as described in any one of claims 10-13.

23. A computer storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9.

24. A computer storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 10-13.

25. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9.

26. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 10-13.

Images