A method and apparatus for data transmission

Abstract

According to embodiments of the present disclosure, a data transmission method and apparatus are provided. The method comprises determining latency information of a plurality of transmission links between an autonomous vehicle and a remote assistance device, the plurality of transmission links corresponding to a plurality of network transmission devices of the autonomous vehicle; determining, based on the latency information, a target transmission mode corresponding to the plurality of transmission links, the target transmission mode at least indicating transmission links corresponding to a plurality of video streams collected by the autonomous vehicle, the plurality of video streams corresponding to different image collection devices of the autonomous vehicle; and transmitting, based on the target transmission mode, at least one of the plurality of video streams to the remote assistance device. In this way, embodiments of the present disclosure can improve the quality and efficiency of data transmission.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to a method, apparatus, device, computer-readable storage medium, and computer program product for data transmission. Background Technology

[0002] Autonomous driving is a technology that uses computers to replace or assist human drivers in perceiving the vehicle's surroundings, planning its trajectory, and controlling it to reach its destination. During an autonomous vehicle's journey, unexpected factors may disrupt its operation. In such cases, human assistance is needed to help the autonomous vehicle overcome obstacles using road condition videos uploaded by the vehicle. Therefore, efficient transmission of video data is crucial for improving the safety and reliability of autonomous driving operations. Summary of the Invention

[0003] In a first aspect of this disclosure, a data transmission method is provided. The method includes: determining latency information of multiple data transmission links between an autonomous vehicle and a remotely assisted device, the multiple transmission links corresponding to multiple network transmission devices of the autonomous vehicle; determining a target transmission mode corresponding to the multiple transmission links based on the latency information, the target transmission mode indicating at least the data transmission links corresponding to multiple video streams acquired by the autonomous vehicle, the multiple video streams corresponding to different image acquisition devices of the autonomous vehicle; and sending at least one video stream from the multiple video streams to the remotely assisted device based on the target transmission mode.

[0004] In a second aspect of this disclosure, a data transmission apparatus is provided. The apparatus includes: a latency determination module configured to determine latency information of multiple data transmission links between an autonomous vehicle and a remote assistance device, the multiple transmission links corresponding to multiple network transmission devices of the autonomous vehicle; a transmission mode determination module configured to determine a target transmission mode corresponding to the multiple transmission links based on the latency information, the target transmission mode indicating at least the data transmission links corresponding to multiple video streams acquired by the autonomous vehicle, the multiple video streams corresponding to different image acquisition devices of the autonomous vehicle; and a transmission module configured to transmit at least one of the multiple video streams to the remote assistance device based on the target transmission mode.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0008] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart illustrating data transmission according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic structural block diagram of an example apparatus for data transmission according to certain embodiments of the present disclosure is shown; and

[0013] Figure 4 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0016] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0018] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0019] As briefly mentioned earlier, autonomous driving is a technology that uses computers to replace or assist human drivers in perceiving the vehicle's surroundings, planning its trajectory, and controlling it to reach its designated destination. During an autonomous vehicle's journey, unexpected factors may disrupt its operation. In such cases, human assistance is needed to help the autonomous vehicle overcome obstacles using road condition videos uploaded by the vehicle. Therefore, efficient transmission of video data is a crucial aspect of improving the safety and reliability of autonomous driving operations.

[0020] Embodiments of this disclosure propose a data transmission scheme. According to various embodiments of this disclosure, latency information of multiple transmission links between an autonomous vehicle and a remotely assisted device can be determined, where each transmission link corresponds to multiple network transmission devices of the autonomous vehicle; based on the latency information, a target transmission mode corresponding to the multiple transmission links is determined, where the target transmission mode at least indicates the data transmission link corresponding to multiple video streams acquired by the autonomous vehicle, where each video stream corresponds to different image acquisition devices of the autonomous vehicle; and based on the target transmission mode, at least one video stream from the multiple video streams is sent to the remotely assisted device.

[0021] In this way, embodiments of the present disclosure can determine the target transmission mode based on the latency information of multiple transmission links, and perform data transmission based on the target transmission mode. Therefore, embodiments of the present disclosure can improve the quality and efficiency of data transmission, thereby enhancing the travel safety of autonomous vehicles.

[0022] Example Environment

[0023] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Environment 100 includes an autonomous vehicle 101, a remote device 120, and a user interface 130. Electronic devices 110 and multiple network transmission devices are deployed in the autonomous vehicle 101. Electronic devices 110 can communicate with the remote device 120 via the multiple network transmission devices. For example, electronic devices 150 can communicate with the remote device 120 via appropriate wireless communication methods. Electronic devices 150 can be any computing-capable device that can control the autonomous vehicle 101 according to target instructions issued by the remote device 120.

[0024] In some embodiments, the electronic device 110 can acquire latency information of multiple transmission links corresponding to multiple network transmission devices between the autonomous vehicle 101 and the remote device 120. Further, the electronic device 110 can determine a target transmission mode based on the latency information and send multiple video streams to the remote device 120 based on the target transmission mode, so that the multiple video streams are displayed in the user interface 130.

[0025] In environment 100, assistant 140 can interact directly with remote device 120, or via an attached device to remote device 120. Remote device 120 can present user interface 130 to assistant 110 for viewing multiple video streams transmitted by autonomous vehicle 101.

[0026] The remote device 120 may include, for example, a cloud device or an edge computing device. In some embodiments, such a remote device 120 may acquire multiple video feeds captured by the vehicle 101 to present at least one video feed on the user interface 130.

[0027] In some embodiments, the autonomous vehicle 101 can be any type of vehicle capable of carrying people and / or goods and moving via a power system such as an engine, including but not limited to cars, trucks, buses, electric vehicles, motorhomes, etc. The autonomous vehicle 101 can be an automated driving vehicle (also known as an autonomous vehicle) that integrates functions such as environmental perception, planning and decision-making, and multi-level assisted driving.

[0028] It should be understood that the structure and function of environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0029] Example process

[0030] Figure 2 A flowchart illustrating an example process 200 for data transmission according to some embodiments of the present disclosure is shown. Process 200 can be implemented at an autonomous vehicle 101, for example, in electronic devices 110 deployed within the autonomous vehicle 101. Reference will be made below. Figure 1 To describe an example data transmission flow 200 according to some embodiments of the present disclosure.

[0031] like Figure 2 As shown, in block 210, electronic device 110 determines the latency information of multiple transmission links between the autonomous vehicle and the remote assistance device, the multiple transmission links corresponding to multiple network transmission devices of the autonomous vehicle.

[0032] As an example, an autonomous vehicle 101 deploys electronic devices 110 and multiple network transmission devices. These network transmission devices can be deployed within or independently of the electronic devices 110 and communicate with them. For example, a network interface card (NIC) deployed within the autonomous vehicle 101 can be used. The electronic devices 110 can establish multiple transmission links with a remote device 120 (i.e., a remote assistance device) through these network transmission devices. Latency information can indicate the network latency between the electronic devices 110 and the remote device 120 (i.e., the remote assistance device).

[0033] Electronic device 110 can detect the latency information of multiple transmission links using any appropriate network detection tool (e.g., Ping, Traceroute, or MTR).

[0034] Continue to refer to Figure 2In box 220, electronic device 110 determines the target transmission mode corresponding to multiple transmission links based on time delay information. The target transmission mode at least indicates the transmission link corresponding to the multiple video streams collected by the autonomous vehicle. The multiple video streams correspond to different image acquisition devices of the autonomous vehicle.

[0035] As an example, after obtaining delay information from multiple transmission links, electronic device 110 can determine a target transmission mode that matches the delay information from a preset set of transmission modes. The target transmission mode could, for example, indicate that transmission link A is used to transmit video stream A.

[0036] In some embodiments, the multiple video streams include a forward-view video stream, a rear-view video stream, a surround-view video stream, and a trigger video stream. The forward-view video stream indicates the video view in front of the autonomous vehicle. The rear-view video stream indicates the video view behind the autonomous vehicle. The surround-view video stream indicates multi-directional video views captured from above the autonomous vehicle. The trigger video stream is configured to be triggered and presented at the remote assistance device based on the autonomous vehicle's driving scenario.

[0037] As an example, the multiple video streams can be captured by camera equipment deployed in the autonomous vehicle 101, including video footage from multiple angles of the autonomous vehicle 101. In some scenarios, the left side of the autonomous vehicle 101 is scraped (i.e., the driving scenario is an accident), in which case the electronic device 110 can control the camera equipment to actively capture the left-side video stream of the autonomous vehicle 101. Furthermore, the electronic device 110 can transmit the left-side video stream of the autonomous vehicle 101 to the remote device 120, so that the left-side video stream can be presented as a trigger video stream in the remote device 120 (i.e., the remote assistance device).

[0038] In some embodiments, the target transmission mode also indicates the transmission quality of the target video stream, including resolution and / or bitrate. As an example, electronic device 110 can detect network latency across multiple transmission links. When the detected network latency is low, electronic device 110 can transmit multiple video streams of the autonomous vehicle 101 at a higher quality (e.g., 4K resolution, 120 frames per second). When the detected network latency is high, electronic device 110 can transmit multiple video streams of the autonomous vehicle at a lower quality (e.g., 360p resolution, 30 frames per second).

[0039] In some embodiments, the electronic device 110 can determine a primary transmission link and an auxiliary transmission link from a plurality of transmission links. Specifically, the plurality of transmission links include a primary transmission link and an auxiliary transmission link, wherein the first historical delay of the primary transmission link is lower than the second historical delay of the auxiliary transmission link.

[0040] As an example, electronic device 110 can acquire the historical latency of multiple transmission links. Furthermore, electronic device 110 can determine the primary and secondary transmission links based on the historical latency. By dividing the transmission links into primary and secondary links, data transmission tasks can be allocated to them in a more organized manner, thereby improving data transmission efficiency.

[0041] In some embodiments, the electronic device 110 can determine a target transmission mode based on a first transmission delay of the primary transmission link and a second transmission delay of the secondary transmission link. Specifically, the electronic device 110 can determine the target transmission mode based on a comparison of the first and second transmission delays with corresponding thresholds. The target transmission mode indicates the data transmitted by the primary and secondary transmission links respectively.

[0042] As an example, when a first transmission delay is less than a threshold and a second transmission delay is greater than a threshold, the electronic device 110 can determine a target transmission mode from a set of preset transmission modes. The target transmission mode indicates the data transmitted by the main transmission link and the auxiliary transmission link, respectively. For example, the main transmission link transmits a forward-looking video stream and a surround-view video stream, while the auxiliary transmission link transmits a rear-view video stream and a sync video stream.

[0043] In some embodiments, the electronic device 110 may determine that multiple data transmission links correspond to a first transmission mode in response to the latency information indicating that the transmission latency of multiple data transmission links is greater than a first threshold. The first transmission mode indicates that the main transmission link is used to transmit the forward-looking video stream and the induction video stream of the autonomous vehicle, and the auxiliary transmission link is used to transmit the surround-view video stream and the rear-view video stream of the autonomous vehicle.

[0044] As an example, when the network latency (i.e., transmission latency) of multiple data transmission links is greater than a first threshold, electronic device 110 can use a first transmission mode for data transmission. The first threshold can be, for example, 300 milliseconds. After electronic device 110 uses the first transmission mode for data transmission, it can evenly distribute the data to be transmitted, that is, transmit the forward-looking video stream and the initiation video stream through the main transmission link, and transmit the surround-view video stream and the rear-view video stream through the auxiliary transmission links.

[0045] In some embodiments, the electronic device 110 may determine that multiple transmission links correspond to a second transmission mode in response to a first transmission delay being greater than a first threshold and a second transmission delay being less than the first threshold. The second transmission mode indicates that the main transmission link is used to transmit the forward-looking video stream of the autonomous vehicle, and the auxiliary transmission link is used to transmit the surround-view video stream, the rear-view video stream, and the sync video stream of the autonomous vehicle.

[0046] As an example, when the delay of the main transmission link is large (i.e., greater than the first threshold) but the delay of the auxiliary transmission link is small (i.e., less than the first threshold), the electronic device 110 can allow the auxiliary transmission link to undertake more transmission tasks and reduce the transmission tasks of the main transmission link.

[0047] In some embodiments, the electronic device 110 may determine that multiple transmission links correspond to a third transmission mode in response to a first transmission delay not exceeding a first threshold and a second transmission delay exceeding a second threshold. The third transmission mode indicates that the primary transmission link is used to transmit the forward-looking video stream, the induction video stream, and the surround-view video stream of the autonomous vehicle. The auxiliary transmission link is used to transmit the rear-view video stream of the autonomous vehicle.

[0048] As an example, when the latency of the primary transmission link is relatively small (i.e., less than the first threshold), but the latency of the secondary transmission link is relatively large (i.e., greater than the second threshold), the electronic device 110 can allow the primary transmission link to handle more transmission tasks and reduce the transmission tasks of the secondary transmission link. The second threshold can be, for example, 400 milliseconds.

[0049] In some embodiments, the electronic device 110 may determine that multiple transmission links correspond to a fourth transmission mode in response to a first delay greater than a first threshold and a second delay greater than a second threshold. The fourth transmission mode indicates that the primary transmission link is used to transmit the forward-looking video stream and the induction video stream of the autonomous vehicle. The auxiliary transmission links are used to transmit the rear-view video stream and the surround-view video stream according to a preset quality, which is lower than the acquisition quality of the rear-view video stream and the surround-view video stream.

[0050] As an example, when both the primary and secondary transmission links have high latency, electronic device 110 can reduce the quality of data transmission. For instance, the acquisition quality of the rearview video stream and the surround view video stream is 4K resolution at 120 frames per second. When both the primary and secondary transmission links have high latency, electronic device 110 can transmit the rearview video stream and the surround view video stream at 360p resolution and 120 frames per second via the secondary transmission link.

[0051] In some embodiments, the electronic device 110 may determine that multiple transmission links correspond to a fifth transmission mode in response to the condition that, after a first delay detection period, the first delay is still greater than a first threshold and the second delay is still greater than a second threshold. The fifth transmission mode indicates that: the primary transmission link is used to transmit the forward-looking video stream and transmits the initiation video stream according to a preset quality; the auxiliary transmission link is used to transmit the backward-looking video stream according to a preset quality; and the surround-view video stream is not transmitted.

[0052] As an example, after one latency detection cycle, if electronic device 110 detects that the first latency is still greater than the first threshold and the second latency is still greater than the second threshold, electronic device 110 can further reduce the data transmission quality based on the fourth transmission mode. For example, it can stop transmitting the surround-view video stream and reduce the transmission quality of the initiating video stream. The latency detection cycle can be, for example, once every 2 seconds.

[0053] In some embodiments, the electronic device 110 may determine that multiple transmission links correspond to a sixth transmission mode in response to the following: after a second delay detection period, the first delay is still greater than a first threshold and the second delay is still greater than a second threshold. The sixth transmission mode indicates that the main transmission link is used to transmit the forward-looking video stream and the initiation video stream according to a preset quality. The auxiliary transmission links stop transmitting data. The rear-looking video stream and the surround-looking video stream are not transmitted.

[0054] As an example, if the network latency of the main transmission link and the auxiliary transmission link is still high after the second latency detection period (i.e., the first latency is greater than the first threshold and the second latency is greater than the second threshold), the electronic device 110 can transmit data using the sixth transmission mode.

[0055] In some embodiments, the electronic device 110 may determine that multiple transmission links correspond to a seventh transmission mode in response to the condition that, after a third delay detection period, a first delay is still greater than a first threshold and a second delay is still greater than a second threshold. The seventh transmission mode indicates that the primary transmission link is used to transmit the forward-looking video stream according to a preset quality. The auxiliary transmission links stop transmitting data. The initiation video stream, the rear-view video stream, and the surround-view video stream are not transmitted.

[0056] As an example, if the primary and secondary transmission links still maintain high network latency after three latency detection cycles, electronic device 110 will only transmit the primary forward-looking video stream to remote device 120 according to a preset quality to ensure that the forward-looking video stream can be displayed in remote device 120.

[0057] By dynamically changing the data transmission mode based on the latency information of multiple transmission links, the transmission quality and requirements of key data can be ensured, thereby guaranteeing the stability and security of data transmission.

[0058] Continue to refer to Figure 2 In box 230, electronic device 110 sends at least one video stream from a set of multiple video streams to a remote device based on a target transmission mode. As an example, after determining the target transmission mode, electronic device 110 can transmit video streams to remote device 120 based on the association between the multiple video streams indicated by the target transmission mode and multiple transmission links, and a preset transmission quality.

[0059] In some embodiments, the electronic device 110 may also send perception information associated with the autonomous vehicle to the remote assistance device based on a target transmission mode, so that the remote assistance device can present visualization information associated with the environmental information of the autonomous vehicle based on the perception information.

[0060] As an example, when network latency across multiple transmission links is poor, electronic device 110 can transmit only a portion of the perceived information. For instance, the perceived information collected by autonomous vehicle 101 is road condition information within a 20-meter radius. When network latency across multiple transmission links is poor, electronic device 110 can send only the road condition information within a 10-meter radius to remote device 120 (i.e., remote assistance device). Upon receiving the road condition information (i.e., perceived information), remote device 120 can present a map (i.e., visualization information) related to the autonomous vehicle's environment based on the road condition information (i.e., perceived information). In this way, the data transmission pressure across multiple transmission links can be reduced.

[0061] In this way, embodiments of the present disclosure can determine the target transmission mode based on the latency information of multiple transmission links, and perform data transmission based on the target transmission mode. Therefore, embodiments of the present disclosure can improve the quality and efficiency of data transmission, thereby enhancing the travel safety of autonomous vehicles.

[0062] Example devices and equipment

[0063] Figure 3 A schematic structural block diagram of a data transmission apparatus 300 according to certain embodiments of the present disclosure is shown. The apparatus 300 may be implemented as or included in an autonomous vehicle 101. The various modules / components in the apparatus 300 may be implemented by hardware, software, firmware, or any combination thereof.

[0064] As shown in the figure, the device 300 includes a latency determination module 310, configured to determine latency information of multiple data transmission links between the autonomous vehicle and the remote assistance device, wherein the multiple transmission links correspond to multiple network transmission devices of the autonomous vehicle; a transmission mode determination module 320, configured to determine a target transmission mode corresponding to the multiple transmission links based on the latency information, wherein the target transmission mode at least indicates the data transmission links corresponding to the multiple video streams acquired by the autonomous vehicle, wherein the multiple video streams correspond to different image acquisition devices of the autonomous vehicle; and a transmission module 330, configured to send at least one video stream from the multiple video streams to the remote assistance device based on the target transmission mode.

[0065] In some embodiments, the multiple video streams include a number of the following: a forward-view video stream indicating the video view in front of the autonomous vehicle; a rear-view video stream indicating the video view behind the autonomous vehicle; a surround-view video stream indicating multi-directional video views captured from above the autonomous vehicle; and a trigger video stream configured to be triggered and presented at the remote assistance device based on the autonomous vehicle's driving scenario.

[0066] In some embodiments, the target transmission mode also indicates the transmission quality of the target video stream, including resolution and / or bitrate.

[0067] In some embodiments, the multiple transmission links include a primary transmission link and an auxiliary transmission link, wherein the first historical delay of the primary transmission link is lower than the second historical delay of the auxiliary transmission link.

[0068] In some embodiments, the delay information indicates a first transmission delay of the primary transmission link and a second transmission delay of the secondary transmission link. The transmission mode determination module 320 is further configured to: determine a target transmission mode based on a comparison of the first transmission delay, the second transmission delay, and a corresponding threshold. The target transmission mode indicates the data transmitted by the primary transmission link and the secondary transmission link, respectively.

[0069] In some embodiments, the transmission mode determination module 320 is further configured to: in response to the delay information indicating that the transmission delay of multiple data transmission links is greater than a first threshold, determine that the multiple transmission links correspond to a first transmission mode, wherein the first transmission mode indicates that: the main transmission link is used to transmit the forward-looking video stream and the initiation video stream of the autonomous vehicle, and the auxiliary transmission link is used to transmit the surround-view video stream and the rear-view video stream of the autonomous vehicle.

[0070] In some embodiments, the transmission mode determination module 320 is further configured to: in response to a first transmission delay being greater than a first threshold and a second transmission delay being less than the first threshold, determine multiple transmission links corresponding to a second transmission mode, wherein the second transmission mode indicates that: the main transmission link is used to transmit the forward-looking video stream of the autonomous vehicle, and the auxiliary transmission link is used to transmit the surround-view video stream, the rear-view video stream, and the induction video stream of the autonomous vehicle.

[0071] In some embodiments, the transmission mode determination module 320 is further configured to: in response to a first transmission delay not exceeding a first threshold and a second transmission delay exceeding a second threshold, determine multiple transmission links corresponding to a third transmission mode, wherein the third transmission mode indicates that: the main transmission link is used to transmit the forward-looking video stream, the induction video stream, and the surround-view video stream of the autonomous vehicle, and the auxiliary transmission link is used to transmit the rear-view video stream of the autonomous vehicle.

[0072] In some embodiments, the transmission mode determination module 320 is further configured to: in response to a first transmission delay being greater than a first threshold and a second transmission delay being greater than a second threshold, determine multiple transmission links corresponding to a fourth transmission mode, wherein the fourth transmission mode indicates that: the main transmission link is used to transmit the forward-looking video stream and the initiation video stream of the autonomous vehicle, and the auxiliary transmission link is used to transmit the rear-looking video stream and the surround-looking video stream according to a preset quality, wherein the preset quality is lower than the acquisition quality of the rear-looking video stream and the surround-looking video stream.

[0073] In some embodiments, the apparatus 300 further includes a first determining module configured to determine, in response to a first transmission delay still being greater than a first threshold and a second transmission delay still being greater than a second threshold after a first delay detection period, a plurality of transmission links corresponding to a fifth transmission mode, wherein the fifth transmission mode indicates that: the main transmission link is used to transmit the forward-looking video stream and transmits the initiation video stream according to a preset quality, the auxiliary transmission link is used to transmit the rear-looking video stream according to a preset transmission quality, and the surround-looking video stream is not transmitted.

[0074] In some embodiments, the device 300 further includes a second determining module configured to determine multiple transmission links corresponding to a sixth transmission mode in response to a first transmission delay still being greater than a first threshold and a second transmission delay still being greater than a second threshold after a second delay detection period. The sixth transmission mode indicates that the main transmission link is used to transmit the forward-looking video stream and the initiation video stream according to a preset quality, the auxiliary transmission link stops transmitting data, and the rear-looking video stream and the surround-looking video stream are not transmitted.

[0075] In some embodiments, the device 300 further includes a third determining module configured to determine multiple transmission links corresponding to a seventh transmission mode in response to a first transmission delay still being greater than a first threshold and a second transmission delay still being greater than a second threshold after a third delay detection period. The seventh transmission mode indicates that the main transmission link is used to transmit the forward-looking video stream according to a preset quality, the auxiliary transmission link stops transmitting data, and the initiation video stream, the rear-view video stream, and the surround-view video stream are not transmitted.

[0076] In some embodiments, the apparatus 300 further includes a perception information transmission module configured to transmit perception information associated with the autonomous vehicle to a remote device based on a target transmission mode, so that the remote device presents visual information associated with the autonomous vehicle's environment based on the perception information.

[0077] Figure 4 A block diagram illustrating a computing device 400 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 4 The computing device 400 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 4 The computing device 400 shown can be used to implement Figure 1 Electronic devices 110.

[0078] like Figure 4 As shown, computing device 400 is in the form of a general-purpose computing device. Components of computing device 400 may include, but are not limited to, one or more processors or processing units 410, memory 420, storage devices 430, one or more communication units 440, one or more input devices 450, and one or more output devices 460. Processing unit 410 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 420. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 400.

[0079] Computing device 400 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to computing device 400, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 420 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 430 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 400.

[0080] The computing device 400 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 4 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 420 may include computer program product 425 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0081] The communication unit 440 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 400 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 400 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0082] Input device 450 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 460 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 400 can also communicate as needed with one or more external devices (not shown) via communication unit 440. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 400, or with any device (e.g., network card, modem, etc.) that enables computing device 400 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0083] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0084] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0085] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0086] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A data transmission method, comprising: Determine the latency information of multiple data transmission links between the autonomous vehicle and the remote assistance device. These multiple transmission links correspond to multiple network transmission devices of the autonomous vehicle. Based on latency information, the target transmission mode corresponding to multiple transmission links is determined. The target transmission mode at least indicates the data transmission link corresponding to the multiple video streams collected by the autonomous vehicle. The multiple video streams correspond to different image acquisition devices of the autonomous vehicle. as well as Based on the target transmission mode, send at least one video stream from multiple video streams to the remote assistance device.

2. The method according to claim 1, wherein the multiple video streams include a number of the following: A forward-looking video stream, wherein the forward-looking video stream indicates the video view in front of the front of the autonomous vehicle; Rearview video stream, the rearview video stream indicating the video view behind the rear of the autonomous vehicle; A surround-view video stream, which indicates multi-directional video footage captured from above the autonomous vehicle; A video stream is invoked, which is configured to be triggered and presented on the remote assistance device based on the driving scenario of the autonomous vehicle.

3. The method of claim 1, wherein the target transmission mode further indicates the transmission quality of the target video stream, the transmission quality including resolution and / or bitrate.

4. The method according to claim 1, wherein the plurality of transmission links include a primary transmission link and an auxiliary transmission link, wherein the first historical delay of the primary transmission link is lower than the second historical delay of the auxiliary transmission link.

5. The method according to claim 4, wherein the delay information indicates a first transmission delay of the primary transmission link and a second transmission delay of the auxiliary transmission link, and determining the target transmission mode corresponding to the plurality of transmission links based on the delay information includes: Based on the comparison between the first transmission delay and the second transmission delay and the corresponding threshold, the target transmission mode is determined, and the target transmission mode indicates the data transmitted by the main transmission link and the auxiliary transmission link respectively.

6. The method according to claim 5, wherein determining the target transmission mode based on a comparison of the first transmission delay and the second transmission delay with a corresponding threshold comprises: In response to the delay information indicating that the transmission delays of the multiple data transmission links are all greater than a first threshold, it is determined that the multiple transmission links correspond to a first transmission mode, wherein the first transmission mode indicates: The main transmission link is used to transmit the forward-looking video stream and the induction video stream of the autonomous vehicle. The auxiliary transmission link is used to transmit the surround-view video stream and the rear-view video stream of the autonomous vehicle.

7. The method according to claim 5, wherein determining the target transmission mode based on a comparison of the first transmission delay and the second transmission delay with a corresponding threshold comprises: In response to the first transmission delay being greater than a first threshold and the second transmission delay being less than the first threshold, it is determined that the plurality of transmission links correspond to a second transmission mode, wherein the second transmission mode indicates: The main transmission link is used to transmit the forward-looking video stream of the autonomous vehicle. The auxiliary transmission link is used to transmit the surround view video stream, rear view video stream, and sync video stream of the autonomous vehicle.

8. The method according to claim 5, wherein determining the target transmission mode based on a comparison of the first transmission delay and the second transmission delay and a corresponding threshold comprises: In response to the first transmission delay not being greater than a first threshold and the second transmission delay being greater than a second threshold, it is determined that the plurality of transmission links correspond to a third transmission mode, the third transmission mode indicating: The main transmission link is used to transmit the forward-looking video stream, the induction video stream, and the surround-view video stream of the autonomous vehicle. The auxiliary transmission link is used to transmit the rear-view video stream of the autonomous vehicle.

9. The method according to claim 5, wherein determining the target transmission mode based on a comparison of the first transmission delay and the second transmission delay with a corresponding threshold comprises: In response to the first transmission delay being greater than a first threshold and the second transmission delay being greater than a second threshold, it is determined that the plurality of transmission links correspond to a fourth transmission mode, the fourth transmission mode indicating: The main transmission link is used to transmit the forward-looking video stream and the induction video stream of the autonomous vehicle. The auxiliary transmission link is used to transmit rear-view video streams and surround-view video streams according to a preset quality, wherein the preset quality is lower than the acquisition quality of the rear-view video streams and surround-view video streams.

10. The method of claim 9, further comprising: In response to the situation where, after the first delay detection period, the first transmission delay is still greater than the first threshold and the second transmission delay is still greater than the second threshold, it is determined that the plurality of transmission links correspond to a fifth transmission mode, wherein the fifth transmission mode indicates: The main transmission link is used to transmit the forward-looking video stream, and the video stream is triggered according to a preset quality. The auxiliary transmission link is used to transmit the rear-view video stream according to a preset transmission quality. The surround view video stream is not transmitted.

11. The method of claim 10, further comprising: In response to the situation where, after the second delay detection period, the first transmission delay is still greater than the first threshold and the second transmission delay is still greater than the second threshold, it is determined that the plurality of transmission links correspond to a sixth transmission mode, the sixth transmission mode indicating: The main transmission link is used to transmit the forward-looking video stream and the initiation video stream according to a preset quality. The auxiliary transmission link stops transmitting data. Rear view and surround view video streams are not transmitted.

12. The method of claim 11, further comprising: In response to the situation where, after the third delay detection period, the first transmission delay is still greater than the first threshold and the second transmission delay is still greater than the second threshold, it is determined that the plurality of transmission links correspond to a seventh transmission mode, the seventh transmission mode indicating: The main transmission link is used to transmit the forward-looking video stream according to a preset quality. The auxiliary transmission link stops transmitting data. The initiation video stream, rearview video stream, and surround view video stream are not transmitted.

13. The method according to claim 1, further comprising: Based on the target transmission mode, perception information associated with the autonomous vehicle is sent to the remote device, so that the remote device can present visualization information associated with the environment of the autonomous vehicle based on the perception information.

14. An apparatus for data transmission, comprising: The latency determination module is configured to determine the latency information of multiple data transmission links between the autonomous vehicle and the remote assistance device, where the multiple transmission links correspond to multiple network transmission devices of the autonomous vehicle. The transmission mode determination module is configured to determine the target transmission mode corresponding to multiple transmission links based on latency information. The target transmission mode at least indicates the data transmission link corresponding to the multiple video streams acquired by the autonomous vehicle. The multiple video streams correspond to different image acquisition devices of the autonomous vehicle. as well as The sending module is configured to send at least one video stream from a set of multiple video streams to a remote assistance device based on the target transmission mode.

15. An electronic device comprising: At least one processing unit; as well as At least one memory is coupled to at least one processing unit and stores instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 13.

17. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 13.

Images