Video stream processing method, device, electronic equipment, vehicle and system

Abstract

The invention provides a video stream processing method and device, electronic equipment, a vehicle and a system, and relates to the technical field of artificial intelligence, in particular to the fields of automatic driving, cloud designated driving, video processing and the like. According to the specific implementation scheme, a target leaf node is determined from a decision tree model according to the range of the numerical value of at least one network performance parameter for a vehicle, and the target leaf node represents the network quality for the vehicle; determining a target coding strategy from the plurality of candidate coding strategies according to the network quality; the target coding strategy comprises at least one video stream identifier used for indicating at least one target video stream to be coded in the plurality of original video streams, and respective code rates of the at least one target video stream; and according to the respective code rate of the at least one target video stream, encoding the at least one target video stream indicated by the at least one video stream identifier to obtain at least one encoded video stream.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the fields of automatic driving, cloud driving, video processing, and the like, and more specifically, the present disclosure provides a video stream processing method, a video stream processing apparatus, an electronic device, a storage medium, a computer program product, an automatic driving vehicle, and a video stream processing system. BACKGROUND

[0002] In the cloud driving scenario, the vehicle needs to continuously send multiple video streams to the remote device through the network, so as to show the environmental information around the vehicle, and the remote driver makes driving decisions through the multiple video streams and controls the vehicle to drive.

[0003] In some technical solutions, a fixed code rate and a fixed number of paths can be pre-configured. After the driver remotely takes over the vehicle, the vehicle encodes the original video stream of the fixed number of paths according to the fixed code rate, and then sends the encoded video stream to the remote device.

[0004] However, in actual application, the network condition of the vehicle will fluctuate sharply due to movement, signal switching, etc. The fixed code rate or fixed number of paths cannot adapt to such changes. When the network quality of the vehicle is high, the excess bandwidth cannot be fully utilized to improve the quality of the video stream, which wastes resources. When the network quality of the vehicle is poor, all video streams are still transmitted, so the quality of each video stream cannot be guaranteed, which can easily cause all video streams to be stalled, black screen or even interrupted at the same time, resulting in that the remote driver cannot view the environment around the vehicle in time, which poses a safety risk.

[0005] In addition, after the remote device sends a control signal such as steering and braking to the vehicle end, the vehicle end returns a response message to the remote device. If the video stream occupies a large amount of uplink bandwidth, it will squeeze the normal transmission of the response message between the vehicle and the remote device, increase the delay of the system response, and further cause safety hazards. SUMMARY

[0006] The present disclosure provides a video stream processing method, a video stream processing apparatus, an electronic device, a storage medium, a computer program product, an automatic driving vehicle, and a video stream processing system.

[0007] According to an aspect of the present disclosure, a video stream processing method is provided, including: determining a target leaf node from a decision tree model according to a range in which a value of at least one network performance parameter for a vehicle falls, the target leaf node representing a network quality for the vehicle; determining a target encoding strategy from a plurality of candidate encoding strategies according to the network quality; the target encoding strategy including at least one video stream identifier for indicating at least one target video stream to be encoded from a plurality of original video streams, and a respective code rate of each of the at least one target video stream; and encoding the at least one target video stream indicated by the at least one video stream identifier according to the respective code rate of each of the at least one target video stream, to obtain at least one encoded video stream.

[0008] According to another aspect of the present disclosure, a video stream processing apparatus is provided, including: a node determining module, a strategy determining module, and an encoding module. The node determining module is configured to determine a target leaf node from a decision tree model according to a range in which a value of at least one network performance parameter for a vehicle falls, the target leaf node representing a network quality for the vehicle. The strategy determining module is configured to determine a target encoding strategy from a plurality of candidate encoding strategies according to the network quality; the target encoding strategy including at least one video stream identifier for indicating at least one target video stream to be encoded from a plurality of original video streams, and a respective code rate of each of the at least one target video stream. The encoding module is configured to encode the at least one target video stream indicated by the at least one video stream identifier according to the respective code rate of each of the at least one target video stream, to obtain at least one encoded video stream.

[0009] According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method provided by the present disclosure.

[0010] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the method provided by the present disclosure.

[0011] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, the computer program being executed by a processor to implement the method provided by the present disclosure.

[0012] According to another aspect of the present disclosure, an autonomous vehicle is provided, including the electronic device described above.

[0013] According to another aspect of the present disclosure, there is provided a video stream processing system, comprising the electronic device and a remote device. The remote device is configured to acquire at least one encoded video stream processed by the electronic device, and to present based on the at least one encoded video stream.

[0014] It should be understood that the description of the contents described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0016] Figure 1 is an application scenario diagram of a video stream processing method and device according to an embodiment of the present disclosure;

[0017] Figure 2 is a schematic flowchart of a video stream processing method according to an embodiment of the present disclosure;

[0018] Figure 3 is a structural schematic diagram of a decision tree model according to an embodiment of the present disclosure;

[0019] Figure 4 is a schematic flowchart of determining network quality using a decision tree model according to an embodiment of the present disclosure;

[0020] Figure 5 is a principle schematic diagram of a decision tree model according to an embodiment of the present disclosure;

[0021] Figure 6 is a schematic principle diagram of a video stream processing method according to an embodiment of the present disclosure;

[0022] Figure 7 is a schematic structural block diagram of a video stream processing device according to an embodiment of the present disclosure; and

[0023] Figure 8 is a structural block diagram of an electronic device for implementing a video stream processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0025] The collection, storage, use, processing, transmission, provision and disclosure of the user personal information in the technical solutions of the present disclosure comply with relevant laws and regulations and do not violate public order and good customs.

[0026] In the technical solutions of the present disclosure, the authorization or consent of the user is obtained before the user personal information is acquired or collected.

[0027] Figure 1 is a schematic diagram of an application scenario of the video stream processing method and device according to an embodiment of the present disclosure.

[0028] It should be noted that Figure 1 The system architecture shown is only an example of a system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be applied to other devices, systems, environments or scenarios.

[0029] As Figure 1 shown, the system architecture 100 according to the embodiment can include a vehicle 110, a network 120, a remote device 130, and the like. The network 120 is a medium for providing a communication link between the vehicle 110 and the remote device 130. The network 120 can include various connection types, such as wired and / or wireless communication links, and the like.

[0030] The vehicle 110 can be integrated with sensors such as vehicle-mounted cameras, which can interact with the server 130 through the network 120 to receive or send messages, and the like. The vehicle 110 can collect network performance parameters and determine network quality, and then determine a target encoding strategy based on the network quality. Subsequently, the vehicle 110 encodes a plurality of original video streams based on the target encoding strategy and sends the encoded video streams to the remote device 130.

[0031] The remote device 130 can be a remote cockpit, which can obtain at least one encoded video stream processed by the vehicle 110 and display based on the at least one encoded video stream. For example, the remote device 130 can decode the encoded video stream and the like, and render the plurality of video streams to the display screen of the remote device, to provide the driver with a continuous and reliable environmental perception field of view, so that the driver knows the surrounding environment of the vehicle. The remote device 130 can output control instructions to the vehicle 110 according to the behavior of the driver, so as to control the vehicle 110 to travel.

[0032] It should be noted that the video stream processing method provided by the embodiments of the present disclosure can generally be executed by the vehicle 110. Correspondingly, the video stream processing device provided by the embodiments of the present disclosure can also be arranged in the vehicle 110.

[0033] It should be understood that Figure 1The number of vehicles, networks, and servers shown is merely illustrative. Any number of vehicles, networks, and servers can be included depending on implementation needs.

[0034] Figure 2 This is a schematic flowchart of a video stream processing method according to an embodiment of the present disclosure.

[0035] like Figure 2 As shown, the video stream processing method 200 can be executed by a vehicle, which can be an autonomous vehicle or a vehicle with an autonomous driving mode. The video stream processing method 200 may include operations S210 to S230.

[0036] In operation S210, a target leaf node is determined from the decision tree model based on the range of values ​​of at least one network performance parameter for the vehicle. The target leaf node represents the network quality for the vehicle.

[0037] For example, network performance parameters include Round-Trip Time (RTT) and packet loss rate. RTT is the time elapsed from when the vehicle sends a video stream to when it receives an acknowledgment from the remote device, representing the network status between the vehicle and the remote device. The unit of RTT is milliseconds, and the packet loss rate is a percentage value. For example, network quality is categorized as strong, medium, and weak.

[0038] A decision tree model consists of multiple nodes, each corresponding to a range of network performance parameters. For example, one node might have a network round-trip time (RTT) of less than or equal to 80 milliseconds, while another node might have a RTT of less than or equal to 80 milliseconds and a packet loss rate greater than 30%. The leaf node whose value range includes the network performance parameter can be selected as the target leaf node.

[0039] The target leaf node can represent the confidence level of a strong, medium, or weak network. For example, if the node value of the target leaf node is [0.7, 0.2, 0.1], it means that the confidence levels of the strong, medium, and weak networks are 0.7, 0.2, and 0.1, respectively. The network quality with the highest confidence level among the node values ​​can be determined as the vehicle's network quality. In other examples, the network quality represented by the target leaf node can be configured according to requirements.

[0040] In operation S220, a target coding strategy is determined from multiple candidate coding strategies based on network quality. The target coding strategy includes: at least one video stream identifier indicating at least one target video stream to be encoded from multiple original video streams, and the bitrate of each of the at least one target video streams.

[0041] The camera can be used to collect the environment around the vehicle to obtain the original video stream. The original video stream includes, for example, the video stream of the front, left front, right front, left rear, right rear, rear, and surround view of the vehicle.

[0042] A plurality of candidate encoding strategies can be pre-configured. The candidate encoding strategy includes video stream identifiers id1 and id2, indicating that the two original video streams of the video stream identifiers id1 and id2 are to be encoded. In addition, the candidate encoding strategy also includes the code rate of each of the two original video streams indicated by the video stream identifiers id1 and id2, for example, the code rate of the original video stream indicated by the video stream identifier id1 is 1500kbps, and the code rate of the original video stream indicated by the video stream identifier id2 is 300kbps. It should be noted that in the candidate encoding strategy, the video stream identifier can be determined based on the importance of each original video stream. The code rate can be the encoding code rate used for encoding and the transmission code rate used for transmission, and transmission refers to the vehicle sending the encoded video stream to the remote device. The encoding code rate and the transmission code rate have the same value.

[0043] In operation S230, at least one target video stream indicated by at least one video stream identifier is encoded according to the code rate of each of the at least one target video stream to obtain at least one encoded video stream.

[0044] For example, the original video stream indicated by the video stream identifier id1 is encoded at 1500kbps, and the original video stream indicated by the video stream identifier id2 is encoded at 300kbps, so as to obtain two encoded video streams. Then the vehicle can send the two encoded video streams to the remote device, so that the remote device can display the two encoded video streams based on the two encoded video streams, so that the remote driver can know the environment around the vehicle.

[0045] According to the technical scheme provided by the embodiment of the present disclosure, on the one hand, the network quality is determined by using a lightweight decision tree model. The decision tree model has a simple structure and small calculation overhead, can quickly evaluate the network quality by using the real-time collected network performance parameters, and avoid the high delay problem caused by the algorithm based on a large buffer, thereby meeting the basic requirement of real-time performance of remote driving. On the other hand, the method can dynamically adjust the encoding strategy and the transmission strategy according to the network quality of the vehicle. For example, when the network quality is poor, the original video stream of a small number of paths is encoded and transmitted by using a low code rate. When the network quality is high, the original video stream of a large number of paths is encoded and transmitted by using a high code rate. Because the code rate in the encoding process and the number of original video streams to be encoded are flexibly adjusted according to the network quality of the vehicle, the redundant bandwidth can be fully utilized to improve the video quality when the network quality is high. When the network quality is poor, not only the code rate of the single original video stream is reduced, but also the number of paths of the original video stream that is encoded and transmitted is reduced, and the smooth playing of the video stream of an important perspective is preferentially ensured, so that the smooth video display of the remote driving cabin is ensured, and the driving safety is improved. Moreover, the normal transmission of the response message of the encoded video stream sent by the vehicle end to the remote device can be avoided, and the safety is further improved.

[0046] The embodiment of the present disclosure is suitable for the remote driving scene of the autonomous vehicle, and provides remote safety monitoring, fault processing and remote assistance service for the autonomous vehicle.

[0047] Figure 3 FIG. 1 is a structural schematic diagram of the decision tree model according to the embodiment of the present disclosure.

[0048] In the embodiment, the decision tree model includes a plurality of nodes. The node without a child node is referred to as a leaf node. The node with a child node is referred to as a decision node. For example, the decision node includes a root node and an intermediate node, and the intermediate node refers to the node between the root node and the leaf node. As shown in FIG. 1, the node Node_31 is the root node, the nodes Node_34, Node_35, Node_38 and Node_39 are the leaf nodes, and the nodes Node_32, Node_33, Node_36 and Node_37 are the intermediate nodes. Figure 3

[0049] Each node has parameters. For example, the parameters of the node include a child node index, a feature index, a threshold value and a node value. Figure 3 Only the feature index and the threshold value are shown, and all the parameters are not shown.

[0050] ​The child node indices can include left child node indices children_left and right child node indices children_right, which together define the topology of the decision tree model. For the i-th node, children_left[i] stores the index of the left child node to visit next when the condition is satisfied. Similarly, children_right[i] stores the index of the right child node to visit next when the condition is not satisfied. If the left child node index children_left and the right child node index children_right are -1 or other predetermined characters, they can also be null, indicating that the node is a leaf node. For example, the left child node index children_left of the node Node_31 is the index of the node Node_32, and the right child node index children_right is the index of the node Node_33. For example, the left child node index children_left and the right child node index children_right of the node Node_34 are predetermined characters.

[0051] The feature index feature defines the basis for judging each decision node. feature[i] indicates which network performance parameter in at least one network performance parameter will be used as the target parameter by the i-th node. For example, the value of feature[i] is 0, indicating that the target parameter is the network round-trip delay, and the value of feature[i] is 1, indicating that the target parameter is the packet loss rate. The value of feature[i] of a leaf node can be configured as -2 or other characters or null.

[0052] The threshold threshold defines the judgment threshold of each decision node. The decision rule is: for the i-th node, if the feature[i] of the node is less than or equal to the threshold threshold[i], the left child node is selected as the target child node, otherwise the right child node is selected as the target child node. The value of the threshold threshold of the leaf node can be configured as -3 or other values or null. For example, for the node Node_31, if the network round trip delay is less than or equal to 80 milliseconds, the node Node_31 is selected as the target child node, otherwise the node Node_33 is selected as the target child node. For example, for the node Node_32, if the packet loss rate is less than or equal to 30%, the node Node_34 is selected as the target child node, otherwise the node Node_35 is selected as the target child node. For example, for the node Node_33, if the network round trip delay is less than or equal to 180 milliseconds, the node Node_36 is selected as the target child node, otherwise the node Node_37 is selected as the target child node. For example, for the node Node_37, if the packet loss rate is less than or equal to 60%, the node Node_38 is selected as the target child node, otherwise the node Node_39 is selected as the target child node.

[0053] The node value value defines the output value of each node, and the node value value can be a multi-dimensional array. For example, when the network quality is divided into three levels of strong network, medium network and weak network, the node value value is a three-dimensional array [[x, y, z]], which represents the weight or probability of the tendency to strong network, medium network and weak network obtained by statistics on the corresponding leaf node, that is, the confidence of strong network, medium network and weak network. For the decision node, it is important to determine the next target child node, and the node value value is not important, and the node value value can be regarded as a statistical summary of the training data. For the leaf node, the node value value can be mapped to the network quality, for example, the sizes of x, y and z in the node value value can be compared, and then the network quality with the highest confidence is selected as the network quality of the vehicle.

[0054] It should be noted that the node corresponds to a numerical range of network performance indicators, and the numerical range can represent the condition that the network performance parameter needs to meet when reaching the node. Taking the value of feature[i] as 0 to represent that the target parameter is the network round-trip delay (unit: millisecond), and the value of feature[i] as 1 to represent that the target parameter is the packet loss rate (proportion value) as an example. The numerical range corresponding to the node Node_32 is that the network round-trip delay is less than or equal to 80 milliseconds. The numerical range corresponding to the node Node_33 is that the network round-trip delay is greater than 80 milliseconds. The numerical range corresponding to the node Node_34 is that the network round-trip delay is less than or equal to 80 milliseconds, and the packet loss rate is less than or equal to 30%. The numerical range corresponding to the node Node_35 is that the network round-trip delay is less than or equal to 80 milliseconds, and the packet loss rate is greater than 30%. The numerical range corresponding to the node Node_36 is that the network round-trip delay is greater than 80 milliseconds and less than or equal to 180 milliseconds. The numerical range corresponding to the node Node_37 is that the network round-trip delay is greater than 180 milliseconds. The numerical range corresponding to the node Node_38 is that the network round-trip delay is greater than 180 milliseconds, and the packet loss rate is less than or equal to 60%. The numerical range corresponding to the node Node_39 is that the network round-trip delay is greater than 180 milliseconds, and the packet loss rate is greater than 60%.

[0055] For example, if the network performance parameter of the vehicle is that the network round-trip delay is 50 milliseconds, and the packet loss rate is 40%, the numerical value of the network performance parameter is in the numerical range corresponding to the node Node_35, and therefore the node Node_35 is the target leaf node. The node value value of the node Node_35 can be mapped to the network quality.

[0056] After constructing the decision tree model, in one example, the numerical range of the network performance indicators corresponding to each leaf node of the decision tree model can be recorded, and then the numerical range is directly compared with the numerical value of the obtained network information parameter, so as to determine the target leaf node.

[0057] After constructing the decision tree model, in another example, the decision tree model can be recursively traversed from the root node of the decision tree model. At each decision node, the corresponding target parameter is obtained from at least one network performance parameter according to the stored feature index, and then the target parameter is compared with the threshold value of the node. According to the comparison result, the left child node or the right child node is selected as the next node to be processed, until the leaf node is reached. Then the node value of the leaf node can be read, so as to map the node value to the network quality. Next, in combination with the network information parameter, the network quality can be determined. Figure 4 The present embodiment will be described.

[0058] Figure 4 is a schematic flowchart of determining network quality by using a decision tree model according to an embodiment of the present disclosure.

[0059] like Figure 4 As shown, in this embodiment, the method for determining network quality may include operations S411 to S418.

[0060] In operation S411, the root node is set as the current node.

[0061] In operation S412, determine whether the current node is a leaf node. If not, proceed to operation S413. If yes, it means the current node is the target leaf node, and proceed to operation S418.

[0062] In operation S413, the target parameter is determined from at least one network performance parameter based on the feature index of the current node.

[0063] In operation S414, determine whether the target parameter is less than or equal to the threshold of the current node. If yes, proceed to operation S415; otherwise, proceed to operation S416.

[0064] It should be noted that operation S414 mainly determines the target child node from at least two child nodes of the current node based on the target parameter and the threshold of the current node. In other examples, operation S415 can also be entered when the target parameter is greater than the threshold of the current node, and operation S416 can be entered otherwise.

[0065] In operation S415, the left child node is set as the target child node.

[0066] In operation S416, the right child node is set as the target child node.

[0067] In operation S417, the target child node is updated to the current node, and then the process returns to operation S412.

[0068] In operation S418, the current node is identified as the target leaf node, and the network quality is determined based on the target leaf node. For example, the node value of the target leaf node is mapped to the network quality.

[0069] In this embodiment, the target parameters are first read using feature indexing, and then compared with the threshold of the node to determine the traversal path. This way, a single node does not need to process all network performance parameters, but dynamically selects key target parameters for judgment, reducing computational complexity and improving the efficiency of determining the target leaf node, thus meeting the real-time requirements of cloud-based remote driving. Furthermore, the decision path of this decision tree is more intuitive and clear, allowing developers to intuitively understand under what threshold conditions the target parameters affect the classification results, facilitating their understanding and adjustment of the decision tree.

[0070] Figure 5 This is a schematic diagram illustrating the principle of a decision tree model according to an embodiment of this disclosure.

[0071] like Figure 5As shown, in the embodiment, the decision tree model includes nodes Node_51~Node_59. Taking the network performance parameter of the vehicle as an example, the network round-trip delay is 30 ms, and the packet loss rate is 50%. First, the node Node_51 is taken as the current node. For example, the feature index in the parameter of the node Node_51 indicates the network round-trip delay, and the threshold value is 70 ms. Since the network round-trip delay is less than the threshold value of the node Node_51, the left sub-tree is selected, and the target sub-node is the node Node_52 at this time. For example, the feature index in the parameter of the node Node_52 indicates the packet loss rate, and the threshold value is 40%. Since the node Node_52 is not a leaf node, the comparison between the packet loss rate and the threshold value of the node Node_52 is continued. Since the packet loss rate is greater than the threshold value of the node Node_52, the right sub-tree is selected, and the target sub-node is the node Node_55 at this time. Since the node Node_55 is a leaf node, the node Node_55 is taken as the target leaf node. The node value value of the node Node_55 can be mapped to the network quality.

[0072] It should be noted that the threshold value rule-based method needs to manually set a large number of threshold values and parameters. These parameters need to be repeatedly debugged and optimized for different network environments and different regions. The maintenance and debugging cost is very high, and it is difficult to deploy on a large scale. Therefore, in actual application, the decision tree model can be trained using pre-constructed training data, which can reduce the cost of manual configuration of rules.

[0073] In another embodiment, the difference between the embodiment and the previous embodiment is that the parameter of a node in the previous embodiment only involves one network performance parameter and the threshold value of the network performance parameter. In the embodiment, the parameter of a node can involve multiple network performance parameters and the threshold values of the network performance parameters.

[0074] Correspondingly, the above process of determining the target leaf node from the decision tree model according to at least one network performance parameter of the vehicle can include: any node in the decision tree model can be selected as the current node, for example, the root node is taken as the current node. Next, for the current node in the decision tree model, it is determined whether the current node is a leaf node. If yes, the current node is the target leaf node. If no, the target sub-node is determined from the at least two sub-nodes of the current node according to the at least one network performance parameter and the threshold value of the current node. For example, if the value of the network performance parameter is less than the threshold value, the left sub-node is selected as the target sub-node, otherwise the right sub-node is selected as the target sub-node. Next, the target sub-node is updated as the current node, and the operation of determining whether the current node is a leaf node is returned until the current node is a leaf node.

[0075] In this embodiment, a single node can determine a path based on one or more network performance parameters at the same time. The decision tree structure is more compact and has fewer layers, thereby improving computational efficiency and meeting the real-time requirements of cloud driving.

[0076] According to another embodiment of the present disclosure, the candidate encoding strategies satisfy the following conditions: in the same candidate encoding strategy, the multiple original video streams are divided into multiple video level different original video streams, for example, the video level of the original video stream in the front is the highest, the video level of the original video stream in the left front, right front, left rear, right rear and rear is medium, and the video level of the original video stream in the remaining view angle is the lowest. In the same candidate encoding strategy, for any two video level different original video streams, the code rate of the original video stream with high video level is greater than the code rate of the original video stream with low video level, for example, the code rate of the original video stream in the front is higher than the code rate of the original video stream in the left front. In this embodiment, the original video streams are classified and assigned with differentiated code rates according to the importance of the view angle within the same encoding strategy, so that the video stream with key view angle (such as the front) can have high definition under limited network bandwidth, and accurate visual information can be provided for the remote driver. At the same time, the code rate of the non-key view angle can avoid occupying a large amount of network bandwidth.

[0077] According to another embodiment of the present disclosure, the candidate encoding strategies satisfy the following conditions: the strategy levels of the multiple candidate encoding strategies are different from each other, and in any two strategy level different candidate encoding strategies, the number of video stream identifiers in the candidate encoding strategy with high strategy level is greater than the number of video stream identifiers in the candidate encoding strategy with low strategy level. For example, the strong network corresponds to the candidate encoding strategy Strategy1, the medium network corresponds to the candidate encoding strategy Strategy2, and the strategy level of the candidate encoding strategy Strategy1 is higher than the strategy level of the candidate encoding strategy Strategy2. The candidate encoding strategy Strategy1 includes 10 video stream identifiers, indicating that the vehicle needs to encode and transmit 10 original video streams to the remote device. The candidate encoding strategy Strategy2 includes 9 video stream identifiers, indicating that the vehicle needs to encode and transmit 9 original video streams to the remote device. In this embodiment, the strategy level is associated with the number of video streams to be transmitted, so that the number of video streams to be transmitted can be dynamically adjusted based on the network quality. When the network is strong, more view angle video streams are transmitted to provide comprehensive environmental information for the driver. When the network is weak, the number of transmitted video streams is reduced, and the normal transmission of the video stream with important view angle is preferentially guaranteed, thereby improving the safety of remote driving.

[0078] According to another embodiment of the present disclosure, the candidate encoding strategies satisfy the following conditions: the strategy levels of the plurality of candidate encoding strategies are different from each other, when any two candidate encoding strategies with different strategy levels include video stream identifiers for indicating the same target video stream, the code rate for the same target video stream in the candidate encoding strategy with a higher strategy level is greater than or equal to the code rate for the same target video stream in the candidate encoding strategy with a lower strategy level. For example, the code rate for the original video stream in front in the candidate encoding strategy Strategy1 is 1500kbps, and the code rate for the original video stream in front in the candidate encoding strategy Strategy2 is 750kbps. In this embodiment, the code rates of the same original video stream under different strategy levels are different, so that clearer video streams can be transmitted in strong networks, and smooth transmission of video streams can be ensured by reducing the code rate in weak networks.

[0079] According to another embodiment of the present disclosure, an example of the preconfigured candidate encoding strategies is shown in Table 1.

[0080] Table 1: Example of candidate encoding strategies

[0081]

[0082] It should be noted that the code rate of 0 in the candidate encoding strategy in Table 1 can represent that the remote device does not need to display the original video stream based on the road. For example, for a weak network, the remote device does not need to display the video streams of the four perspectives of the left rear, the right rear, the low magnification, and the traffic light magnification. The four original video streams are non-key video streams, and the remaining six original video streams are key video streams.

[0083] It can be understood that, since the remote device can only display key video streams and does not need to display non-key video streams, for a cloud driving scenario, the non-key video streams can not be encoded and sent. In other examples, if other downstream modules need to use non-key video streams, the non-key video streams can still be encoded by using other instructions. In the encoding process, the code rate of the last used candidate encoding strategy can be continued to be used for encoding, so as to avoid switching the code rate. Other preconfigured code rates can also be used for encoding, which can meet the use requirements of the downstream modules. This embodiment does not limit whether the non-key video streams are encoded and the code rate used for encoding the non-key video streams.

[0084] As can be seen from Table 1, the embodiment preconfigures a mapping relationship between network quality-code rate and video processing number. Based on the mapping relationship, the vehicle can react to network changes with low delay, shorten the time length from identifying network changes to adjusting the encoding strategy, ensure the accuracy of the encoding strategy, and reduce the system complexity, which is suitable for stable operation on a vehicle-mounted device.

[0085] According to Table 1, the network quality is divided into strong network, medium network and weak network in the embodiment.

[0086] The strong network means that the network quality is excellent, and the characteristics are low delay, low packet loss rate and high available bandwidth. The strong network can support stable transmission of multiple video streams, and meet the transmission requirements of high code rate and low delay for remote driving. Therefore, when the network quality is strong network, high code rate is used to encode and transmit 10 original video streams, to provide comprehensive and clear environmental perception information for the remote driver.

[0087] The medium network means that the network quality is general, and there is a certain delay or slight packet loss. The video transmission can maintain basic smoothness under the medium network. Therefore, when the network quality is medium network, medium code rate is used to encode and transmit 9 original video streams, to provide as much visual information as possible under the premise of ensuring the smoothness of the core driving field of view.

[0088] The weak network means that the network quality is poor, and there are problems such as high delay, high packet loss or serious bandwidth limitation. Under the weak network, there is a high risk of freezing and black screen in video transmission. Therefore, when the network quality is weak network, low code rate is used to encode and transmit 6 original video streams, to concentrate the limited bandwidth resources on guaranteeing the continuity and low delay of the core field of view in poor network quality through the way of reducing code rate and reducing the number of video streams, and to avoid local black screen and freezing caused by bandwidth overload.

[0089] In actual application, a vehicle can deploy multiple camera devices to collect multiple original video streams. In this way, after encoding, multiple encoded video streams corresponding to different perspectives can be obtained. Next, the video stream of each perspective is described, which can be an original video stream, an encoded video stream or a video stream obtained after decoding by a remote device.

[0090] For example, the video stream in the front direction as the main view channel provides a wide-angle view of the front of the vehicle, covering the main driving path, and is the main basis for the remote driver to make main driving decisions (such as lane keeping and following). The video stream in the right front direction covers the right front blind area of the vehicle, and is used to monitor the right front cutting vehicle, pedestrians and obstacles, which is important when turning right or changing lanes to the right. The video stream in the left front direction covers the left front blind area of the vehicle, and has similar functions to the video stream in the right front direction, mainly used in left turn, left lane change and other scenes. The video stream in the right rear direction covers the right rear blind area of the vehicle, and assists in monitoring the right rear vehicle to ensure the safety of the right lane change. The video stream in the left rear direction covers the left rear blind area of the vehicle, and is used for left rear road condition monitoring and lane change assistance. The video stream in the rear direction provides a direct view of the rear of the vehicle, mainly used for reversing and monitoring the rear following vehicle. The 360-degree fisheye surround view video stream can include a top panoramic view spliced from four fisheye cameras, providing a bird's eye view of the vehicle's surrounding environment for the remote driver, suitable for low-speed parking, narrow space passing and other scenes. The video stream of the rear fisheye can provide an original rear ultra-wide-angle view without splicing and with large distortion, which can provide a wider view than the rear video when reversing, reducing the blind area. The low magnification video stream is obtained by regionally cropping and digitally magnifying the original video, and is used to focus on low obstacles, potholes, curb stones and other low obstacles near the ground of the vehicle, making it easier for the driver to detect such obstacles. The red light magnification video stream locks the front traffic signal area through image recognition and digitally magnifies it, clearly and unobstructedly delivering key traffic instruction information to the remote driver, ensuring that the driver can accurately and timely respond to traffic signals.

[0091] According to another embodiment of the present disclosure, in response to receiving the video acquisition request, the plurality of original video streams are encoded according to a candidate encoding strategy with the highest strategy level among the plurality of candidate encoding strategies. Next, in response to detecting that the duration of receiving the video acquisition request is greater than or equal to a predetermined duration, an operation of determining a target leaf node from the decision tree model is triggered.

[0092] For example, the vehicle can send some blocking events to the remote device to indicate that the vehicle currently encounters problems during driving. The remote device is deployed with a scheduling algorithm, which determines whether to take over the vehicle by a remote driver based on the received information, and if so, the remote device sends a video acquisition request to the vehicle. After receiving the video acquisition request, the vehicle first encodes each raw video stream at a high code rate according to the candidate strategy corresponding to the strong network, and sends the encoded video stream to the remote device. In addition, a predetermined time length is also pre-configured, which can be 5 seconds or other time lengths, and the embodiment is not limited. Within the predetermined time length, each raw video stream is encoded at a high code rate and transmitted according to the candidate strategy corresponding to the strong network. After the predetermined time length, the network performance parameters are collected and input into the decision tree model to evaluate the network quality, and the encoding strategy is dynamically adjusted based on the network quality. In this way, it can ensure that the remote driver can obtain a comprehensive and clear environmental view at the initial stage of session establishment, thereby ensuring the safety of operation.

[0093] According to another embodiment of the present disclosure, in the process of determining the target encoding strategy from the plurality of candidate encoding strategies, it can be determined whether the network quality meets a predetermined stable condition according to the network quality and at least one historical network quality in the past predetermined time length. If so, the candidate encoding strategy corresponding to the network quality in the plurality of candidate encoding strategies is determined as the target encoding strategy. If not, the last executed candidate encoding strategy is determined as the target encoding strategy.

[0094] The predetermined stable condition is used to determine whether the network quality is stable. For example, the predetermined stable condition includes that the network quality is the same as at least one historical network quality, and the network quality is different from a reference network quality, which is the network quality corresponding to the last executed candidate encoding strategy.

[0095] For example, in the past period of time, the network quality is strong network, so the candidate encoding strategy corresponding to the strong network is used for encoding in the past, which is called the last executed candidate encoding strategy, and correspondingly, the strong network is the reference network quality. At the same time, the network performance parameters are collected in real time and the network quality is evaluated, taking the latest evaluated network quality as the medium network for example. After obtaining the latest network quality as the medium network, the historical network states evaluated in the past period of time are read to determine whether the network of the vehicle is continuously stable in the medium network. If the duration of the medium network is less than the predetermined time length, it indicates that the network fluctuates, and the encoding strategy is not adjusted, and the candidate encoding strategy corresponding to the strong network is still used for encoding. If the duration of the medium network is greater than or equal to the predetermined time length, the candidate encoding strategy corresponding to the medium network is used for encoding.

[0096] The embodiment adopts a smoothing mechanism, that is, the encoding strategy is adjusted only when the duration of network quality change reaches a predetermined duration. The smoothing mechanism can avoid frequent switching of the encoding strategy caused by network fluctuations or transient noise, thereby ensuring the stability of the video stream and avoiding interference to the remote driver caused by frequent fluctuations in picture quality.

[0097] According to another embodiment of the present disclosure, a historical encoded video stream for presenting a predetermined perspective of the vehicle can be used as a reference video stream, and a network performance parameter of the reference video stream is determined as at least one network performance parameter of the vehicle. Since the video stream in the front direction provides a wide-angle view of the front of the vehicle and covers the main driving path, it is the main basis for the remote driver to make main driving decisions. Therefore, the predetermined perspective can include the front direction. In other examples, the predetermined direction can also include a left front perspective, etc. Next, the network performance parameters of the original video stream of the predetermined perspective can be continuously collected at a predetermined frequency.

[0098] According to another embodiment of the present disclosure, the network performance parameter of the reference video stream can include at least one of the network round-trip delay and the packet loss rate of the reference video stream. In this embodiment, the network round-trip delay is accurately calculated by analyzing the complete transmission loop of the video data packet from the vehicle end to the reception confirmation of the remote device, and the packet loss rate is obtained by counting the loss of the data packet. The network round-trip delay is a core indicator for measuring network delay and directly affects the real-time performance of data transmission. The packet loss rate is a key indicator for measuring network stability and reliability. Therefore, by using the network round-trip delay and the packet loss rate as the network performance parameter, the embodiment can more accurately represent the level of network performance.

[0099] Figure 6 is a schematic diagram of a video stream processing method according to an embodiment of the present disclosure.

[0100] As Figure 6 shown, in this embodiment, network performance parameters 601 are first collected, for example, for the video stream transmission channel in the front direction between the vehicle and the remote device, the network round-trip delay and the packet loss rate of the video stream transmission channel are collected.

[0101] Next, the network performance parameters 601 can be processed by the decision tree model to obtain the network quality. If the network quality is strong network 602, a first candidate encoding strategy 605 can be used as the target encoding strategy, for example, high rate encoding and transmission of 10 original video streams. If the network quality is medium network 603, a second candidate encoding strategy 606 can be used as the target encoding strategy, for example, medium rate encoding and transmission of 9 original video streams. If the network quality is weak network 604, a third candidate encoding strategy 607 can be used as the target encoding strategy, for example, low rate encoding and transmission of 6 original video streams. Next, the encoding module can be called to encode each original video stream, and then sent to the remote device.

[0102] Next, the next detection period is waited for, and if the next detection period is reached, the network performance parameters 601 are re-acquired to re-determine the target encoding strategy.

[0103] Figure 7 is a schematic structural block diagram of a video stream processing apparatus according to an embodiment of the present disclosure.

[0104] As shown in Figure 7 , the video stream processing apparatus 700 can include a node determination module 710, a strategy determination module 720, and an encoding module 730.

[0105] The node determination module 710 is configured to determine a target leaf node from a decision tree model according to a range in which a value of at least one network performance parameter of the vehicle is located, the target leaf node indicating a network quality of the vehicle.

[0106] The strategy determination module 720 is configured to determine a target encoding strategy from a plurality of candidate encoding strategies according to the network quality, the target encoding strategy including at least one video stream identifier indicating at least one target video stream to be encoded from a plurality of original video streams, and a respective code rate of each of the at least one target video stream.

[0107] The encoding module 730 is configured to encode at least one target video stream indicated by the at least one video stream identifier according to the respective code rate of each of the at least one target video stream to obtain at least one encoded video stream.

[0108] According to another embodiment of the present disclosure, the node determining module comprises a judging submodule, a child node determining submodule and a processing submodule. The judging submodule is configured to determine whether the current node is a leaf node for the current node in the decision tree model. The child node determining submodule is configured to determine the target child node from the at least two child nodes of the current node according to the at least one network performance parameter and the threshold of the current node in response to determining that the current node is not a leaf node. The processing submodule is configured to update the target child node as the current node and return to the operation of determining whether the current node is a leaf node until the current node is a leaf node.

[0109] According to another embodiment of the present disclosure, the child node determining submodule comprises a parameter determining unit and a child node determining unit. The parameter determining unit is configured to determine the target parameter from the at least one network performance parameter according to the feature index of the current node. The child node determining unit is configured to determine the target child node from the at least two child nodes of the current node according to the target parameter and the threshold of the current node.

[0110] According to another embodiment of the present disclosure, the candidate encoding strategies satisfy at least one of the following conditions: in the same candidate encoding strategy, the code rate of the original video stream with a high video level is greater than the code rate of the original video stream with a low video level for any two original video streams with different video levels; the strategy levels of the plurality of candidate encoding strategies are different from each other, and the number of video stream identifiers in the candidate encoding strategy with a high strategy level is greater than the number of video stream identifiers in the candidate encoding strategy with a low strategy level for any two candidate encoding strategies with different strategy levels; and the strategy levels of the plurality of candidate encoding strategies are different from each other, and the code rate of the same target video stream in the candidate encoding strategy with a high strategy level is greater than or equal to the code rate of the same target video stream in the candidate encoding strategy with a low strategy level when any two candidate encoding strategies with different strategy levels include the video stream identifier for indicating the same target video stream.

[0111] According to another embodiment of the present disclosure, the above method further comprises an initial encoding module and a triggering module. The initial encoding module is configured to encode the plurality of original video streams according to the candidate encoding strategy with the highest strategy level in the plurality of candidate encoding strategies in response to receiving the video acquisition request. The triggering module is configured to trigger the decision tree model operation of determining the target leaf node from the decision tree model in response to detecting that the time length of receiving the video acquisition request is greater than or equal to the predetermined time length.

[0112] According to another embodiment of this disclosure, the strategy determination module includes: a judgment submodule, a first strategy determination submodule, and a second strategy determination submodule. The judgment submodule is used to determine whether the network quality meets a predetermined stability condition based on the network quality and at least one historical network quality within a predetermined time period. The first strategy determination submodule is used, in response to determining that the network quality meets the predetermined stability condition, to determine the candidate coding strategy corresponding to the network quality from a plurality of candidate coding strategies as the target coding strategy. The second strategy determination submodule is used, in response to determining that the network quality does not meet the predetermined stability condition, to determine the previously executed candidate coding strategy as the target coding strategy.

[0113] According to another embodiment of this disclosure, the predetermined stability conditions include: the network quality is the same as at least one historical network quality, and the network quality is different from the reference network quality, where the reference network quality is the network quality corresponding to the previously executed candidate coding strategy.

[0114] According to another embodiment of this disclosure, the method further includes: a network performance parameter determination module, used to determine the network round-trip time and packet loss rate of the reference video stream as at least one network performance parameter, wherein the reference video stream is a historical encoded video stream used to present the front of the vehicle.

[0115] According to embodiments of this disclosure, this disclosure also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the video stream processing method described above.

[0116] Another embodiment of this disclosure also provides a vehicle, which can be an autonomous vehicle or a manually driven vehicle, and the vehicle may include the aforementioned electronic equipment.

[0117] Another embodiment of this disclosure also provides a video stream processing system, which includes the aforementioned electronic device and a remote device. The remote device may be a remote cockpit. The remote device can acquire at least one encoded video stream processed by the aforementioned electronic device or autonomous vehicle, and display it based on at least one encoded video stream.

[0118] According to embodiments of this disclosure, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the video stream processing method described above.

[0119] According to embodiments of this disclosure, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described video stream processing method.

[0120] Figure 8 is a block diagram of the structure of an electronic device for implementing the video stream processing method according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0121] As shown in FIG. 8, Figure 8 The device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0122] Various components in the device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc., an output unit 807, such as various types of displays, speakers, etc., the storage unit 808, such as a magnetic disk, an optical disk, etc., and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0123] The computing unit 801 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as the video stream processing method. For example, in some embodiments, the video stream processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded onto the RAM 803 and executed by the computing unit 801, one or more steps of the video stream processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the video stream processing method by any other suitable means, such as by means of firmware.

[0124] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0125] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0126] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0128] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0129] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0130] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, which is not limited herein.

[0131] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A video stream processing method, comprising: Based on the range of values ​​for at least one network performance parameter for a vehicle, a target leaf node is determined from the decision tree model, the target leaf node representing the network quality for the vehicle. Based on the network quality, a target coding strategy is determined from multiple candidate coding strategies; The target encoding strategy includes: at least one video stream identifier for indicating at least one target video stream to be encoded in a plurality of original video streams, wherein the bitrate of each of the at least one target video stream is specified. as well as Based on the bitrate of each of the at least one target video streams, the at least one target video stream indicated by the at least one video stream identifier is encoded to obtain at least one encoded video stream.

2. The method according to claim 1, wherein, The step of determining the target leaf node from the decision tree model based on the range of values ​​for at least one network performance parameter for the vehicle includes: For the current node in the decision tree model, determine whether the current node is a leaf node; In response to determining that the current node is not a leaf node, a target child node is determined from at least two child nodes of the current node based on the at least one network performance parameter and a threshold of the current node; and Update the target child node to the current node, and return the operation to determine whether the current node is a leaf node, until the current node is a leaf node.

3. The method according to claim 2, wherein, The step of determining the target child node from at least two child nodes of the current node based on the at least one network performance parameter and the threshold of the current node includes: Based on the feature index of the current node, the target parameter is determined from the at least one network performance parameter; Based on the target parameter and the threshold of the current node, the target child node is determined from at least two child nodes of the current node.

4. The method according to claim 1, wherein, The candidate encoding strategy satisfies at least one of the following conditions: In the same candidate coding strategy, for any two original video streams with different video levels, the bitrate of the original video stream with the higher video level is greater than the bitrate of the original video stream with the lower video level. The multiple candidate coding strategies have different strategy levels. Among any two candidate coding strategies with different strategy levels, the number of video stream identifiers in the candidate coding strategy with a higher strategy level is greater than the number of video stream identifiers in the candidate coding strategy with a lower strategy level. as well as The multiple candidate coding strategies have different strategy levels. When any two candidate coding strategies with different strategy levels include a video stream identifier for indicating the same target video stream, the bitrate of the candidate coding strategy with the higher strategy level for the same target video stream is greater than or equal to the bitrate of the candidate coding strategy with the lower strategy level for the same target video stream.

5. The method according to claim 4, further comprising: In response to receiving a video acquisition request, the multiple original video streams are encoded according to the candidate encoding strategy with the highest strategy level among the multiple candidate encoding strategies; as well as In response to detecting that the duration of the received video acquisition request is greater than or equal to a predetermined duration, the operation of determining the target leaf node from the decision tree model is triggered.

6. The method according to claim 1, wherein, The step of determining the target coding strategy from multiple candidate coding strategies based on the network quality includes: Based on the network quality and at least one historical network quality within a predetermined time period, determine whether the network quality meets the predetermined stability conditions; In response to determining that the network quality meets the predetermined stability condition, the candidate coding strategy corresponding to the network quality among the plurality of candidate coding strategies is determined as the target coding strategy; and In response to determining that the network quality does not meet the predetermined stability condition, the previously executed candidate coding strategy is determined as the target coding strategy.

7. The method according to claim 6, wherein, The predetermined stability conditions include: the network quality is the same as at least one historical network quality, and the network quality is different from the reference network quality, wherein the reference network quality is the network quality corresponding to the candidate coding strategy executed in the last time.

8. The method according to claim 1, further comprising: The network round-trip latency and packet loss rate of the reference video stream are determined as the at least one network performance parameter; wherein the reference video stream is a historical encoded video stream used to present the front of the vehicle.

9. A video stream processing apparatus, comprising: A node determination module is used to determine a target leaf node from a decision tree model based on the range of values ​​of at least one network performance parameter for a vehicle, wherein the target leaf node represents the network quality for the vehicle. A strategy determination module is used to determine a target coding strategy from multiple candidate coding strategies based on the network quality. The target encoding strategy includes: at least one video stream identifier for indicating at least one target video stream to be encoded in a plurality of original video streams, wherein the bitrate of each of the at least one target video stream is specified. as well as An encoding module is configured to encode the at least one target video stream indicated by the at least one video stream identifier according to the bitrate of each of the at least one target video stream, to obtain at least one encoded video stream.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 8.

13. An autonomous vehicle, comprising: The electronic device according to claim 10.

14. A video stream processing system, comprising: The electronic device according to claim 10; as well as A remote device is used to acquire at least one encoded video stream processed by the electronic device and to display it based on the at least one encoded video stream.

Images