Cloud resource processing method, device and equipment

Abstract

The invention relates to a cloud resource processing method, device and equipment. The cloud resource processing method comprises the steps of determining a time feature, a regional feature and a vehicle type feature based on to-be-processed time information, regional information and vehicle type information of cloud resources; processing the time feature, the regional feature and the vehicle type feature through a prediction model, predicting the vehicle message flow to be input to the cloud equipment, and obtaining a prediction result; the prediction result is used for representing the size of the current vehicle message flow to be input into the cloud equipment; and adjusting the configuration of the cloud resources based on the prediction result, so that the configuration of the cloud resources is matched with the prediction result. Wherein the consumer instances in the cloud resources are increased under the condition that the current vehicle message flow to be input into the cloud equipment, which is represented by the prediction result, is greater than a second threshold value; and under the condition that the current vehicle message flow to be input into the cloud equipment represented by the prediction result meets the first preset condition, reducing consumer instances in the cloud resources.

Description

Technical Field

[0001] This invention relates to the field of vehicle network data synchronization technology, specifically to a method, apparatus, and device for processing cloud resources. Background Technology

[0002] With the rapid development of vehicle-to-everything (V2X) technology and the popularization of intelligent vehicles, the number of vehicle messages generated by vehicles has exploded, reaching hundreds of billions every day. These vehicle messages are aggregated to form vehicle message traffic, which is crucial for applications such as vehicle status monitoring, driving behavior analysis, and intelligent traffic management. Summary of the Invention

[0003] This application provides a method, apparatus, and device for processing cloud resources.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, embodiments of this application provide a method for processing cloud resources. The method is applied to a cloud device and includes: determining time characteristics, regional characteristics, and vehicle type characteristics based on time information, regional information, and vehicle type information of the cloud resources to be processed; processing the time characteristics, regional characteristics, and vehicle type characteristics using a prediction model to predict vehicle packet traffic to be input to the cloud device, obtaining a prediction result; the prediction result being used to characterize the current size of vehicle packet traffic to be input to the cloud device; and adjusting the configuration of the cloud resources based on the prediction result to match the configuration of the cloud resources with the prediction result; wherein, adjusting the configuration of the cloud resources based on the prediction result includes: determining time characteristics, regional characteristics, and vehicle type characteristics based on time information, regional information, and vehicle type information of the cloud resources to be processed; processing the time characteristics, regional characteristics, and vehicle type characteristics using a prediction model to predict vehicle packet traffic to be input to the cloud device, obtaining a prediction result; the prediction result being used to characterize the current size of vehicle packet traffic to be input to the cloud device; and adjusting the configuration of the cloud resources based on the prediction result to match the configuration of the cloud resources with the prediction result; and adjusting the configuration of the cloud resources based on the prediction result to characterize the current size of vehicle packet traffic to be input to the cloud device. If the vehicle packet traffic exceeds a second threshold, the number of consumer instances in the cloud resource is increased; if the current vehicle packet traffic to be input to the cloud device, as represented by the prediction result, meets a first preset condition, the number of consumer instances in the cloud resource is decreased; wherein, the first preset condition includes: the current vehicle packet traffic to be input to the cloud device, as represented by the prediction result, is less than a third threshold, or, the current vehicle packet traffic to be input to the cloud device, as represented by the prediction result, is less than the third threshold and exceeds a specified time; the more consumer instances there are, the faster the cloud resource can process the vehicle packet traffic; the fewer consumer instances there are, the slower the cloud resource can process the vehicle packet traffic; wherein, the third threshold is less than or equal to the second threshold.

[0005] It is understood that the solution provided in this application's embodiments has different vehicle packet traffic at different times, different regions, and different vehicle types. By determining the time, region, and vehicle type information to be processed by the cloud resources, the prediction target of the prediction model can be clearly defined, i.e., the prediction model needs to predict the vehicle packet traffic of which vehicle at which time in which region. Based on the time, region, and vehicle type information, time characteristics, region characteristics, and vehicle type characteristics are determined, and the prediction model is used to analyze the time characteristics, region characteristics, and vehicle type characteristics. Vehicle type feature processing predicts the vehicle packet traffic to be input to the cloud device, obtaining prediction results to achieve dynamic prediction of vehicle packet traffic. The prediction results are used to characterize the current size of the vehicle packet traffic to be input to the cloud device. Based on the prediction results, the configuration of cloud resources is adjusted to match the prediction results. On the one hand, this can cope with fluctuations in vehicle packet traffic and prevent the inability to handle surges in vehicle packet traffic. On the other hand, it can achieve cloud resource coordination, enabling cloud resources to process the predicted current vehicle packet traffic in a timely manner, improving synchronization and avoiding waste of cloud resources. Additionally, if the current vehicle packet traffic to be input to the cloud device exceeds the second threshold, increasing the number of consumer instances in the cloud resources can ensure that the vehicle packet traffic can be processed in a timely manner, improve the processing efficiency of vehicle packet traffic, and reduce the backlog of vehicle packet traffic. If the current vehicle packet traffic to be input to the cloud device is less than the third threshold, it indicates that the current vehicle packet traffic is relatively low. By reducing the number of consumer instances in the cloud resources, excess consumer instances can be released, reducing resource waste. Alternatively, considering misjudgment, to reduce the occurrence of misjudgment, a time factor can be incorporated, that is, if the predicted current vehicle packet traffic is less than the third threshold and exceeds a specified time, the number of consumer instances in the cloud resources can be reduced.

[0006] In some embodiments, adjusting the configuration of the cloud resources based on the prediction results includes: adding throughput nodes to the cloud resources when the current vehicle packet traffic to be input to the cloud device, as represented by the prediction results, exceeds a first processing capacity of the cloud resources; and reducing throughput nodes in the cloud resources when the current vehicle packet traffic to be input to the cloud device, as represented by the prediction results, is lower than a second processing capacity of the cloud resources; wherein the first processing capacity is higher than the second processing capacity.

[0007] It is understood that the solution provided in this application embodiment limits the adjustment based on the prediction results to the throughput nodes in the cloud resources. The throughput nodes of the cloud resources represent the cloud resources' processing capacity for the predicted current vehicle packet traffic. When the current vehicle packet traffic to be input to the cloud device exceeds the cloud resources' first processing capacity, increasing the throughput nodes can cope with surges in vehicle packet traffic, ensuring that the vehicle packet traffic can be fully processed, thereby improving the processing efficiency of the cloud resources. When the current vehicle packet traffic to be input to the cloud device is lower than the cloud resources' second processing capacity, reducing the throughput nodes can ensure that the vehicle packet traffic can be fully processed, releasing excess throughput nodes, thereby reducing the waste of cloud resources.

[0008] In some embodiments, adjusting the configuration of the cloud resources based on the prediction result includes: when the prediction result indicates that the first vehicle packet traffic in the current vehicle packet traffic to be input to the cloud device exceeds a first threshold, determining the increase amount of the first vehicle packet traffic; and adding a partition to the first area of ​​the cloud resources corresponding to the processing of the first vehicle packet traffic according to the increase amount of the first vehicle packet traffic, so that the first area after adding the partition can process the first vehicle packet traffic.

[0009] It is understood that the solution provided in this application embodiment limits the adjustment based on the prediction results to a first region of cloud resources. By determining the increase in the first vehicle packet traffic when the first vehicle packet traffic in the current vehicle packet traffic to be input to the cloud device, as represented by the prediction results, exceeds a first threshold, the solution adds a partition to the first region based on the increase in the first vehicle packet traffic. This allows the first region with added partitions to handle the first vehicle packet traffic, thus addressing surges in the first vehicle packet traffic, improving the stability of cloud resources in processing vehicle packet traffic, and reducing latency or backlog.

[0010] In some embodiments, adjusting the configuration of the cloud resources based on the prediction results includes: based on the current vehicle packet traffic to be input to the cloud device represented by the prediction results, and if it is determined that the current vehicle packet traffic belongs to a low period of vehicle packet traffic: based on the priority of the vehicle packet traffic being processed in the cloud resources, determining at least two low-activity regions in the cloud resources; determining a target region and an idle region in the at least two low-activity regions; the target region is used to process the current vehicle packet traffic and the vehicle packet traffic being processed in the two low-activity regions; the idle region serves as a backup region.

[0011] It is understood that the solution provided in this application's embodiments limits the adjustment based on the prediction results to the cloud resource region. Cloud resources have multiple regions, each capable of processing vehicle packet traffic. If it is determined that the current vehicle packet traffic is in a low-intensity period, indicating relatively low traffic, a target region and an idle region can be identified from at least two low-intensity regions. The target region is used to process the current vehicle packet traffic and the vehicle packet traffic being processed in the two low-intensity regions. The idle region serves as a backup region to avoid resource congestion. Essentially, when it is determined that the current vehicle packet traffic is in a low-intensity period, the vehicle packet traffic being processed in at least two low-intensity regions is merged into the target region for continued processing. This frees up the remaining low-intensity regions, which are called idle regions. These idle regions can be used as backups to expand capacity in the event of a subsequent surge in vehicle packet traffic or a peak period, ensuring timely processing of vehicle packet traffic and reducing delays or backlogs.

[0012] In some embodiments, adjusting the configuration of cloud resources based on the prediction results includes: classifying each vehicle packet in the current vehicle packet traffic to be input to the cloud device according to the importance of the vehicle packets' services, as represented by the prediction results, and determining the importance level of each vehicle packet; determining the region for processing high-importance vehicle packets as an independent region in the cloud resources, or determining the throughput node for processing the high-importance vehicle packets as an independent throughput node; the high-importance vehicle packets are vehicle packets including fault alarm information; compressing low-importance vehicle packets before transmission; the low-importance vehicle packets are vehicle packets including regular status information.

[0013] It is understood that the solution provided in this application embodiment limits the allocation of different resources based on the importance of each vehicle packet in the vehicle traffic packets. By determining that the processing area for high-importance vehicle packets is an independent area in cloud resources or that the throughput node for processing high-importance vehicle packets is an independent throughput node, it can be ensured that high-importance vehicle packets are processed in a timely manner, reducing latency; by compressing low-importance vehicle packets before transmission, resource waste can be reduced.

[0014] In some embodiments, after adjusting the configuration of the cloud resources based on the prediction results to match the prediction results, the method further includes: processing the prediction results, which characterize the current vehicle packet traffic to be input to the cloud device, based on the cloud resources to obtain the vehicle packet traffic to be sent; determining target compression algorithms corresponding to vehicle packets of different importance levels based on the importance level of each vehicle packet in the vehicle packet traffic to be sent; transmitting vehicle packets of different importance levels after compression by the corresponding target compression algorithms; wherein, for high-importance vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm is a low-latency compression algorithm; the high-importance vehicle packets are vehicle packets including fault alarm information; for low-importance vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm is a high compression ratio algorithm; the low-importance vehicle packets are vehicle packets including regular status information.

[0015] It is understood that the solution provided in this application embodiment limits the vehicle message traffic to be sent to a method of compression before transmission, which can free up bandwidth resources and improve transmission rate. By using a low-latency compression algorithm for high-priority vehicle messages in the vehicle message traffic to be sent, transmission efficiency and real-time performance can be balanced; by using a high compression ratio algorithm for low-priority vehicle messages in the vehicle message traffic to be sent, the amount of transmission can be reduced.

[0016] In some embodiments, the vehicle packets of different importance levels are compressed and transmitted after being compressed by the corresponding target compression algorithm, including: determining the health scores of at least two candidate transmission paths for the vehicle packet traffic to be sent; selecting the transmission path with the highest health score as the target transmission path from the at least two candidate transmission paths; and transmitting the compressed vehicle packets of different importance levels using the target transmission path; wherein the health score of the transmission path is used to evaluate the network latency, network bandwidth, and network load of the transmission path.

[0017] It is understood that the solution provided in this application embodiment limits the transmission path with the highest health score to be selected as the target transmission path during transmission, which can ensure the stability of transmission, improve the transmission rate, reduce latency, and thus make the synchronization of transmission faster.

[0018] In some embodiments, determining the health scores of at least two candidate transmission paths for the vehicle message traffic to be sent includes: for each of the at least two candidate transmission paths, performing: determining the network latency, network bandwidth, and network load of the candidate transmission path; and determining the health score of the candidate transmission path based on the network latency, network bandwidth, and network load of the candidate transmission path.

[0019] It is understood that the solution provided in this application embodiment defines how to score the health of a transmission path. By determining the network latency of a candidate transmission path, the time efficiency of transmitting data through the candidate network can be known; by determining the network bandwidth of a candidate transmission path, the data transmission capacity of the candidate transmission path per unit time can be known; by determining the network load of a candidate transmission path, the current network resource usage pressure of the candidate transmission path can be known. By determining the health score of a transmission path based on its network latency, network bandwidth, and network load, the real-time performance, throughput, and stability of the candidate transmission path can be evaluated. Thus, a transmission path with good real-time performance, high throughput, and high stability can be selected from the candidate transmission paths as the target transmission path for transmission, which can improve the transmission rate while reducing the occurrence of congestion.

[0020] In some embodiments, the method further includes: in the event of an anomaly in the target transmission path, switching to an alternative transmission path to transmit compressed vehicle messages of different importance levels through the alternative transmission path; the alternative target transmission path is the candidate transmission path with the second highest health score.

[0021] It is understood that the solution provided in this application embodiment can further improve the stability of transmission by switching to an alternative transmission path when the target transmission path is abnormal.

[0022] Secondly, embodiments of this application provide a cloud resource processing apparatus, which is deployed on a cloud device. The apparatus includes: a first determining unit and a first adjusting unit; the first determining unit is configured to determine time characteristics, regional characteristics, and vehicle type characteristics based on time information, regional information, and vehicle type information of the cloud resource to be processed; the first adjusting unit is configured to process the time characteristics, regional characteristics, and vehicle type characteristics through a prediction model to predict the vehicle packet traffic to be input to the cloud device, and obtain a prediction result; adjust the configuration of the cloud resource based on the prediction result so that the configuration of the cloud resource matches the prediction result; the prediction result is used to characterize the current vehicle packet traffic to be input to the cloud device.

[0023] Thirdly, embodiments of this application provide an electronic device, which includes a memory and a processor. The memory stores a computer program or instructions, and when the computer program or instructions are executed by the processor, the method described in the first aspect is implemented.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium, the memory storing a computer program executable on a processor, characterized in that the processor executes the program to implement the method described by the first party.

[0025] Fifthly, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed, implement the method described in the first aspect. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0027] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0028] Figure 1 A flowchart illustrating a cloud resource processing method provided in an embodiment of this application; Figure 2 A flowchart illustrating a training method for a prediction model provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for dynamically adjusting cloud resources based on a predictive model, provided in an embodiment of this application; Figure 4 A schematic diagram of a cloud resource processing device provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of a training device for a prediction model provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0031] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0032] The descriptions such as "first," "second," and "third" appearing in the embodiments of this application do not have a specific meaning (such as no order, nor do they indicate a special limitation on the number of devices in the embodiments of this application), but are merely for the purpose of clearly describing the embodiments of this application and do not constitute any limitation on the embodiments of this application.

[0033] Before providing a more detailed description of the embodiments of this application, the nouns and terms that may be involved in the embodiments of this application will be explained. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0034] Distributed event streaming platforms (DLPs) are primarily responsible for the real-time processing, storage, and distribution of large-scale event data, providing efficient and reliable data pipelines for modern distributed systems. Commonly used DLP platforms include Kafka.

[0035] Kafka: It solves the data flow and coordination problems in modern distributed systems through decoupling, asynchronous communication, and real-time data stream processing.

[0036] Cloud resources: Cloud resources include public clouds and / or private clouds. Public clouds are generally deployed on the cloud; private clouds can be deployed on the cloud or on-premises. The following explanation will use on-premises deployment as an example.

[0037] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies or terms of the embodiments of this application are described below. The following relevant technologies or terms are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0038] With the rapid development of vehicle-to-everything (V2X) technology and the popularization of intelligent vehicles, the number of vehicle messages generated by vehicles has exploded, reaching hundreds of billions every day. These vehicle messages are aggregated to form vehicle message traffic, which is crucial for applications such as vehicle status monitoring, driving behavior analysis, and intelligent traffic management.

[0039] Based on this, the inventors of this application discovered through research and analysis that the relevant solutions have the problem of not being able to dynamically adjust the configuration of cloud resources based on the predicted vehicle message traffic.

[0040] Based on this, embodiments of this application provide the following cloud resource processing methods, apparatus, and devices.

[0041] Examples of cloud resource processing methods: It should be noted that the cloud resource processing method provided in this application embodiment is applied to cloud devices.

[0042] Figure 1 A flowchart illustrating a cloud resource processing method provided in this application embodiment is shown below. Figure 1 As shown, the processing method for this cloud resource includes the following steps: S101. Determine time characteristics, regional characteristics, and vehicle type characteristics based on the time information, regional information, and vehicle type information to be processed from cloud resources. S102. By processing time features, geographical features, and vehicle type features through a prediction model, the vehicle message traffic to be input to the cloud device is predicted, and the prediction result is obtained. The prediction result is used to characterize the current size of the vehicle message traffic to be input to the cloud device. S103. Adjust the configuration of cloud resources based on the prediction results so that the configuration of cloud resources matches the prediction results.

[0043] It is understandable that vehicle packet traffic varies at different times, in different regions, and for different vehicle types. By determining the time, region, and vehicle type information to be processed by cloud resources, the prediction target of the prediction model can be clarified. That is, the prediction model needs to predict the vehicle packet traffic of which vehicle in which region at what time. Based on the time, region, and vehicle type information, time characteristics, region characteristics, and vehicle type characteristics are determined. The prediction model processes these characteristics to predict the vehicle packet traffic to be input to the cloud device, obtaining the prediction result and achieving dynamic prediction of vehicle packet traffic. The prediction result is used to characterize the current size of the vehicle packet traffic to be input to the cloud device. Based on the prediction result, the configuration of cloud resources is adjusted to match the prediction result. On the one hand, this can cope with fluctuations in vehicle packet traffic and prevent the inability to handle surges in vehicle packet traffic. On the other hand, it can achieve cloud resource coordination, enabling the cloud resources to process the predicted current vehicle packet traffic in a timely manner, improving synchronization and avoiding waste of cloud resources.

[0044] The following sections will describe further optional implementation methods for each of the above steps, as well as related terms.

[0045] S101. Determine time characteristics, regional characteristics, and vehicle type characteristics based on the time information, regional information, and vehicle type information to be processed from cloud resources.

[0046] For example, cloud resources include on-cloud Kafka clusters and / or on-premises Kafka clusters. The following description will use the example of cloud resources including on-cloud Kafka clusters and on-premises Kafka clusters.

[0047] It is understandable that vehicle message traffic varies at different times, in different regions, and for different vehicle types. Therefore, by determining the time, region, and vehicle type information to be processed by cloud resources, the prediction target of the prediction model can be clarified. That is, the prediction model needs to predict the vehicle message traffic of which region, at what time, and from which vehicle.

[0048] In one optional embodiment, processing based on time information, geographical information, and vehicle type information may include a prediction model determining time features from time information, geographical features from geographical information, and vehicle type features from vehicle type information.

[0049] For example, time features may include, but are not limited to, timestamps, hours, days of the week, and whether it is a holiday; wherein, timestamps are used to reflect the generation time of vehicle message data.

[0050] For example, regional characteristics may include, but are not limited to, region code, region type, and the proportion of vehicle message traffic in a region; wherein, the proportion of vehicle message traffic in a region refers to the ratio between the vehicle message traffic of a certain region in a certain time period and the vehicle message traffic of the whole country in that time period.

[0051] For example, vehicle type characteristics may include, but are not limited to: vehicle type number, vehicle status, and frequency of generation of a single vehicle message data; wherein, vehicle type may include, but are not limited to, passenger car, commercial vehicle, and new energy vehicle, etc.; wherein, vehicle status may include, but are not limited to, driving, idling, and off, etc.

[0052] Since the prediction model is trained using time features, geographic features, and vehicle type features, after determining the time information, geographic information, and vehicle type information, the prediction model still needs to perform feature extraction on the time information, geographic information, and vehicle type information in order to predict the vehicle message traffic to be input to the cloud device.

[0053] S102. By processing time features, geographical features, and vehicle type features through a prediction model, the vehicle message traffic to be input to the cloud device is predicted, and the prediction result is obtained.

[0054] The prediction results are used to characterize the current vehicle message traffic volume to be input to the cloud device.

[0055] In one alternative embodiment, the processing of time features, geographical features, and vehicle type features by the prediction model can be some internal processing of the prediction model, that is, the necessary processing done to obtain the prediction result, which is related to the structure of the prediction model, such as convolution processing, etc.

[0056] S103. Adjust the configuration of cloud resources based on the prediction results so that the configuration of cloud resources matches the prediction results.

[0057] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction results to match the prediction results can be achieved by expanding the cloud resources when there are many prediction results and shrinking the cloud resources when there are few prediction results.

[0058] The configuration of cloud resources can include throughput nodes, number of partitions, and number of consumer instances, etc., and this application does not impose any special limitations on this. In other words, the throughput nodes, number of partitions, and number of consumer instances of cloud resources can be dynamically adjusted based on the prediction results to cope with the fluctuations in vehicle packet traffic and avoid resource waste.

[0059] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction results may include: adding a throughput node (Broker) to the cloud resources if the current vehicle message traffic to be input to the cloud device, as represented by the prediction results, exceeds the first processing capacity of the cloud resources; and reducing a throughput node in the cloud resources if the current vehicle message traffic to be input to the cloud device, as represented by the prediction results, is lower than the second processing capacity of the cloud resources.

[0060] The first processing capacity is higher than the second processing capacity.

[0061] It is understandable that the throughput nodes in the cloud resources process the vehicle packet traffic. The vehicle packet traffic includes several vehicle packet data (i.e., several vehicle packets). The amount of vehicle packet data that a throughput node can process is limited. Therefore, the number of throughput nodes can be dynamically adjusted according to the predicted current vehicle packet traffic so that the predicted current vehicle packet traffic can be processed in a timely manner, without backlog, and also to reduce resource waste.

[0062] For example, the first processing capacity may refer to the throughput node's ability to process 80% of the preset vehicle message data volume; the second processing capacity may refer to the throughput node's ability to process 30% of the preset vehicle message data volume. It is understood that 80% and 30% are examples of the first and second processing capacities, respectively, and are not intended to limit the first and second processing capacities. Both the first and second processing capacities can be adjusted according to actual conditions, as long as the first processing capacity is higher than the second processing capacity.

[0063] Furthermore, in an alternative embodiment, the number of Brokers can be dynamically adjusted by calling the cloud platform API.

[0064] For example, a blue-green deployment strategy can be adopted to ensure uninterrupted service during the scaling process.

[0065] Understandably, the throughput nodes of cloud resources represent the cloud resources' processing capacity for the predicted current vehicle packet traffic. When the current vehicle packet traffic to be input to the cloud device exceeds the cloud resources' primary processing capacity, increasing the throughput nodes can cope with the surge in vehicle packet traffic, ensuring that the vehicle packet traffic can be fully processed, thereby improving the processing efficiency of the cloud resources. When the current vehicle packet traffic to be input to the cloud device is lower than the cloud resources' secondary processing capacity, reducing the throughput nodes can ensure that the vehicle packet traffic can be fully processed, freeing up excess throughput nodes, thereby reducing the waste of cloud resources.

[0066] Understandably, since these are cloud resources, which include both cloud-based and on-premises Kafka clusters, to ensure synchronization, the throughput nodes of the cloud-based Kafka cluster increase. Correspondingly, the throughput nodes of the on-premises Kafka cluster should also increase to ensure that vehicle message traffic processed on the cloud can be fully received on-premises in a timely manner, reducing the occurrence of missed connections. Conversely, if the throughput nodes of the cloud-based Kafka cluster decrease, the throughput nodes of the on-premises Kafka cluster should also decrease to ensure that neither cloud nor on-premises resources are wasted.

[0067] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction result includes: determining the increase amount of the first vehicle message traffic when the prediction result indicates that the first vehicle message traffic in the current vehicle message traffic to be input to the cloud device exceeds a first threshold; and adding a partition to the first area of ​​the cloud resources corresponding to the processing of the first vehicle message traffic according to the increase amount of the first vehicle message traffic, so that the first area after adding the partition can process the first vehicle message traffic.

[0068] The first region is any region in the cloud resources.

[0069] Further, as an optional embodiment, the increase in the first vehicle packet traffic can be the difference obtained by subtracting the first threshold from the first vehicle packet traffic; as another optional embodiment, the increase in the first vehicle packet traffic is the ratio obtained by dividing the difference obtained by subtracting the first threshold from the first vehicle packet traffic by the first vehicle packet traffic. This application does not impose any particular limitation on this, and the following explanation will exemplify the use of the ratio obtained by dividing the difference obtained by subtracting the first threshold from the first vehicle packet traffic by the first vehicle packet traffic.

[0070] Furthermore, as an optional embodiment, adding partitions to the first area based on the increase in the first vehicle packet traffic can be done by proportionally allocating new partitions according to the increase in the first vehicle packet traffic. For example, if the predicted increase in the first vehicle packet traffic is 20%, then 20% more partitions will be added to the first area. It is understood that 20% is merely an example of the increase in the first vehicle packet traffic, not a limitation thereof. In practical applications, the increase in the first vehicle packet traffic is calculated based on the actual situation.

[0071] For example, adding a new partition can be achieved by calling the Kafka Admin Application Programming Interface (API).

[0072] By managing Kafka programmatically rather than manually, an automated cloud resource coordination mechanism can be achieved.

[0073] It is understandable that since vehicle message data is constantly being generated, if the consumer in the cloud resources cannot consume (i.e. process) the generated vehicle message data in a timely manner, information backlog will occur. Therefore, as another optional embodiment, if the message backlog in the first area exceeds the first preset backlog, it can be determined that the first vehicle message traffic exceeds the first threshold.

[0074] Understandably, by determining the increase in the first vehicle message traffic when the first vehicle message traffic in the current vehicle message traffic to be input to the cloud device, as represented by the prediction results, exceeds a first threshold, the first region is partitioned according to the increase in the first vehicle message traffic. This allows the first region with the added partitions to handle the first vehicle message traffic, thus addressing the surge in the first vehicle message traffic, improving the stability of cloud resources for vehicle message traffic processing, and reducing latency or backlog.

[0075] Understandably, since these are cloud resources, including both on-premises and cloud-based Kafka clusters, to ensure synchronization, the first region of the on-premises Kafka cluster has been partitioned. Correspondingly, the first region of the on-premises Kafka cluster should also have the same number of partitions to ensure that vehicle message traffic processed on the cloud is promptly and completely received by the on-premises cluster. The first region of the on-premises Kafka cluster refers to the region corresponding to the first region of the on-premises Kafka cluster, used to receive the output from the first region of the on-premises Kafka cluster.

[0076] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction results includes: based on the current vehicle packet traffic to be input to the cloud device as characterized by the prediction results, and if it is determined that the current vehicle packet traffic belongs to a low period of vehicle packet traffic: based on the priority of the vehicle packet traffic being processed in the cloud resources, identifying at least two low-activity areas in the cloud resources; identifying a target area and an idle area in the at least two low-activity areas; the target area is used to process the current vehicle packet traffic and the vehicle packet traffic being processed in the two low-activity areas; the idle area serves as a backup area.

[0077] The low-activity area is used to process low-priority vehicle packet traffic. This means that low-priority vehicle packet traffic consists of various low-priority vehicle packets.

[0078] The priority of vehicle messages is related to their importance: the higher the importance of a vehicle message, the higher its priority; the lower the importance of a vehicle message, the lower its priority.

[0079] As an optional implementation, vehicle messages can be divided into high-priority vehicle messages and low-priority vehicle messages according to the importance of the service. High-priority vehicle messages may include fault alarm information, while low-priority vehicle messages may include general information.

[0080] The idle area serves as a backup area, which can handle vehicle packet traffic exceeding the processing capacity of the target area when it exceeds the target area's processing capacity, or it can be merged with the first area to expand capacity when it is necessary to add partitions to the first area.

[0081] It's understandable that cloud resources have multiple regions, each capable of processing vehicle packet traffic. If the current vehicle packet traffic is determined to be in a low-traffic period, it means there's relatively little traffic. By identifying a target region and an idle region from at least two low-activity regions, the target region handles the current vehicle packet traffic along with the traffic being processed in the two low-activity regions. The idle region serves as a backup, preventing resource congestion. Essentially, when the current vehicle packet traffic is determined to be in a low-traffic period, the traffic being processed in at least two low-activity regions is merged into the target region for further processing. This frees up the remaining low-activity regions, known as idle regions, which can be used as backups. This allows for capacity expansion in case of subsequent surges or peak traffic periods, ensuring timely processing of vehicle packets and reducing latency or backlog.

[0082] Understandably, since these are cloud resources, which include both cloud-based and on-premises Kafka clusters, the low-activity regions in the cloud-based Kafka clusters have been merged. Correspondingly, the low-activity regions in the on-premises Kafka clusters also need to be merged in order to save resources on both the cloud and on-premises.

[0083] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction results includes: increasing the number of consumer instances in the cloud resources when the current vehicle message traffic to be input to the cloud device, as represented by the prediction results, is greater than a second threshold; and decreasing the number of consumer instances in the cloud resources when the current vehicle message traffic to be input to the cloud device, as represented by the prediction results, meets a first preset condition.

[0084] Understandably, the more consumer instances there are, the faster the cloud resources can process vehicle packet traffic; conversely, the fewer consumer instances there are, the slower the cloud resources can process vehicle packet traffic.

[0085] As an optional embodiment, the first preset condition may include: the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is less than a third threshold.

[0086] The third threshold is less than or equal to the second threshold.

[0087] To reduce the occurrence of misjudgments, as another optional embodiment, the first preset condition may include: the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is less than a third threshold and exceeds a specified duration.

[0088] For example, the specified duration could be 10 minutes. It is understood that 10 minutes is just an example of the specified duration and does not mean that the specified duration is limited to 10 minutes. In actual application, the specified duration can be set according to actual needs.

[0089] Furthermore, as an optional implementation, the current vehicle message traffic to be input to cloud devices can be monitored through the Kafka Consumer Group API, and the number of consumer instances can be adjusted in conjunction with container orchestration tools.

[0090] Understandably, in cloud resources, the role of consumer instances is to actively request and process data from throughput nodes. The more consumer instances there are, the higher the throughput capacity of the throughput nodes.

[0091] It is understandable that since vehicle message data is constantly being generated, if the consumer in the cloud resources cannot consume (i.e. process) this generated vehicle message data in a timely manner, information backlog will occur. Therefore, as another optional implementation, consumer instances can be automatically added or removed based on the message backlog.

[0092] For example, when the message backlog (Lag) exceeds a second preset backlog, it is determined that the current vehicle message traffic to be input to the cloud device is greater than a second threshold, and consumer instances can be added; when the message backlog is less than the second preset backlog, it is determined that the current vehicle message traffic to be input to the cloud device is less than a third threshold, and consumer instances can be reduced; or, when the message pair metering is less than the second preset backlog and exceeds 10 minutes, it is determined that the current vehicle message traffic to be input to the cloud device is less than the third threshold and exceeds 10 minutes, and consumer instances can also be reduced.

[0093] For example, message backlog can also be monitored through the Kafka Consumer Group API, and the number of consumer instances can be adjusted using container orchestration tools.

[0094] Understandably, when the current vehicle packet traffic to be input to the cloud device exceeds the second threshold, increasing the number of consumer instances in the cloud resources can ensure that the vehicle packet traffic can be processed in a timely manner, improve the processing efficiency of vehicle packet traffic, and reduce the backlog of vehicle packet traffic. When the current vehicle packet traffic to be input to the cloud device is less than the third threshold, it indicates that the current vehicle packet traffic is relatively low. By reducing the number of consumer instances in the cloud resources, excess consumer instances can be released, reducing resource waste. Alternatively, considering misjudgment, to reduce the occurrence of misjudgment, a time factor can be added, that is, when the predicted current vehicle packet traffic is less than the third threshold and exceeds a specified time, the number of consumer instances in the cloud resources can be reduced.

[0095] Understandably, since these are cloud resources, which include both cloud-based and on-premises Kafka clusters, reducing the number of consumer instances in the cloud-based Kafka cluster necessitates reducing the number of consumer instances in the on-premises Kafka cluster as well, in order to conserve resources in both categories.

[0096] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction results includes: triggering partition reallocation based on the increase or decrease of consumer instances to ensure load balancing among consumers. For example, if the number of consumer instances increases, a new partition is added to the region where the consumer instance is located; conversely, if the number of consumer instances decreases, the region where the consumer instance is located is reduced.

[0097] For example, Kafka's sticky assigner strategy can be used to maintain the assignment relationship, which greatly optimizes the consumer rebalancing process and reduces unnecessary partition movement and related overhead.

[0098] In one optional embodiment, adjusting the configuration of cloud resources based on the prediction results includes: dividing the current vehicle message traffic to be input to the cloud device according to the importance of the vehicle message service, and determining the importance level of each vehicle message; determining the area for processing high-importance vehicle messages as an independent area in the cloud resources, or determining the throughput node for processing high-importance vehicle messages as an independent throughput node; and compressing low-importance vehicle messages before transmission.

[0099] Among them, high-priority vehicle messages include fault alarm information; low-priority vehicle messages include routine status information.

[0100] Understandably, by designating the processing area for high-priority vehicle messages as an independent area within cloud resources or by designating the throughput nodes for processing high-priority vehicle messages as independent throughput nodes, it is possible to ensure that high-priority vehicle messages are processed in a timely manner and reduce latency; by compressing low-priority vehicle messages before transmission, resource waste can be reduced.

[0101] Understandably, since these are cloud resources, including both cloud-based and on-premises Kafka clusters, dedicated throughput nodes or regions are allocated to high-priority vehicle messages in the cloud-based Kafka cluster. Correspondingly, dedicated throughput nodes or regions should also be allocated to high-priority vehicle messages in the on-premises Kafka cluster. Similarly, low-priority vehicle messages in the cloud-based Kafka cluster are compressed before transmission, and correspondingly, low-priority vehicle messages in the on-premises Kafka cluster also need to be compressed before transmission to ensure synchronization and consistent processing capabilities between cloud and on-premises transmissions.

[0102] It is understandable that after adjusting the cloud resource configuration based on the prediction results, data transmission is still required. In order to improve transmission efficiency and stability, as an optional embodiment, the cloud resource processing method provided in this application embodiment also includes a transmission method.

[0103] The transmission method dynamically selects the optimal data transmission path by comprehensively analyzing network status, data traffic characteristics, and resource load. First, it can collect real-time data on network latency, bandwidth utilization, and node load metrics between on-premises and cloud-based Kafka clusters to construct a multi-dimensional evaluation model. Second, based on the evaluation results, transmission paths are dynamically prioritized, selecting those with low latency and high availability. For example, if the network latency of a path exceeds a threshold, it automatically switches to a backup path. Furthermore, the transmission method supports differentiated scheduling strategies, allocating dedicated network channels for high-priority data to ensure real-time synchronization of critical messages.

[0104] As an optional embodiment, the transmission method may include the following steps: S103A1. Based on cloud resources, the prediction results are processed to characterize the current vehicle message traffic to be input to the cloud device, and the vehicle message traffic to be sent is obtained.

[0105] S103A2. Based on the importance level of each vehicle message in the vehicle message traffic to be sent, determine the target compression algorithm corresponding to vehicle messages of different importance levels.

[0106] In one optional embodiment, for high-priority vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm for the high-priority vehicle packets is determined to be a low-latency compression algorithm; for low-priority vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm for the low-priority vehicle packets is determined to be a high compression ratio algorithm.

[0107] As described above, high-priority vehicle messages are fault alarm messages, while low-priority vehicle messages are regular messages. For example, the low-latency compression algorithm could be Snappy compression, and the high-compression-ratio algorithm could be the Z-standard compression algorithm.

[0108] It is understandable that by using a low-latency compression algorithm for high-priority vehicle packets in the outgoing vehicle packet traffic, a balance can be struck between transmission efficiency and real-time performance; by using a high compression ratio algorithm for low-priority vehicle packets in the outgoing vehicle packet traffic, the amount of data transmitted can be reduced.

[0109] S103A3: Vehicle messages of different importance levels are compressed using the corresponding target compression algorithm before transmission.

[0110] It is understandable that during transmission, there are multiple transmission paths. Since transmission has not yet occurred, each transmission path can be referred to as a candidate transmission path. As an optional embodiment, each candidate path can be scored for health, and then the candidate transmission path with the highest health score can be selected as the target transmission path for transmission. As another optional embodiment, at least two candidate paths can be scored for health, and then the candidate transmission path with the highest health score can be selected as the target transmission path for transmission.

[0111] This application does not impose any particular limitations on this. In practical applications, the choice can be made based on the needs and the availability of processing resources. That is, if there are spare processing resources and / or the goal is to ensure optimal transmission efficiency and the most stable transmission, then a health score can be assigned to each candidate path, and the candidate transmission path with the highest health score can be selected as the target transmission path. If processing resources are limited and / or a relatively fast transmission rate and relatively stable transmission are desired, then a health score can be assigned to at least two candidate paths, and the candidate transmission path with the highest health score can be selected as the target transmission path.

[0112] The following example illustrates how to evaluate the health of at least two candidate paths and then select the candidate transmission path with the highest health score as the target transmission path. It is understood that those skilled in the art, knowing the principle of evaluating the health of at least two candidate paths and then selecting the candidate transmission path with the highest health score as the target transmission path, can easily deduce the scheme of evaluating the health of each candidate path and then selecting the candidate transmission path with the highest health score as the target transmission path; therefore, it will not be elaborated upon here.

[0113] For example, vehicle messages of different importance levels are compressed using the corresponding target compression algorithm before transmission, including the following steps: S103A3a: Determine the health scores of at least two candidate transmission paths for the vehicle message traffic to be sent.

[0114] Among them, the health of the transmission path is used to evaluate the network latency, network bandwidth, and network load of the transmission path.

[0115] Among them, network latency reflects the time efficiency of data transmission through the candidate transmission path; network bandwidth reflects the data transmission capacity of the candidate transmission path per unit time; and network load reflects the current network resource usage pressure of the candidate transmission path.

[0116] Further, as an optional embodiment, for each of the at least two candidate transmission paths, the following steps are performed: determining the network latency, network bandwidth, and network load of the candidate transmission path; and determining the health score of the candidate transmission path based on the network latency, network bandwidth, and network load of the candidate transmission path.

[0117] For example, the health score of a candidate transmission path can be determined by the following formula (1): (1) Among them, S i Represents the health score of the i-th candidate transmission path; Normalized() represents the normalization process, scaling each indicator to the range [0, 1]; RTT i Represents the network delay of the i-th candidate transmission path; Bandwidth i Represents the network bandwidth of the i-th candidate transmission path; Load i Indicates the network load of the i-th candidate transmission path; Indicates network latency weights; Indicates network bandwidth weight; This indicates the network load weight.

[0118] Furthermore, as an optional embodiment, , and It can be adjusted according to the actual situation.

[0119] In one alternative embodiment, the size can be dynamically adjusted based on the prediction results. For example, if the predicted current vehicle message traffic to be input to the cloud device is at its peak, the size can be increased. (For example, increasing from 0.4 to 0.5), decrease .

[0120] In another alternative embodiment, improvements can be made during network congestion. .

[0121] Furthermore, as another alternative embodiment, reinforcement learning can be used to optimize the weights. , and This can minimize the historical average latency.

[0122] As an optional embodiment, the health scores of the target path and other candidate paths can be updated every preset time interval to record the actual transmission delay and prediction deviation. For example, the preset time interval can be 5 minutes. It is understood that 5 minutes is just an example of the preset time interval and not a limitation. In actual applications, the preset time interval can be set according to the actual situation.

[0123] S103A3b: Among at least two candidate transmission paths, select the transmission path with the highest health score as the target transmission path.

[0124] S103A3C: Transmits compressed vehicle messages of different importance levels using the target transmission path.

[0125] Understandably, compressing vehicle message traffic before transmission can free up bandwidth resources and improve transmission rates.

[0126] It is understandable that choosing the transmission path with the highest health score as the target transmission path can ensure transmission stability, improve transmission speed, reduce latency, and thus make the transmission synchronization faster.

[0127] Understandably, determining the network latency of candidate transmission paths reveals the time efficiency of data transmission over those paths; determining the network bandwidth reveals the data transmission capacity per unit time; determining the network load reveals the current network resource usage pressure. By determining the health score of a transmission path based on its network latency, bandwidth, and load, the real-time performance, throughput, and stability of the candidate paths can be evaluated. This allows for the selection of transmission paths with good real-time performance, high throughput, and strong stability as the target transmission path, thereby improving transmission speed while reducing congestion.

[0128] To improve transmission stability, as an optional implementation, in the event of an anomaly in the target transmission path, a switch to an alternative transmission path can be made to transmit compressed vehicle messages of different importance levels through the alternative transmission path.

[0129] Among them, the backup target transmission path is the candidate transmission path with the second highest health score.

[0130] In one example, when the network latency of the target transmission path is greater than the preset network latency, the target transmission path can be considered to be abnormal; in another example, when the packet loss rate of the target transmission path is greater than the preset packet loss rate, the target transmission path can be considered to be abnormal.

[0131] For example, the preset network latency can be 50 milliseconds (ms); the preset packet loss rate can be 5%. It is understood that 50ms is an example of a preset network latency, not a limitation thereof. The preset network latency can be set according to actual conditions and requirements, and this application does not impose any special limitations on it. Similarly, 5% is an example of a preset packet loss rate, not a limitation thereof. The preset packet loss rate can be set according to actual conditions and requirements, and this application does not impose any special limitations on it.

[0132] As another optional embodiment, the transmission method provided in this application can also implement batch processing. For example, based on cloud resources, the prediction results are processed to characterize the current vehicle message traffic to be input to the cloud device, resulting in the vehicle message traffic to be sent. When the vehicle message traffic to be sent is less than a preset vehicle message traffic, the vehicle message traffic to be sent can be merged into batch messages before transmission. The batch processing window size can be dynamically adjusted according to the prediction results: the window is reduced during peak traffic periods to decrease latency, and expanded during off-peak periods to increase throughput.

[0133] As another optional embodiment, the transmission method provided in this application also includes fault recovery and consistency assurance.

[0134] For example, a multi-tiered fault recovery mechanism was designed to handle network outages or cluster anomalies. When a synchronization failure is detected, the system first attempts to retry automatically, gradually increasing the retry interval to avoid a cascading failure. If the retry fails, a local caching mechanism is triggered to temporarily store the data in a disk queue, awaiting resynchronization once the network is restored.

[0135] Among them, "failed retry" means that if the number of retries reaches or exceeds the preset number, and the retries have not been successful, the retries are considered to have failed.

[0136] To ensure data consistency, the system implements exactly-once semantics based on the Kafka transaction mechanism and prevents data loss or duplication through periodic offset verification. Furthermore, the system records synchronization status checkpoints, supporting rapid recovery from breakpoints and reducing the resource consumption associated with full synchronization.

[0137] As another optional embodiment, the transmission method provided in this application embodiment also includes load balancing technology.

[0138] For example, dynamic allocation of synchronization tasks avoids overloading individual nodes. Distributed load balancers are deployed between the cloud-based and on-premises Kafka clusters to monitor message backlog and processing latency on each throughput node in real time. Based on monitoring data, a weighted round-robin algorithm is used to distribute synchronization tasks to lighter-loaded nodes, while rate limiting is implemented for high-load nodes. To address partition-level load imbalance, message distribution across partitions is analyzed periodically, and the mapping between partitions and throughput nodes is automatically adjusted using Kafka's partition reallocation tool to maximize resource utilization.

[0139] By employing advanced transmission methods and load balancing technologies, the efficiency and stability of data transmission between cross-cloud Kafka clusters are improved. Based on traffic prediction results and real-time resource status, data transmission paths, compression strategies, and fault recovery mechanisms are dynamically optimized to ensure efficient synchronization of vehicle message data between cloud and on-premises environments.

[0140] Examples of training methods for prediction models: This application also provides a method for training a prediction model, which is the prediction model in the embodiment of the cloud resource processing method.

[0141] Figure 2 A flowchart illustrating a training method for a prediction model provided in this application is shown below. Figure 2 As shown, the training method for this prediction model includes the following steps: S201. Based on the time information, geographical information and vehicle type information of historical vehicle message traffic, determine the time characteristics, geographical characteristics and vehicle type characteristics to form the first feature dataset.

[0142] Vehicle message traffic exhibits significant temporal correlation, geographical distribution characteristics, and vehicle type differences. Different vehicle types correspond to different vehicle message traffic, different regions correspond to different vehicle message traffic, and different times correspond to different vehicle message traffic. Based on the time, geographical, and vehicle type information of historical vehicle message traffic, feature engineering is used to determine (or extract) time features, geographical features, and vehicle type features, which can provide data support for subsequent training and prediction models.

[0143] For example, the following steps may be included: S201A: Collect historical vehicle messages to form historical vehicle message traffic.

[0144] It is understandable that historical vehicle message traffic includes a large number of historical vehicle messages (also known as historical vehicle message data), which can come from cloud-based Kafka clusters and on-premises Kafka clusters.

[0145] As an optional implementation, data can be collected in the following manner: First, Kafka consumer group subscription: Consumer groups are deployed in both cloud-based and on-premises Kafka clusters to subscribe to vehicle message data in real time and persist the vehicle message data to the distributed file system (Hadoop Distributed File System, HDFS). Secondly, scheduled batch retrieval: To avoid the impact of real-time data collection on system performance, a scheduled batch retrieval strategy is adopted, such as retrieving data every 5 minutes.

[0146] S201B: Extract features from historical vehicle traffic flow.

[0147] After obtaining historical vehicle packet traffic, which includes time information, geographical information, and vehicle type information, one optional embodiment can directly extract features from the historical vehicle packet traffic. To improve the quality of the historical vehicle packet traffic and reduce processing resource consumption, another optional embodiment can be to first clean the historical vehicle packet traffic before extracting features. This application does not impose any particular limitation on this approach; the following explanation will exemplify cleaning the historical vehicle packet traffic before feature extraction.

[0148] As an optional implementation, cleaning historical vehicle message traffic may include, but is not limited to, deduplication, verification, missing value handling, and outlier handling.

[0149] In one example, deduplication can be performed by using message identifiers (IDs) or timestamps to remove duplicate vehicle message data from historical vehicle message traffic.

[0150] In one example, data integrity can be ensured through a verification algorithm.

[0151] The verification algorithm can be any of the existing verification algorithms such as parity check or cyclic redundancy check (CRC), and this application does not make any special limitation on it.

[0152] In one example, missing value handling could involve using interpolation or default values ​​to fill in missing timestamps or geographic data in historical vehicle message traffic to preserve the integrity of the vehicle message data.

[0153] In one example, outlier handling could involve using the Z-score method to identify and smooth or remove anomalous vehicle packet data in historical vehicle packet traffic, thereby improving the accuracy of historical vehicle packet traffic.

[0154] Furthermore, to facilitate subsequent feature extraction and improve its efficiency, as an optional implementation method, the cleaned historical vehicle message traffic can be classified first, and then features can be extracted.

[0155] The classification of historical vehicle message traffic can be achieved by first collecting the timestamps of each vehicle message in the historical vehicle message traffic to form a timestamp dataset, then collecting the regional information of each vehicle message in the historical vehicle message traffic to form a regional dataset, then collecting the vehicle type information of each vehicle message in the historical vehicle message traffic to form a vehicle type dataset, and finally collecting the message content of each vehicle message in the historical vehicle message traffic to form a message content dataset.

[0156] The timestamp dataset records the timestamp generated for each historical vehicle message, which can be used to analyze the temporal distribution characteristics of vehicle message traffic. For example, the recording accuracy can be down to the millisecond level.

[0157] The geographic dataset includes the geographical locations of vehicles, such as provinces, cities, and regions, which can be used to analyze differences in vehicle packet traffic across different regions. The regions within the geographic location refer to different administrative areas, such as rural areas, urban areas, and suburbs. For example, analyzing differences in vehicle packet traffic across different regions could involve differences in packet traffic between different cities within the same province, differences in packet traffic within the same administrative region across different provinces, differences in packet traffic between different administrative regions within the same city, and so on.

[0158] The vehicle type dataset includes vehicle types such as commercial vehicles, passenger vehicles, and new energy vehicles, which can be used to analyze the generation patterns of vehicle message traffic for different types of vehicles.

[0159] The message content data includes the specific message content of each historical vehicle message, such as vehicle speed, engine status, and fault codes, which can be used to assist in analyzing the characteristics of vehicle message traffic.

[0160] Furthermore, in some optional embodiments, extracting time features, geographical features, and vehicle type features from historical vehicle message traffic includes: S201B1. Because vehicle message traffic has a significant time correlation, the following time characteristics of historical vehicle message traffic can be extracted to obtain time characteristic data, which, for example, include: 1) Hourly cycle: It can analyze the changes in vehicle message traffic at different times of the day. For example, the peak vehicle message traffic during the morning peak (7:00-9:00) and evening peak (17:00-19:00) will be higher than the vehicle message traffic at other times.

[0161] 2) Daily cycle: It can analyze the difference in vehicle message traffic between weekdays and weekends. For example, the vehicle message traffic during weekday commuting hours is significantly higher than the vehicle message traffic during the same period on weekends.

[0162] 3) Seasonal cycles: It can analyze changes in vehicle message traffic in different seasons. For example, vehicle fault messages may increase in winter.

[0163] 4) Holiday characteristics: Mark statutory holidays, such as Spring Festival and National Day, to analyze changes in vehicle message traffic during holidays.

[0164] 5) Special event characteristics: such as the impact of extreme weather such as rainstorms and heavy snow or large-scale events such as auto shows and sports events on vehicle message traffic.

[0165] S201B2. Due to significant differences in vehicle density and driving habits across different regions, the following regional characteristics can be extracted from historical vehicle message traffic to obtain regional characteristic data, which, for example, include: 1) Traffic Distribution: Statistics on vehicle message traffic for different city types or different regional types. For example, vehicle message traffic in first-tier cities is significantly higher than that in third- and fourth-tier cities. Regional type refers to different administrative regions, which may include, but are not limited to, urban areas, suburbs, and rural areas.

[0166] 2) Traffic characteristics: Analyze the differences in vehicle message flow between congested and non-congested areas.

[0167] S201B3. Since different types of vehicles have different message generation patterns, the following vehicle type characteristics can be extracted from historical vehicle message traffic to obtain vehicle type characteristic data, which, for example, includes: 1) Vehicle type distribution: For example, we can count the proportion of vehicle messages from passenger cars, commercial vehicles and new energy vehicles in the total vehicle message traffic.

[0168] 2) Vehicle status characteristics: Analyze the impact of vehicle operating status (such as driving, idling, and engine shutdown) on the frequency of vehicle message generation.

[0169] Understandably, when extracting the time characteristics of historical vehicle message traffic, it can be extracted directly from the timestamp data; when extracting the regional characteristics of historical vehicle message traffic, it can be extracted directly from the regional data; and when extracting the vehicle type of historical vehicle message traffic, it can be extracted directly from the vehicle type data. In this way, each characteristic can be obtained quickly.

[0170] As an optional embodiment, feature statistics can also be performed on the collected historical vehicle message traffic. These feature statistics may include: 1) Mean and variance: For example, the mean and variance of historical hourly or daily vehicle message traffic can be statistically analyzed. This can be used to measure the stability of historical vehicle message traffic and provide a reference for predicting whether future vehicle message traffic will be stable.

[0171] 2) Peak and Valley Values: It can identify the peak and valley values ​​in historical vehicle packet traffic, which can provide a reference for predicting whether future vehicle packet traffic belongs to the peak or valley value, which is beneficial for subsequent resource allocation.

[0172] 3) Growth rate: By calculating the rate of change of historical vehicle message traffic in adjacent time periods, the trend of future vehicle message traffic can be predicted.

[0173] It is understood that, in one optional embodiment, the feature data obtained in step S201B can be directly output as the first feature dataset to train the candidate model; to improve the training effect and enable the obtained prediction model to have better prediction results, in another optional embodiment, step S201C can be executed, and the feature data after executing step S201C can be output as the first feature dataset to train the candidate model. This application does not make any special limitation on this, and the following will take the example of using the feature data after executing step S201C as the first feature dataset to train the candidate model for illustrative purposes.

[0174] S201C: Convert each feature data into machine language.

[0175] In some alternative embodiments, the machine language conversion of the feature data may include, but is not limited to, feature encoding and feature standardization.

[0176] Different types of feature data can be encoded using different feature encoding methods.

[0177] For example, for regional feature data and vehicle type feature data, one-hot encoding can be used to eliminate the erroneous correlation of unordered categorical variables in the numerical process, ensuring that the machine learning model can accurately interpret discrete features.

[0178] For example, for time-feature data, timestamps can be converted into numerical features, which can solve the problems of storage, calculation and compatibility of the original time-feature data.

[0179] This requires standardizing all encoded feature data to eliminate the influence of units.

[0180] To facilitate subsequent training of candidate models and improve training efficiency, as an optional implementation, the first feature dataset can be output in tabular form.

[0181] The first feature dataset may include a time feature table, a geographical feature table, a vehicle type feature table, and a traffic statistics table.

[0182] Understandably, the time feature table is composed of time features extracted from historical vehicle message traffic, which may include fields such as timestamp, hour, day of the week, and whether it is a holiday.

[0183] It is understandable that the regional feature table is composed of regional features extracted from historical vehicle message traffic, which may include fields such as city identification (ID), region type, and region traffic share.

[0184] Among them, the city ID is also the area code, and each city has a corresponding identifier, which is different for different cities; the area type is also called the region type, which can include but is not limited to urban areas, suburbs, and rural areas; the area traffic share can also be called the area vehicle traffic share, which is the ratio between the vehicle message traffic of a certain area in a certain time period and the vehicle message traffic of the whole country in that time period.

[0185] It is understandable that the vehicle type feature table is composed of vehicle type features extracted from historical vehicle packet traffic, and may include fields such as vehicle type ID, vehicle status, and packet generation frequency.

[0186] Among them, the vehicle type ID can also be called the vehicle type number, that is, each type of vehicle has a corresponding identifier, and different types of vehicles have different identifiers; the message generation frequency is the frequency at which a single vehicle message data is generated.

[0187] It is understandable that the traffic statistics table is obtained by performing feature statistics on the collected historical vehicle message traffic, and may include fields such as the mean, variance, peak value, valley value and growth rate of historical vehicle message traffic.

[0188] Understandably, outputting the first feature dataset in tabular form as input for training candidate models not only provides a large amount of data support for the training of candidate models, but also improves training efficiency.

[0189] S202. Train the candidate model using the first feature dataset to obtain the prediction model; so as to dynamically adjust the configuration of cloud resources according to the prediction results of the prediction model.

[0190] The prediction results are used to characterize the current vehicle message traffic volume to be input to the cloud device.

[0191] The goal of training the prediction model is to analyze historical vehicle message traffic using machine learning algorithms, build a high-precision prediction model, and capture the periodicity, trend, and suddenness characteristics of historical vehicle message traffic.

[0192] Understandably, since vehicle messages have significant time correlation, geographical distribution characteristics, and vehicle type differences, time features, geographical features, and vehicle type features can be determined based on the time information, geographical information, and vehicle type information of historical vehicle message traffic to form a first feature dataset. This dataset can provide data support for subsequent training of candidate models and can also capture the inherent patterns of the data in the first feature dataset, enabling the trained model to predict the trend of vehicle message traffic. By dynamically adjusting the configuration of cloud resources based on the prediction results of the prediction model, cloud resources can be fully utilized and resource waste can be reduced.

[0193] In the embodiments of this application, the candidate model may include one or more. In order to improve the accuracy of prediction, the following will take the example of having multiple candidate models as an example for illustrative explanation.

[0194] When there are multiple candidate models, the candidate models are trained using the first feature dataset to obtain the prediction model, including the following steps: S202A. The second feature dataset is obtained by filtering the first feature dataset using correlation analysis algorithms and / or feature importance assessment algorithms.

[0195] It is understandable that the first feature dataset obtained in step S201 contains subjective factors, that is, engineers extract features based on their own opinions on which features affect the prediction of vehicle message traffic. This extraction is not rational enough. Therefore, in order to ensure rationality and improve scientific rigor, as an optional embodiment, it is necessary to use correlation analysis algorithms and / or feature importance assessment algorithms to filter the first feature dataset to obtain a second feature dataset. After filtering the first feature dataset using correlation analysis algorithms and / or feature importance assessment algorithms, the obtained feature data are the features that have the most impact on the prediction of vehicle message traffic.

[0196] For example, the correlation analysis algorithm can be the Pearson coefficient correlation analysis algorithm; the feature importance assessment algorithm can be a tree-based feature importance ranking method.

[0197] For example, the obtained second feature dataset may include time features such as hour, day of the week, and whether it is a holiday; the obtained second feature dataset may include regional features such as region type and region traffic share; the obtained second feature dataset may include vehicle type features such as vehicle type distribution and vehicle status.

[0198] S202B: Train each of the multiple candidate models using the second feature dataset.

[0199] In one optional embodiment, a time-series partitioning strategy can be employed to divide the second feature dataset into a training set, a validation set, and a test set. This ensures the temporal continuity of the training, validation, and test sets, preventing data omissions or leaks. Understandably, the training set is used to train each candidate model, the validation set is used to evaluate the effectiveness of the hyperparameters (e.g., learning rate and number of network layers) of each candidate model, and the test set is used to evaluate the final performance (e.g., prediction accuracy and inference time) of each prediction model.

[0200] For example, 70% of the second feature dataset is allocated to the training set, 15% to the validation set, and 15% to the test set. It is understood that the 70%, 15%, and 15% figures are examples of how much of the dataset is allocated to the training, validation, and test sets, respectively, and are not limiting. In practical applications, other values ​​can be used, and this application does not impose any particular limitations on this, only requiring that the data in the training set be greater than the sum of the data in the validation and test sets.

[0201] For example, candidate models can be divided into three categories: the first category includes traditional time series models, the second category includes tree models, and the third category includes deep learning models.

[0202] Furthermore, exemplarily, traditional time series models may include: the Autoregressive Integrated Moving Average (ARIMA) model and the Seasonal Autoregressive Integrated Moving Average (SARIMA) model. It is understood that in practical applications, traditional time series models may include more, or may include only one.

[0203] Furthermore, exemplarily, tree models may include: eXtreme Gradient Boosting (XGBoost) and Light Gradient Boosting Machine (LightGBM), suitable for capturing non-linear feature interactions. It is understood that in practical applications, tree models may include more, or may include only one.

[0204] Furthermore, exemplarily, a deep learning model may include a Long Short-Term Memory (LSTM) network and a Transformer, suitable for handling long-term dependencies. It is understood that in practical applications, a deep learning model may include more, or may include only one.

[0205] In order to improve the prediction accuracy of the candidate models obtained by training, as another optional embodiment, the hyperparameters of each candidate model can be tuned and a loss function can be added during the training process.

[0206] For example, the hyperparameters of each prediction model can be tuned through grid search or Bayesian optimization. For instance, the number of layers, the number of hidden units, and the learning rate of LSTM can be tuned; the tree depth, the learning rate, and the subsampling ratio of XGBoost can be tuned.

[0207] For example, the loss function can be the mean squared error (MSE) or the mean absolute error (MAE). Understandably, the loss function is included to evaluate the training performance and to see if the difference between the predicted and target values ​​is too large.

[0208] S202C: Evaluate the prediction accuracy and real-time performance of each trained candidate model, and select the candidate model that meets the preset evaluation conditions as the prediction model.

[0209] For example, the prediction accuracy of each trained candidate model can be evaluated using at least one of the following: for example, MSE, MAE, and Fraction.

[0210] Among them, for When evaluating the prediction accuracy of each candidate model using scores, The score values ​​typically range from [0, 1], and the candidate model's The closer the score is to 1, the stronger the explanatory power of the candidate model.

[0211] For example, real-time performance refers to the inference time of each candidate model.

[0212] In one alternative embodiment, the preset evaluation conditions are related to the evaluation method and real-time performance selected when evaluating the prediction accuracy of each trained candidate model.

[0213] For example, when the evaluation method uses MSE, the preset evaluation condition can be to satisfy both the minimum MSE and the shortest inference time. The prediction condition can also be the best overall performance (i.e., a small MSE and a short inference time). From the candidate models, for example, the candidate model with the smallest MSE and the shortest inference time can be selected first as the prediction model. If there is no candidate model that can simultaneously satisfy the minimum MSE and the shortest inference time, the candidate model with a small MSE and a short inference time can be selected as the prediction model.

[0214] For example, when the evaluation method is based on MAE, the preset evaluation condition can be to satisfy both the minimum MAE and the shortest inference time. The prediction condition can also be the best overall performance (i.e., a small MAE and a short inference time). For example, from the candidate models, the candidate model with the smallest MAE and the shortest inference time can be selected first as the prediction model. If there is no candidate model that can simultaneously satisfy both the minimum MAE and the shortest inference time, the candidate model with a small MAE and a short inference time can be selected as the prediction model.

[0215] For example, in the evaluation method, it is used When conducting score assessments, the preset assessment criteria can be those that both meet the requirements and are not met. The prediction condition can also be the optimal overall performance (i.e., the maximum value that simultaneously satisfies the shortest inference time). (Larger in size and shorter inference time). For example, from the candidate models, priority can be given to selecting the best one. The candidate model with the highest score and the shortest inference time is selected as the prediction model. If none of the candidate models simultaneously meet these conditions, then... When choosing the candidate model with the highest score and shortest inference time, you can select... Candidate models with high scores and short inference times are used as prediction models.

[0216] For example, when the evaluation method is MSE, MAE and When evaluating scores, the preset evaluation criteria can be those that satisfy both the minimum MSE and MAE. The prediction condition can also be to maximize the overall performance while simultaneously minimizing inference time (i.e., minimizing both MSE and MAE). (Large size and short inference time). For example, from the candidate models, the one with the smallest MSE, smallest MAE, and smallest MSE can be selected first. The candidate model with the highest score and the shortest inference time is used as the prediction model. If none of the candidate models can simultaneously satisfy the minimum MSE, minimum MAE, and minimum inference time, then... When choosing a candidate model that has the highest score and the shortest inference time, one can select a model that simultaneously satisfies the following conditions: low MSE, low MAE, and... Candidate models with high scores and short inference time are used as prediction models for prediction.

[0217] To achieve higher accuracy and robustness in the determined prediction model, S202C, as another optional embodiment, can also involve: evaluating the prediction accuracy and real-time performance of each trained candidate model, and determining the prediction model based on each trained candidate model and its corresponding weight. This allows the candidate models to work together to predict vehicle message traffic, significantly improving prediction performance.

[0218] The weights corresponding to each candidate model are related to the prediction accuracy and real-time performance of each candidate model.

[0219] In one optional embodiment, determining the prediction model based on the trained candidate models and their corresponding weights includes: First, determine the accuracy weight corresponding to the prediction accuracy of each candidate model and the real-time weight corresponding to the real-time performance of each candidate model.

[0220] The accuracy weights of each candidate model can be the same or different. The accuracy weights of each candidate model can be dynamically adjusted according to the performance of each candidate model. For example, the accuracy weight of a candidate model with high prediction accuracy is high, and the accuracy weight of a candidate model with low prediction accuracy is low. The real-time weights of each candidate model can also be the same or different. The real-time weights of each candidate model can be dynamically adjusted according to the performance of each candidate model. For example, the real-time weight of a candidate model with high real-time performance (i.e., long inference time) is low, and the real-time weight of a candidate model with low real-time performance (i.e., short inference time) is high.

[0221] Then, the product corresponding to each candidate model is determined.

[0222] Here, the product corresponding to each candidate model refers to the value obtained by multiplying each candidate model by the sum of its corresponding accuracy weight and real-time performance weight.

[0223] Finally, the products of each candidate model are summed to obtain the predicted model.

[0224] It is understood that the above embodiments provide two ways to obtain the prediction model: one is to select one from multiple trained candidate models as the prediction model, or to integrate multiple candidate models to obtain the prediction model. A second feature dataset is obtained by filtering the first feature dataset using correlation analysis algorithms and / or feature importance assessment algorithms. The features in the second feature dataset have a more significant impact on the prediction of the prediction model. Training each candidate model from multiple candidate models using the second feature dataset makes the prediction of vehicle packet traffic more accurate. Evaluating the prediction accuracy and real-time performance of each trained candidate model yields the accuracy of each model's prediction of vehicle packet traffic and the time required to obtain the prediction result. When selecting a prediction model from multiple trained candidate models, using preset evaluation conditions ensures that the selected model better meets the expectations for vehicle packet traffic prediction, adapting to situations with limited computing resources. When integrating multiple candidate models to obtain the prediction model, the prediction model is determined based on the trained candidate models and their corresponding weights, resulting in a prediction model with higher accuracy and robustness. The collaborative prediction of vehicle packet traffic by the candidate models significantly improves prediction performance.

[0225] As another optional embodiment, the prediction model trained in this application embodiment also includes an online learning function.

[0226] For example, online learning functionality may include incremental training and a sliding window mechanism.

[0227] Incremental learning can involve periodically inputting new traffic data into the model for incremental training to adapt to changes in traffic patterns.

[0228] For example, The sliding window mechanism can use data from a fixed time window as the training set to ensure that the model always learns the latest patterns.

[0229] As another optional embodiment, the prediction model trained in this application embodiment also includes a feedback optimization function.

[0230] For example, feedback optimization features may include prediction error monitoring and abnormal traffic detection.

[0231] Among them, prediction error monitoring can calculate the error between the predicted value and the actual value in real time, and trigger model retraining when the error exceeds a threshold.

[0232] Abnormal traffic detection can be achieved by using the Isolation Forest algorithm or the Local Outlier Factor (LOF) algorithm to detect abnormal traffic and dynamically adjust the model parameters.

[0233] As another optional embodiment, the prediction model trained in this application embodiment also includes fault recovery and consistency assurance.

[0234] For example, a multi-tiered fault recovery mechanism was designed to handle network outages or cluster anomalies. When a synchronization failure is detected, the system first attempts to retry automatically, gradually increasing the retry interval to avoid a cascading failure. If the retry fails, a local caching mechanism is triggered to temporarily store the data in a disk queue, awaiting resynchronization once the network is restored.

[0235] To ensure data consistency, the system implements exactly-once semantics based on the Kafka transaction mechanism and prevents data loss or duplication through periodic offset verification. Furthermore, the system records synchronization status checkpoints, supporting rapid recovery from breakpoints and reducing the resource consumption associated with full synchronization.

[0236] The following examples illustrate possible implementation schemes for the cloud resource processing method and the prediction model training method described in one or more of the above embodiments.

[0237] Cloudera Distribution for Hadoop (CDH) is an enterprise-grade big data platform deployed in a cloud computing environment. It combines the traditional CDH distributed computing (Hadoop) ecosystem components with the elastic resource management capabilities of cloud services, providing big data solutions with on-demand scaling, automated operation and maintenance, and hybrid cloud support.

[0238] Cloudera Data Platform (CDP) is an enterprise-grade big data platform launched by Cloudera Corporation. It is deployed locally and mainly runs on the user's own private data center or local server, providing complete big data management capabilities.

[0239] Hot data refers to data that is frequently accessed within a specific time period, is highly time-sensitive, and is crucial to business operations. It requires high-performance storage to achieve fast read and write responses.

[0240] Application Programming Interface (API): An API is a set of predefined functions or protocols used to enable interaction and data sharing between different software systems. Developers can call these functions without needing to understand their internal implementation details.

[0241] Controller Area Network (CAN): CAN is a serial communication protocol widely used in automotive electronics, industrial control, and other fields, primarily for real-time data exchange between devices. Its core features include high reliability and interference immunity, and it is commonly used for communication between sensors, actuators, and controllers.

[0242] The solution provided in this embodiment belongs to the field of vehicle network data synchronization technology, specifically involving a dynamic adjustment method for cloud resources based on a predictive model, which is particularly suitable for vehicle message data synchronization scenarios between public and private clouds.

[0243] With the rapid development of vehicle-to-everything (V2X) technology and the widespread adoption of intelligent vehicles, the CAN bus data (i.e., vehicle message data) generated by vehicles is experiencing explosive growth, reaching hundreds of billions of data points daily. This data is crucial for applications such as vehicle status monitoring, driving behavior analysis, and intelligent traffic management. Currently, the storage and analysis of V2X data primarily employs a combined cloud (public cloud) and on-premises (private cloud) architecture: the cloud-based CDH big data cluster handles hot data storage, while the on-premises CDP big data cluster undertakes the storage and in-depth analysis of all data. Under this architecture, the core data synchronization process involves pulling data from the cloud-based Kafka cluster to the on-premises Kafka cluster for subsequent processing. However, related technical solutions have the following significant shortcomings when addressing the massive data synchronization needs of V2X scenarios: 1. Difficult to handle application scenarios with large data volumes and high real-time requirements: In the context of connected vehicles, with the continuous growth in car sales and the increasing intelligence of vehicles, modern cars generate hundreds of billions of vehicle message data every day. This data includes not only vehicle operating status and sensor information, but also highly real-time business data such as intelligent driving and vehicle-to-machine interaction. Due to the high frequency and high concurrency of vehicle message data, traditional batch data transmission modes are insufficient to meet real-time analysis needs, thus requiring reliance on distributed message queues such as Kafka for low-latency data synchronization. However, current Kafka clusters often experience message backlogs during cross-cloud data synchronization due to surges in data volume, even causing consumer delays and severely impacting the timeliness of downstream data analysis. Furthermore, connected vehicle services have extremely high real-time requirements for data synchronization; for example, scenarios such as autonomous driving data analysis and remote fault diagnosis require millisecond-level responses, while existing synchronization mechanisms struggle to maintain stable low-latency transmission during data surges.

[0244] 2. There are processing deficiencies during data surges: Current Kafka cross-cloud data synchronization solutions typically employ a static resource configuration strategy, pre-allocating fixed computing resources (such as throughput nodes, number of partitions, and consumer group instances). When faced with sudden surges in vehicle message traffic (i.e., the sum of several vehicle message data points), they cannot dynamically adjust based on actual traffic fluctuations. For example, during peak traffic hours, the volume of vehicle message data may increase several times over, leading to a surge in vehicle message traffic. However, current cross-cloud resources lack automatic scaling capabilities, resulting in insufficient processing capacity for the cloud-based Kafka cluster and severe message backlog. Furthermore, when the volume of vehicle message data decreases, the fixed resource allocation results in idle and wasted computing resources, increasing cloud service costs. While some solutions attempt to manually adjust resources to cope with traffic changes, the lack of accurate traffic prediction capabilities means that manual intervention often lags behind actual demand, failing to scale up in advance before data peaks, thus perpetuating synchronization delays.

[0245] 3. Insufficient cross-cloud resource collaboration: In the data architecture of the Internet of Vehicles (IoV), cross-cloud resources include on-cloud Kafka and on-premises Kafka. The on-cloud environment (public cloud) is typically used for storing and processing frequently accessed data, while the on-premises environment (private cloud) is responsible for long-term storage and in-depth analysis of all data. Currently, the allocation of Kafka resources on the cloud (CDH cluster) and on-premises (CDP cluster) is usually static or manually adjusted, lacking an automated cross-cloud resource coordination mechanism, leading to uneven resource allocation. For example, when the on-cloud Kafka cluster is overloaded due to a surge in vehicle message data, on-premises consumers may be unable to consume data in a timely manner due to insufficient resources, and vice versa. Furthermore, there is competition for network bandwidth, storage input (I) / output (O) resources between different cloud environments, and current synchronization strategies fail to dynamically optimize based on network conditions and cluster load, easily causing data transmission bottlenecks. Although some research attempts to optimize Kafka consumer allocation through load balancing techniques, the core problem of cross-cloud resource coordination and scheduling remains unresolved, resulting in low resource utilization and unstable synchronization efficiency.

[0246] 4. Lack of dynamic traffic forecasting capabilities: Most existing Kafka cross-cloud synchronization solutions employ a passive, reactive resource adjustment strategy, triggering resource expansion only after message backlog or increased latency is detected. This approach exhibits significant lag. Since vehicle message traffic exhibits strong spatiotemporal regularities (such as peak hours and holiday travel peaks), combining historical data for traffic prediction would allow for proactive optimization of resource allocation, preventing sudden traffic surges. However, current vehicle-to-everything (V2X) data synchronization systems generally lack machine learning-based dynamic traffic prediction capabilities, failing to accurately predict future vehicle message data volumes, resulting in consistently reactive resource adjustments. Furthermore, vehicle data traffic can be affected by unforeseen events (such as severe weather and large-scale events), and traditional static models struggle to adapt to such dynamic changes, further exacerbating the irrationality of resource allocation.

[0247] I. Technical problem to be solved in this embodiment: In the context of connected vehicles, the amount of CAN message data generated by vehicles (i.e., vehicle message data) is growing exponentially. Data synchronization between cloud (public cloud) and on-premises (private cloud) Kafka clusters faces the following key challenges: 1. Large data volume and high real-time requirements: Vehicles generate hundreds of billions of CAN message data (i.e., vehicle message data) every day. Traditional static resource allocation methods cannot adapt to sudden traffic, resulting in data synchronization delays or loss.

[0248] 2. Data surge handling defects: The relevant Kafka cross-cloud synchronization mechanism lacks the ability to predict traffic fluctuations and cannot adjust resources in advance, resulting in message backlog during traffic surges and affecting the timeliness of data processing.

[0249] 3. Insufficient cloud resource coordination: Uneven resource allocation between public cloud Kafka clusters and private cloud Kafka clusters may result in one side being overloaded while the other side is idle, leading to resource waste or synchronization bottlenecks.

[0250] 4. Lack of dynamic traffic prediction capability: Related solutions usually use fixed resource configuration or simple threshold-triggered scaling up and down, which cannot make intelligent predictions based on historical traffic patterns, resulting in a lag in resource adjustment.

[0251] This embodiment aims to solve the above problems by proposing a dynamic adjustment method for cloud resources based on a predictive model. By predicting traffic trends through machine learning and combining it with a dynamic scaling strategy for the Kafka cluster, the optimal allocation of cloud resources can be achieved, improving data synchronization efficiency and reducing resource waste.

[0252] II. Overview of the technical solution in this embodiment: This embodiment proposes a dynamic adjustment method for cloud resources based on a predictive model, aiming to address issues such as uneven resource allocation, synchronization latency, and insufficient data surge handling capacity when synchronizing massive vehicle message data across clouds in connected vehicle scenarios. This method combines a traffic prediction model with a dynamic resource scheduling strategy to achieve intelligent collaborative allocation of resources between public and private cloud Kafka clusters, improving data synchronization efficiency and reducing resource waste. The core steps of this embodiment are as follows: 1. Data acquisition and feature extraction.

[0253] Historical vehicle traffic data is collected and categorized by timestamp, region, and vehicle type. Periodic features (such as morning and evening rush hours) and event features (such as holidays and extreme weather) are extracted through feature engineering to provide high-value input for the prediction model.

[0254] 2. Train the prediction model.

[0255] Based on historical vehicle traffic data, a predictive model is constructed using time series analysis and machine learning algorithms to identify periodic patterns and sudden changes in traffic flow. Simultaneously, a real-time monitoring and feedback mechanism is introduced to dynamically adjust the parameters of the predictive model to adapt to real-time changes in vehicle traffic flow, thereby improving prediction accuracy.

[0256] 3. Dynamic resource allocation strategy.

[0257] Based on the prediction results of the predictive model, Kafka cluster resources are dynamically adjusted to achieve efficient collaboration across cloud environments. The number of Kafka Broker nodes is elastically scaled up and down via the cloud platform API to ensure resource supply matches traffic demand; consumer group instances are automatically scaled up and down to avoid latency caused by message backlog; when predicted traffic exceeds the current partition's processing capacity, the Kafka Admin API is called to dynamically add partitions, improving parallel processing efficiency. Simultaneously, differentiated priorities are set for different business data (such as emergency alarm messages and routine status data), prioritizing the synchronization of high-time-sensitive data and preventing low-priority data from consuming critical resources.

[0258] 4. Optimized data synchronization.

[0259] By dynamically selecting the optimal data transmission path (such as a dedicated line or public network) through intelligent scheduling algorithms, and combining real-time network status, data traffic, and resource load conditions, cross-cloud synchronization latency is reduced. Load balancing technology is applied across Kafka clusters to ensure even data distribution, avoid single-node overload, and improve overall system stability. Furthermore, by monitoring synchronization progress and resource utilization in real time, data transmission strategies are dynamically adjusted to further optimize synchronization efficiency, ensuring efficient and reliable transmission of massive vehicle messages across cloud environments.

[0260] III. Specific process of the solution provided in this embodiment: Figure 3 A flowchart illustrating a method for dynamically adjusting cloud resources based on a predictive model, as provided in this application embodiment, is shown below. Figure 3 As shown, the method includes the following steps: S301, Data Acquisition and Feature Extraction.

[0261] The data acquisition and feature extraction are performed by the data acquisition and feature processing module, which is the core foundation module of this embodiment. Its main objective is to collect historical vehicle message data and extract key features from it, providing high-quality data support for subsequent prediction model training. In the vehicle-to-everything (V2X) scenario, vehicle message data exhibits significant temporal correlation, geographical distribution characteristics, and vehicle type differences. Therefore, this module needs to classify, clean, and extract features from the raw data from multiple dimensions to capture the inherent patterns in the data.

[0262] S301A, Data Acquisition.

[0263] S301A1, Data Source: The data sources for the data acquisition and feature processing module mainly include Kafka clusters in cloud environments (public cloud) and on-premises environments (private cloud).

[0264] The specific data collected includes: Timestamp data: Records the timestamp of each vehicle message generated, accurate to the millisecond level, used to analyze the temporal distribution characteristics of traffic.

[0265] Geographic data: This includes the geographical location of the vehicles, such as province, city, and region, used to analyze traffic differences in different areas.

[0266] Vehicle type data: Distinguish vehicle types, such as passenger cars, commercial vehicles, and new energy vehicles, to analyze the message generation patterns of different vehicle types.

[0267] Message content data: This includes the specific content of vehicle messages, such as vehicle speed, engine status, fault codes, etc., which are used to assist in traffic flow characteristic analysis.

[0268] S301A2, Data Acquisition Method.

[0269] Data acquisition is achieved through the following technologies: Kafka Consumer Group Subscription: Deploy consumer groups in Kafka clusters in the cloud and on-premises to subscribe to vehicle message data in real time and persist the data to the distributed storage system HDFS.

[0270] Scheduled batch retrieval: To avoid the impact of real-time data collection on system performance, a scheduled batch retrieval strategy is adopted, such as retrieving data every 5 minutes.

[0271] Data deduplication and verification: Duplicate data is deduplicated by message ID or timestamp, and data integrity is ensured by verification algorithm.

[0272] S301B, Feature Extraction.

[0273] Feature extraction includes time-dimensional features, geographical-dimensional features, vehicle-type-dimensional features, and traffic flow statistics features.

[0274] S301B1, Extract time dimension features.

[0275] Vehicle message traffic exhibits significant temporal correlation; therefore, the following temporal features need to be extracted: Hourly cycle: Analyze traffic changes at different times of the day, such as peak traffic during the morning rush hour (e.g., 7:00-9:00) and evening rush hour (e.g., 17:00-19:00).

[0276] Daily cycle: Analyze the traffic differences between weekdays and weekends, for example, traffic during weekday commuting hours is significantly higher than on weekends.

[0277] Seasonal cycles: Analyze traffic flow changes in different seasons, for example, vehicle breakdown reports may increase in winter.

[0278] Holiday characteristics: Mark statutory holidays, such as Spring Festival and National Day, and analyze traffic changes during holidays.

[0279] Special event characteristics: such as the impact of extreme weather like heavy rain or snow, or large-scale events like auto shows or sporting events on traffic flow.

[0280] S301B2, Extract regional dimensional features.

[0281] Vehicle density and driving habits vary significantly across different regions, necessitating the extraction of the following regional characteristics: Traffic distribution: Statistics on message traffic in different cities or regions, for example, traffic in first-tier cities is significantly higher than that in third- and fourth-tier cities.

[0282] Traffic characteristics: Analyze the traffic flow differences between congested and uncongested areas.

[0283] S301B3, Extract vehicle type dimension features.

[0284] Different types of vehicles have different message generation patterns, and the following features need to be extracted: Vehicle type distribution: Statistics on the percentage of messages from passenger cars, commercial vehicles, and new energy vehicles.

[0285] Vehicle status characteristics: Analyze the impact of vehicle operating status (such as driving, idling, and engine off) on message generation frequency.

[0286] S301B4. Extract traffic flow statistics features.

[0287] The following statistical characteristics were calculated based on the raw data: Mean and variance: Statistical analysis of hourly or daily traffic mean and variance to measure traffic stability.

[0288] Peak and Valley Values: Identify peak and valley values ​​in historical traffic to provide a reference for resource allocation.

[0289] Growth rate: Calculates the rate of change in traffic flow between adjacent time periods to predict future traffic trends.

[0290] S301C, Data Preprocessing.

[0291] Data preprocessing includes data cleaning, feature encoding, and feature standardization.

[0292] S301C1, Data Cleaning.

[0293] Missing value handling: For missing timestamps or geographic data, interpolation or default values ​​are used to fill them.

[0294] Outlier handling: Abnormal traffic data is identified using the Z-Score method and then smoothed or removed.

[0295] S301C2, Feature Coding.

[0296] Specifically, this includes categorical feature coding and temporal feature coding.

[0297] Among them, category feature encoding: One-Hot encoding is performed on category features such as region and vehicle type.

[0298] Among them, time feature encoding: converting timestamps into numerical features.

[0299] S301C3, Feature Standardization.

[0300] The encoded features are standardized to eliminate the influence of units.

[0301] S301D, Data Acquisition and Feature Processing Module Output: The final output of the data acquisition and feature extraction module is a structured feature dataset (i.e., the first feature dataset).

[0302] This structured feature dataset includes the following: Time Feature Table: Includes fields such as timestamp, hour, day of the week, and whether it is a holiday.

[0303] Regional characteristics table: includes fields such as region ID, region type (e.g., urban area, suburbs), and region traffic share.

[0304] Vehicle type feature table: contains fields such as vehicle type ID, vehicle status, and message generation frequency.

[0305] Traffic statistics table: Includes fields such as average traffic, peak traffic, and growth rate.

[0306] This output will serve as input to the prediction model, providing data support for subsequent dynamic resource allocation.

[0307] S302, Train the prediction model.

[0308] The training module is responsible for training the prediction model. As one of the core modules in this embodiment, its goal is to analyze historical vehicle message traffic using machine learning algorithms to build a high-precision prediction model. This module can capture the periodicity, trend, and burstiness characteristics of traffic and supports real-time adjustments to adapt to dynamically changing traffic patterns. Based on the prediction results, the system can optimize the resource configuration of the Kafka cluster in advance, avoiding resource waste or synchronization delays.

[0309] S302A, Historical Data Analysis.

[0310] Training a predictive model by analyzing historical data includes the following steps: S302A1, Prepare the data.

[0311] The data preparation work includes dividing the dataset and filtering the structured feature dataset.

[0312] S302A1a, Divide the dataset.

[0313] The structured data output by the feature extraction module is divided into a training set (e.g., 70%), a validation set (e.g., 15%), and a test set (e.g., 15%).

[0314] A time-series partitioning strategy is adopted to ensure the temporal continuity of the training and test sets and to avoid future data leakage.

[0315] S302A1b: Filter the structured feature dataset.

[0316] The second feature dataset is obtained by filtering the structured feature dataset.

[0317] Features that significantly impact traffic forecasting can be identified through Pearson correlation analysis and feature importance ranking based on tree models. For example: Time characteristics: hour, day of the week, whether it is a holiday.

[0318] Regional characteristics: regional traffic share, regional type.

[0319] Vehicle type characteristics: vehicle type distribution, vehicle status.

[0320] S302A2, Model Selection and Training.

[0321] S302A2a, Determine candidate models.

[0322] Based on the time-series characteristics of vehicle message traffic, the following machine learning models were selected for comparative experiments.

[0323] Traditional time series models: ARIMA and SARIMA.

[0324] Tree models: XGBoost and LightGBM, suitable for capturing non-linear feature interactions.

[0325] Deep learning models: LSTM and Transformer, suitable for handling long-term dependencies.

[0326] S302A2b, Model Training.

[0327] Parameter tuning: Model hyperparameters are tuned using grid search or Bayesian optimization. For example: The number of layers, number of hidden units, and learning rate for LSTM; the tree depth, learning rate, and subsampling ratio for XGBoost.

[0328] Loss function: The mean squared error (MSE) or mean absolute error (MAE) is used as the loss function.

[0329] S302A2c, Model Evaluation.

[0330] The performance of each candidate model was evaluated on the validation and test sets, with the main metrics as follows. The candidate model with the best overall performance was selected as the final prediction model.

[0331] Prediction accuracy: MSE, MAE, Fraction.

[0332] Real-time performance: Model inference takes time.

[0333] For a detailed description of the optimal overall performance, please refer to the aforementioned related expressions, which will not be repeated here.

[0334] As another optional embodiment, model ensemble can be performed, and the ensembled overall model can be used as the final prediction model, as can be referred to in step S302A2d below.

[0335] S302A2d, Model Integration.

[0336] To improve prediction robustness, a weighted average method can be used to integrate the models, which involves weighting and fusing the prediction results of multiple candidate models, with the weights dynamically adjusted based on the model performance.

[0337] S302B, real-time adjustment mechanism.

[0338] The real-time adjustment mechanism includes online learning and feedback optimization.

[0339] S302B1, Online Learning.

[0340] Incremental training: Regularly input new traffic data into the prediction model for incremental training to adapt to changes in traffic patterns.

[0341] Sliding window mechanism: Uses data from a fixed time window as the training set to ensure that the prediction model always learns the latest patterns.

[0342] S302B2, Feedback Optimization.

[0343] Prediction error monitoring: Calculates the error between the predicted value and the actual value in real time, and triggers model retraining when the error exceeds the threshold.

[0344] Abnormal traffic detection: Abnormal traffic is detected using the Isolation Forest or LOF (Local Outlier) algorithm, and the model parameters are dynamically adjusted.

[0345] S303, Dynamic resource allocation strategy.

[0346] The dynamic resource allocation strategy is executed by the dynamic resource allocation module. This module is the core execution module in this embodiment, and its goal is to adjust the resource configuration of the Kafka cluster in real time based on the prediction results output by the prediction model. This includes the number of Broker instances, the number of partitions, and the size of consumer groups, in order to cope with fluctuations in vehicle message traffic. This module ensures the efficiency and stability of cross-cloud data synchronization while avoiding resource waste through elastic scaling, priority management, and load balancing technologies.

[0347] S303A, Flexible Resource Allocation.

[0348] Elastic resource configuration includes dynamic scaling up and down of throughput nodes (Broker).

[0349] The Broker scaling conditions are as follows: (1) When the prediction result exceeds 80% of the current Broker processing capacity (i.e., exceeds the first processing capacity of cloud resources), the expansion operation is triggered to add Broker instances (i.e., increase throughput nodes).

[0350] (2) When the prediction result is lower than 30% of the current Broker processing capacity (i.e. lower than the second processing capacity of cloud resources), a scaling-down operation is triggered to reduce the number of Broker instances (i.e., reduce the number of throughput nodes).

[0351] The method to achieve Broker scaling is to dynamically adjust the number of Brokers by calling the cloud platform API.

[0352] For example, a blue-green deployment strategy can be adopted to ensure uninterrupted service during the scaling process.

[0353] S303B, Dynamic Partition Adjustment.

[0354] Dynamic partition adjustment includes partition expansion and partition shrinkage.

[0355] The triggering condition for dynamic adjustment of partitions is as follows: (1) When the vehicle packet traffic of a single partition (i.e. the first area) exceeds the first threshold, determine the increase in vehicle packet traffic and add a partition to the partition according to the increase in vehicle packet traffic.

[0356] You can add a new partition by calling the Kafka Admin API.

[0357] For example, the number of new partitions is allocated proportionally based on the predicted traffic (e.g., if the traffic increases by 20%, then 20% more partitions are added).

[0358] It is understandable that since vehicle message data is constantly being generated, if the consumer in the cloud resources cannot consume (i.e. process) the generated vehicle message data in a timely manner, information backlog will occur. Therefore, as another optional embodiment, if the message backlog in the first area exceeds the first preset backlog, it can be determined that the vehicle message traffic of a single partition (i.e., the first area) exceeds the first threshold.

[0359] (2) When the prediction results indicate that the vehicle message traffic is in a low period, merge the low-activity partitions to reduce resource consumption.

[0360] S303C, Consumer Group Management.

[0361] S303C1, Consumer instance scaling.

[0362] The triggering conditions for scaling up and down consumer instances are as follows: (1) When the prediction result is greater than the second threshold, increase the number of consumer instances.

[0363] (2) Reduce consumer instances when the prediction result is less than the second threshold and exceeds 10 minutes.

[0364] Understandably, since vehicle message data is constantly being generated, if the consumer in the cloud resources cannot consume (i.e. process) this generated vehicle message data in a timely manner, information backlog will occur. Therefore, as another optional implementation, consumer instances can be automatically added or removed based on message backlog (Lag).

[0365] That is, when Lag exceeds the second preset accumulation amount, the prediction result is determined to be greater than the second threshold, and a consumer instance is added. When Lag remains below the second preset accumulation amount for more than 10 minutes, the prediction result is determined to be less than the second threshold for more than 10 minutes, and a consumer instance is reduced.

[0366] The method for scaling up and down consumer instances is as follows: monitor Lag through the Kafka Consumer Group API and adjust the number of consumer instances in conjunction with container orchestration tools.

[0367] S303C2, Consumer Load Balancing.

[0368] When consumer instances are added or removed, a rebalance is triggered to ensure load balancing across consumers. Specifically, if a consumer instance is added, a new partition is created for the region where that consumer instance resides; if a consumer instance is removed, the region where that consumer instance resides is removed.

[0369] Kafka's StickyAssignor allocation strategy can be used to reduce the performance overhead caused by rebalancing.

[0370] S303D, Priority Management.

[0371] S303D1. First, prioritize the vehicle messages.

[0372] Vehicle messages are categorized into three priorities based on their importance: high, medium, and low. Fault alarm messages are considered high-priority messages (i.e., high-importance vehicle messages), while normal status messages are considered low-priority messages (i.e., low-importance vehicle messages). In other words, vehicle messages containing fault alarm information are considered high-importance vehicle messages, while vehicle messages containing normal status information are considered low-importance vehicle messages.

[0373] S303D2, Resource Allocation Strategy.

[0374] High-priority messages are allocated to exclusive brokers or partitions (i.e., independent areas) to ensure low-latency processing.

[0375] Low-priority messages are transmitted using batch compression to reduce resource consumption.

[0376] S303E, data synchronization optimization.

[0377] Data synchronization optimization aims to improve the efficiency and stability of data transmission between cross-cloud Kafka clusters through intelligent scheduling algorithms and load balancing technology. Based on traffic prediction results and real-time resource status, data synchronization optimization dynamically optimizes data transmission paths, compression strategies, and fault recovery mechanisms to ensure efficient synchronization of vehicle message data between cloud and on-premises environments.

[0378] S303E1, Intelligent Scheduling Algorithm.

[0379] The intelligent scheduling algorithm dynamically selects the optimal data transmission path by comprehensively analyzing network status, data traffic characteristics, and resource load.

[0380] First, the algorithm collects network latency, bandwidth utilization, and node load metrics between on-premises and cloud-based Kafka clusters in real time to build a multi-dimensional evaluation model.

[0381] Secondly, transmission paths are dynamically prioritized based on the evaluation results, with low-latency and high-availability paths being selected first. For example, if the network latency of a certain path exceeds a threshold, the system automatically switches to a backup path.

[0382] In addition, the algorithm supports differentiated scheduling strategies, allocating exclusive network channels for high-priority data to ensure real-time synchronization of critical messages.

[0383] The intelligent scheduling algorithm is designed as follows: Step 1: Assess the health of candidate transmission paths.

[0384] The formula for the health score of each candidate transmission path can be referred to the aforementioned formula (1), and will not be repeated here.

[0385] Step 2: Dynamic weight adjustment.

[0386] Adjusting weights based on real-time traffic prediction results, for example in the following scenarios: Increase latency weight during peak traffic periods. Reduce bandwidth weight from 0.4 to 0.5. When network congestion occurs, the load weight is increased. This triggers a path switch.

[0387] Step 3: Path selection.

[0388] Primary path selection: Select the candidate path with the highest health score as the primary transmission path.

[0389] Backup path: Select the path with the second highest score as a backup. Automatically switch when the round trip time (RTT) of the main path (also known as network latency) is >50ms (i.e., the preset network latency) or the packet loss rate is >5% (i.e., the preset packet loss rate).

[0390] Step 4: Feedback and optimization.

[0391] Short-term feedback: The health scores of the target path and other candidate paths are updated every 5 minutes, and the actual transmission delay and prediction deviation are recorded.

[0392] Long-term learning: Using reinforcement learning to optimize weights , and To minimize the historical average latency.

[0393] S303E2, Data Compression and Batch Processing.

[0394] To reduce network transmission overhead, adaptive data compression technology is also introduced to match the target compression algorithm with vehicle message data of different importance levels.

[0395] For low-priority vehicle message data, a high compression ratio algorithm (e.g., Zstantdrd compression algorithm) is used to reduce the amount of data transmitted; for high-priority vehicle message data, a low-latency compression algorithm (e.g., Snappy compression algorithm) is selected to balance efficiency and real-time performance.

[0396] Simultaneously, an intelligent batch processing mechanism can be implemented to merge small messages into batch messages before transmission. The batch processing window size is dynamically adjusted based on predicted traffic: the window is reduced during peak traffic periods to decrease latency, and expanded during off-peak periods to increase throughput.

[0397] S303E3, Fault Recovery and Consistency Assurance.

[0398] The module incorporates a multi-tiered fault recovery mechanism to handle network outages or cluster anomalies. When a synchronization failure is detected, the system first attempts to retry automatically, gradually increasing the retry interval to avoid a cascading failure. If the retry fails, a local caching mechanism is triggered, temporarily storing the data in a disk queue until the network is restored and resynchronization occurs.

[0399] To ensure data consistency, the module implements exactly-Once semantics based on the Kafka transaction mechanism and prevents data loss or duplication through periodic offset verification. Furthermore, the system records synchronization status checkpoints, supporting rapid recovery from breakpoints and reducing the resource consumption associated with full synchronization.

[0400] S303E4, Load balancing technology.

[0401] Load balancing technology avoids overloading individual nodes by dynamically allocating synchronization tasks. The module deploys a distributed load balancer between cloud and on-premises clusters, monitoring message backlog and processing latency on each broker in real time. Based on monitoring data, a weighted round-robin algorithm distributes synchronization tasks to lightly loaded nodes while implementing rate limiting protection for high-load nodes. To address partition-level load imbalance, the module periodically analyzes the message distribution across partitions and automatically adjusts the mapping between partitions and brokers using Kafka's partition reallocation tool, maximizing resource utilization.

[0402] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0403] Based on the same inventive concept as the foregoing embodiments, this application provides a cloud resource processing device and a prediction model training device. These two devices include the included modules and the units included in each module, which can be implemented by a processor; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be an AI acceleration engine (such as NPU), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0404] Figure 4 This is a schematic diagram of the structure of a cloud resource processing device provided in an embodiment of this application, as shown below. Figure 4 As shown, the cloud resource processing device 40 includes a first determining unit 401 and a first adjusting unit 402.

[0405] Wherein: the first determining unit 401 is configured to determine time characteristics, regional characteristics and vehicle type characteristics based on the time information, regional information and vehicle type information to be processed by cloud resources.

[0406] The first adjustment unit 402 is configured to process time features, regional features and vehicle type features through a prediction model, predict the vehicle message traffic to be input to the cloud device, and obtain the prediction result; adjust the configuration of cloud resources based on the prediction result so that the configuration of cloud resources matches the prediction result.

[0407] The prediction results are used to characterize the current vehicle message traffic volume to be input to the cloud device.

[0408] In some embodiments, the first adjustment unit 402 is further configured to: increase throughput nodes in the cloud resource when the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, exceeds the first processing capacity of the cloud resource; and decrease throughput nodes in the cloud resource when the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is lower than the second processing capacity of the cloud resource.

[0409] The first processing capacity is higher than the second processing capacity.

[0410] In some embodiments, the first adjustment unit 402 is further configured to: increase the number of consumer instances in the cloud resources when the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is greater than a second threshold; and decrease the number of consumer instances in the cloud resources when the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, meets a first preset condition.

[0411] The first preset condition includes: the current vehicle message traffic to be input to the cloud device as represented by the prediction result is less than the third threshold, or the current vehicle message traffic to be input to the cloud device as represented by the prediction result is less than the third threshold and exceeds the specified time.

[0412] The more consumer instances there are, the faster the cloud resources can process vehicle packet traffic; the fewer consumer instances there are, the slower the cloud resources can process vehicle packet traffic.

[0413] The third threshold is less than or equal to the second threshold.

[0414] In some embodiments, the first adjustment unit 402 is further configured to: determine the increase in the first vehicle message traffic when the prediction result indicates that the first vehicle message traffic in the current vehicle message traffic to be input to the cloud device exceeds a first threshold; and add a partition to the first area in the cloud resource corresponding to the processing of the first vehicle message traffic according to the increase in the first vehicle message traffic, so that the first area after adding the partition can process the first vehicle message traffic.

[0415] In some embodiments, the first adjustment unit 402 is further configured to: based on the current vehicle packet traffic to be input to the cloud device characterized by the prediction result, and if it is determined that the current vehicle packet traffic belongs to a low period of vehicle packet traffic: based on the priority of the vehicle packet traffic being processed in the cloud resources, determine at least two low-activity areas in the cloud resources; determine a target area and an idle area in the at least two low-activity areas; the target area is used to process the current vehicle packet traffic and the vehicle packet traffic being processed in the two low-activity areas; the idle area serves as a backup area.

[0416] In some embodiments, the first adjustment unit 402 is further configured to: divide the current vehicle packet traffic to be input to the cloud device according to the importance of the vehicle packets' services, and determine the importance level of each vehicle packet; determine the area for processing high-importance vehicle packets as an independent area in the cloud resources, or determine the throughput node for processing high-importance vehicle packets as an independent throughput node; high-importance vehicle packets are vehicle packets including fault alarm information; low-importance vehicle packets are compressed before transmission; low-importance vehicle packets are vehicle packets including regular status information.

[0417] In some embodiments, the cloud resource processing apparatus provided in this application further includes a first processing unit 403 (not shown), a second determining unit 404 (not shown), and a first compression and transmission unit 405 (not shown).

[0418] The first processing unit 403 is configured to process the prediction results based on cloud resources to characterize the current vehicle message traffic to be input to the cloud device, and obtain the vehicle message traffic to be sent.

[0419] The second determining unit 404 is configured to: determine the target compression algorithm corresponding to vehicle messages of different importance levels based on the importance level of each vehicle message in the vehicle message traffic to be sent.

[0420] The first compression transmission unit 405 is configured to transmit vehicle messages of different importance levels after compression by the corresponding target compression algorithm.

[0421] Specifically, for high-priority vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm is a low-latency compression algorithm; high-priority vehicle packets are those that include fault alarm information; for low-priority vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm is a high compression ratio algorithm; low-priority vehicle packets are those that include regular status information.

[0422] In some embodiments, the first compression transmission unit 405 is further configured to: determine the health scores of at least two candidate transmission paths for the vehicle message traffic to be sent; select the transmission path with the highest health score as the target transmission path from the at least two candidate transmission paths; and transmit the compressed vehicle messages of different importance levels using the target transmission path.

[0423] Among them, the health of the transmission path is used to evaluate the network latency, network bandwidth, and network load of the transmission path.

[0424] In some embodiments, the first compression transmission unit 405 is further configured to: for each of at least two candidate transmission paths, perform: determining the network latency, network bandwidth, and network load of the candidate transmission path; and determining a health score of the candidate transmission path based on the network latency, network bandwidth, and network load of the candidate transmission path.

[0425] In some embodiments, the cloud resource processing apparatus provided in this application further includes a first switching unit 406 (not shown).

[0426] The first switching unit 406 is configured to switch to the backup transmission path when the target transmission path is abnormal, so as to transmit compressed vehicle messages of different importance levels through the backup transmission path.

[0427] Among them, the backup target transmission path is the candidate transmission path with the second highest health score.

[0428] Figure 5 A training apparatus for a prediction model provided in this application embodiment, such as Figure 5 As shown, the training device 50 for the prediction model includes a third determining unit 501 and a first training unit 502.

[0429] The third determining unit 501 is configured to determine time features, regional features, and vehicle type features based on the time information, regional information, and vehicle type information of historical vehicle message traffic, forming a first feature dataset; different vehicle types correspond to different vehicle message traffic; different regions correspond to different vehicle message traffic; and different times correspond to different vehicle message traffic. The first training unit 502 is configured to train a candidate model using a first feature dataset to obtain a prediction model; so as to dynamically adjust the configuration of cloud resources according to the prediction results of the prediction model; the prediction results are used to characterize the current vehicle message traffic to be input to the cloud device.

[0430] In some embodiments, when there are multiple candidate models, the first training unit 502 is further configured to: filter the first feature dataset using a correlation analysis algorithm and / or a feature importance evaluation algorithm to obtain a second feature dataset; train each of the multiple candidate models using the second feature dataset; evaluate the prediction accuracy and real-time performance of each trained candidate model, and select the candidate model that meets the preset evaluation conditions as the prediction model.

[0431] Alternatively, the first training unit 502 is further configured to: evaluate the prediction accuracy and real-time performance of each trained candidate model, and determine the prediction model based on each trained candidate model and the weights corresponding to each candidate model; the weights corresponding to each candidate model are related to the prediction accuracy and real-time performance of each candidate model.

[0432] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0433] It should be noted that the module division in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware, as software functional units, or a combination of software and hardware.

[0434] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0435] This application provides an electronic device. Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, the electronic device 60 includes a memory 601 and a processor 602. The memory 601 stores a computer program that can run on the processor 602. When the processor 602 executes the program, it implements the steps in the method provided in the above embodiments.

[0436] It should be noted that the memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the various modules of the processor 602 and the electronic device 60. It can be implemented by flash memory or random access memory (RAM).

[0437] This application also provides a computer-readable storage medium for storing computer programs.

[0438] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of this application, and the computer program causes the processor or electronic device to perform the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0439] This application also provides a computer program product, including computer program instructions.

[0440] Optionally, the computer program product can be applied to the electronic device in the embodiments of this application, and the computer program instructions cause the processor or electronic device to execute the various methods in the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0441] This application also provides a computer program.

[0442] Optionally, the computer program can be applied to the electronic device in the embodiments of this application. When the computer program runs on the processor or electronic device, it causes the processor or electronic device to execute the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0443] It should be noted that the descriptions of the electronic devices, storage media, computer program products, and computer program embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the electronic devices, storage media, computer program products, and computer program embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0444] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0445] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0446] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0447] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0448] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0449] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0450] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0451] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0452] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0453] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0454] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0455] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for processing cloud resources, characterized in that, The method is applied to a cloud device, and the method includes: Based on the time information, geographical information, and vehicle type information of the cloud resources to be processed, determine the time characteristics, geographical characteristics, and vehicle type characteristics. The time features, geographical features, and vehicle type features are processed by a prediction model to predict the vehicle message traffic to be input to the cloud device, and a prediction result is obtained; the prediction result is used to characterize the current vehicle message traffic to be input to the cloud device. The configuration of the cloud resources is adjusted based on the prediction results to match the configuration of the cloud resources with the prediction results. The step of adjusting the configuration of the cloud resources based on the prediction results includes: If the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is greater than the second threshold, the number of consumer instances in the cloud resources will be increased. If the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, meets the first preset condition, the number of consumer instances in the cloud resources is reduced. The first preset condition includes: the current vehicle message traffic to be input to the cloud device as represented by the prediction result is less than the third threshold, or the current vehicle message traffic to be input to the cloud device as represented by the prediction result is less than the third threshold and exceeds the specified time. The more consumer instances there are, the faster the cloud resources can process the vehicle packet traffic; the fewer consumer instances there are, the slower the cloud resources can process the vehicle packet traffic. The third threshold is less than or equal to the second threshold.

2. The method according to claim 1, characterized in that, Adjusting the configuration of the cloud resources based on the prediction results includes: If the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, exceeds the first processing capacity of the cloud resource, a throughput node is added to the cloud resource. If the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is lower than the second processing capacity of the cloud resource, the throughput nodes in the cloud resource are reduced. The first processing capability is higher than the second processing capability.

3. The method according to claim 1, characterized in that, Adjusting the configuration of the cloud resources based on the prediction results includes: If the prediction result indicates that the first vehicle message traffic in the current vehicle message traffic to be input to the cloud device exceeds a first threshold, the increase in the first vehicle message traffic is determined. Based on the increase in the first vehicle packet traffic, a partition is added to the first area in the cloud resource corresponding to the processing of the first vehicle packet traffic, so that the first area after the partition is added can process the first vehicle packet traffic.

4. The method according to claim 1, characterized in that, Adjusting the configuration of the cloud resources based on the prediction results includes: Based on the prediction results representing the current vehicle packet traffic to be input to the cloud device, and determining that the current vehicle packet traffic belongs to a low period of vehicle packet traffic: Based on the priority of the vehicle packet traffic being processed in the cloud resources, at least two low-activity areas in the cloud resources are identified. A target area and an idle area are determined from the at least two low-activity areas; the target area is used to process the current vehicle packet traffic and the vehicle packet traffic being processed in the two low-activity areas; the idle area serves as a backup area.

5. The method according to claim 1, characterized in that, Adjusting the configuration of the cloud resources based on the prediction results includes: Based on the importance of the vehicle messages, the current vehicle message traffic to be input to the cloud device, as represented by the prediction results, is divided into each vehicle message according to the importance of the service; and the importance level of each vehicle message is determined. The region for processing high-priority vehicle messages is determined to be an independent region within the cloud resources, or the throughput node for processing the high-priority vehicle messages is determined to be an independent throughput node; the high-priority vehicle messages are vehicle messages that include fault alarm information. Vehicle messages of low importance are compressed before transmission; the low importance vehicle messages are vehicle messages that include routine status information.

6. The method according to any one of claims 1-5, characterized in that, After adjusting the configuration of the cloud resources based on the prediction results to match the prediction results, the method further includes: Based on the cloud resources, the prediction results are processed to characterize the current vehicle message traffic to be input to the cloud device, so as to obtain the vehicle message traffic to be sent. Based on the importance level of each vehicle message in the vehicle message traffic to be sent, the target compression algorithm corresponding to vehicle messages of different importance levels is determined. Vehicle messages of different importance levels are compressed using the corresponding target compression algorithm before transmission; Specifically, for high-priority vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm is a low-latency compression algorithm; the high-priority vehicle packets are vehicle packets including fault alarm information; for low-priority vehicle packets in the vehicle packet traffic to be sent, the target compression algorithm is a high compression ratio algorithm; the low-priority vehicle packets are vehicle packets including regular status information.

7. The method according to claim 6, characterized in that, The vehicle messages of different importance levels are compressed using the corresponding target compression algorithm before transmission, including: Determine the health scores of at least two candidate transmission paths for the vehicle message traffic to be sent. Among the at least two candidate transmission paths, the transmission path with the highest health score is selected as the target transmission path. Compressed vehicle messages of different importance levels are transmitted using the target transmission path; The health of the transmission path is used to evaluate the network latency, network bandwidth, and network load of the transmission path.

8. The method according to claim 7, characterized in that, The health scores for at least two candidate transmission paths used to determine the vehicle message traffic to be sent include: For each of the at least two candidate transmission paths, perform: Determine the network latency, network bandwidth, and network load of the candidate transmission path; The health score of the candidate transmission path is determined based on the network latency, network bandwidth, and network load of the candidate transmission path.

9. The method according to claim 7, characterized in that, The method further includes: In the event of an anomaly in the target transmission path, the system switches to an alternate transmission path to transmit compressed vehicle messages of different importance levels through the alternate transmission path. The backup target transmission path is the candidate transmission path with the second highest health score.

10. A cloud resource processing device, characterized in that, The device is deployed on a cloud device, and the device includes: a first determining unit and a first adjusting unit; The first determining unit is configured to determine time characteristics, regional characteristics, and vehicle type characteristics based on the time information, regional information, and vehicle type information of the cloud resources to be processed. The first adjustment unit is configured to process the time features, the geographical features, and the vehicle type features using a prediction model to predict the vehicle message traffic to be input to the cloud device and obtain a prediction result; and to adjust the configuration of the cloud resources based on the prediction result so that the configuration of the cloud resources matches the prediction result. The prediction results are used to characterize the current vehicle message traffic volume to be input to the cloud device; The first adjustment unit is further configured to increase the number of consumer instances in the cloud resources if the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, is greater than a second threshold. If the current vehicle message traffic to be input to the cloud device, as represented by the prediction result, meets the first preset condition, the number of consumer instances in the cloud resources is reduced. The first preset condition includes: the current vehicle message traffic to be input to the cloud device as represented by the prediction result is less than the third threshold, or the current vehicle message traffic to be input to the cloud device as represented by the prediction result is less than the third threshold and exceeds the specified time. The more consumer instances there are, the faster the cloud resources can process the vehicle packet traffic; the fewer consumer instances there are, the slower the cloud resources can process the vehicle packet traffic. The third threshold is less than or equal to the second threshold.

11. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores a computer program or instructions, and when the computer program or instructions are executed by the processor, the method described in any one of claims 1-9 is implemented.

Images