A new energy vehicle data forwarding method based on distributed architecture

Abstract

The present invention discloses a new energy vehicle data forwarding method based on a distributed architecture, comprising the following steps: a. Containerized deployment: selecting a data processing container: first, determining to use a data processing container as a platform for container orchestration and management; b. Building a container image: building a Docker container image for each data forwarding service, ensuring that the image contains all applications and configurations; c. Deploying data: deploying the built container image into the data processing container, and utilizing the automated deployment, expansion, and management functions of the data processing container to implement service deployment and operation. The present invention implements containerized deployment of a distributed processing architecture by using a data processing container (data processing container), splitting a centralized data forwarding service into multiple forwarding services, and significantly improving data processing capabilities and transmission efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of new energy vehicle data processing, and in particular to a new energy vehicle data forwarding method based on a distributed architecture. Background Art

[0002] With the rapid development of the new energy vehicle industry, vehicle data regulation faces new challenges. The simple data forwarding service initially employed—that is, a single data forwarding service that centrally forwarded vehicle data to national and local regulatory platforms—has gradually exposed its shortcomings. With the dramatic increase in the number of vehicles and increasingly stringent local regulatory requirements, this centralized data forwarding model is unable to effectively cope with the demands of large-scale, real-time data transmission. This is especially true when vehicle data is generated in real time and frequently uploaded to enterprise monitoring platforms, which are then forwarded in real time to national and multiple local platforms in accordance with regulatory requirements. The processing capabilities of a single data forwarding service are insufficient. A centralized forwarding service must process data from all vehicles, which can lead to insufficient data processing capacity, especially in situations where data volume surges. For example, in the event of a major traffic accident or emergency, a large amount of data can flood the forwarding service in a short period of time, leading to data processing delays and impacting timeliness and accuracy. Furthermore, under the centralized forwarding model, data must pass through multiple links from vehicle to national and local platforms, each of which can introduce latency. This accumulated latency can lead to inefficient data transmission and make it unsuitable for applications with demanding real-time performance. Summary of the Invention

[0003] In order to overcome the above technical problems, the purpose of the present invention is to provide a new energy vehicle data forwarding method based on a distributed architecture.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A new energy vehicle data forwarding method based on a distributed architecture comprises the following steps:

[0006] Step 1: Containerized deployment

[0007] a. Select a data processing container: First, decide to use a data processing container as the platform for container orchestration and management.

[0008] b. Build container images: Build Docker container images for each data forwarding service, ensuring that the images contain all applications and configurations.

[0009] c. Deploy data: Deploy the built container image to the data processing container, and use the data processing container's automated deployment, expansion, and management capabilities to deploy and run the service.

[0010] Step 2: Split forwarding service

[0011] a. Design service architecture: Based on actual requirements, design a distributed architecture that includes national platform forwarding services, multiple local platform forwarding services, and data re-issuance services.

[0012] b. Implement service splitting: Split the original single centralized forwarding service into multiple targeted services and deploy them separately into data processing containers.

[0013] c. Resource allocation: Allocate system resources based on the amount of vehicle data and pressure to be processed by each service.

[0014] Step 3: Dynamic Scaling and Load Balancing

[0015] a. Configure automatic scaling: Use the automatic scaling feature of the data processing container to dynamically increase or decrease the number of forwarding service instances based on traffic and resource usage.

[0016] b. Implement load balancing: Configure a load balancing strategy within each service to ensure load balancing for each service instance and avoid single point overload.

[0017] Step 4: Monitoring and log management

[0018] a. Integrated monitoring system: Integrate a monitoring system into the data processing container to monitor the performance and resource usage of each forwarding service in real time.

[0019] b. Implement log aggregation and analysis: Use log aggregation and analysis tools to collect, store, and analyze log data from forwarding services.

[0020] Step 5: Continuous Optimization

[0021] a. Performance tuning: Regularly evaluate and optimize the performance of each forwarding service based on monitoring and log data.

[0022] b. Service Upgrade: Continuously update and optimize the functions and performance of the forwarding service based on the development of the new energy vehicle industry and changes in regulatory requirements.

[0023] As a further solution of the present invention: in the step one, a data processing container is selected as the container orchestration platform, and a data processing container distribution is selected based on business requirements and technology stack characteristics, including GKE, EKS or self-built, and initial configuration is performed, which includes node planning, network policy setting, storage solution configuration, and security policy customization based on RBAC (role-based access control). A Docker image repository is selected to store and manage these images, including Docker Hub, Harbor, or a self-built private repository to meet specific security requirements. At the same time, a CI / CD pipeline is constructed, and Jenkins and GitLabCI automation tools are used to realize the automation of image building, standardization of testing processes, and automatic push of images. The data processing containers include but are not limited to cloud service platforms, virtual machines, and Kubernetes (K8S) platforms.

[0024] As a further solution of the present invention: in the step 2, according to the business logic and data flow, the responsibility boundaries of each forwarding service are clarified. The national platform forwarding service handles the data transmission to the national regulatory platform, while the local platform forwarding service is responsible for distributing data to each local regulatory platform. Through the Horizontal Pod Autoscaler (HPA) function of the data processing container, the number of Pods is dynamically adjusted according to the load of the service to cope with traffic fluctuations. The load balancing strategy is configured and the load balancing function of Nginx Ingress Controller or Istio is used to ensure the uniform distribution of traffic between services and avoid single point overload.

[0025] As a further solution of the present invention: in the step three, the implementation of dynamic expansion depends on the Horizontal Pod Autoscaler (HPA) function of the data processing container. HPA monitors the specified resource indicators and automatically adjusts the number of Pod copies according to preset thresholds and policies. When the system traffic increases, causing the resource utilization rate to rise and exceed the set threshold, HPA triggers the Pod expansion operation and increases service instances to cope with the traffic pressure; conversely, when the traffic decreases and the resource utilization rate drops, HPA reduces the service instance to release resources. This is achieved by combining the Service resources of the data processing container and the external load balancer. In the data processing container, the Service resources define the access method and load balancing strategy of the Pod. By configuring the Service type to LoadBalancer and specifying the parameters of the external load balancer, automatic traffic distribution and load balancing are achieved.

[0026] As a further solution of the present invention: in the step 4, the Prometheus and Grafana monitoring components provided by the data processing container are used to realize real-time monitoring and visualization of resources. By configuring the collection rules of Prometheus and the dashboard of Grafana, the usage of system resources, the status of service instances and performance indicators, including request response time and throughput, are viewed. A centralized log collection and processing solution is adopted, including the use of ELKStack. By configuring Logstash or Fluentd log collection tools, the log data scattered in each Pod is uniformly collected into the Elasticsearch log storage system, thereby realizing centralized storage, retrieval and analysis of logs.

[0027] As a further solution of the present invention: in step five, by configuring RBAC, defining roles and role bindings, and assigning different permissions to different users or services, fine-grained access control is achieved. In addition, the Secrets resources of the data processing container are utilized to securely store and manage sensitive information, including database passwords and API keys, to establish an identity authentication and authorization mechanism. For user access, multi-factor authentication technology is adopted, including combining multiple authentication methods such as username and password, SMS verification code, and biometrics to improve account security. For communication between services, two-way TLS authentication technology is used to ensure that communication between services is encrypted and authenticated to prevent man-in-the-middle attacks.

[0028] Beneficial effects of the present invention:

[0029] 1. By using data processing containers (DPCs), we achieve containerized deployment of a distributed processing architecture. Splitting a centralized data forwarding service into multiple forwarding services significantly improves data processing capabilities and transmission efficiency. Each forwarding service handles a portion of all forwarded data, reducing the processing pressure on a single node and thereby improving the overall system's responsiveness and processing capabilities.

[0030] 2. By adopting a multi-node design and load balancing mechanism, the distributed processing architecture effectively reduces the risk of single points of failure. Even if a node fails, other nodes can continue to operate normally, ensuring the continuity of data processing tasks. In addition, data is redundantly stored across multiple nodes, enhancing data security and reliability.

[0031] 3. The distributed processing architecture adopts a modular design, enabling it to flexibly adapt to the growth of data volume and business needs. Nodes can be easily added or removed as needed without requiring large-scale redesign or recoding of the entire system. This high degree of scalability enables the system to respond to changing business needs and technological developments.

[0032] 4. By splitting the system into multiple independent nodes, the distributed processing architecture greatly simplifies system maintenance. Each node is relatively independent, making it easier to locate problems when they occur.

[0033] 5. The modular service-oriented split design enables horizontal expansion. As vehicle data continues to grow, servers with conventional hardware resources can be used, significantly reducing hardware and operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention will be further described below with reference to the accompanying drawings.

[0035] Figure 1 It is a schematic diagram of the method of the present invention;

[0036] Figure 2 This is a diagram of the data forwarding process. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] include Figure 1-2 As shown, a new energy vehicle data forwarding method based on a distributed architecture includes the following steps:

[0039] Step 1: Containerized deployment

[0040] a. Select a data processing container: First, decide to use a data processing container as the platform for container orchestration and management.

[0041] b. Build container images: Build Docker container images for each data forwarding service, ensuring that the images contain all applications and configurations.

[0042] c. Deploy data: Deploy the constructed container image into the data processing container, and use the automated deployment, expansion, and management functions of the data processing container to implement service deployment and operation. In step 1, select the data processing container as the container orchestration platform. Based on business requirements and technology stack characteristics, select the data processing container distribution, including GKE, EKS, or self-built, and perform initial configuration, which includes node planning, network policy setting, storage solution configuration, and security policy customization based on RBAC (role-based access control). Select Docker image repository to store and manage these images, including Docker Hub, Harbor, or self-built private repository to meet specific security requirements. At the same time, build a CI / CD pipeline and use Jenkins and GitLab CI automation tools to realize the automation of image building, standardization of testing processes, and automatic push of images. Data processing containers include but are not limited to cloud service platforms, virtual machines, and Kubernetes (K8S) platforms.

[0043] Step 2: Split forwarding service

[0044] a. Design service architecture: Based on actual requirements, design a distributed architecture that includes national platform forwarding services, multiple local platform forwarding services, and data re-issuance services.

[0045] b. Implement service splitting: Split the original single centralized forwarding service into multiple targeted services and deploy them separately into data processing containers.

[0046] c. Resource allocation: Allocate system resources based on the amount of vehicle data and pressure to be processed by each service. In step 2, the system clarifies the responsibilities of each forwarding service based on business logic and data flow. The national platform forwarding service focuses on data transmission to the national regulatory platform, while the local platform forwarding service is responsible for data distribution to local regulatory platforms. In order to meet the data forwarding needs of specific brands or models, the system further refines the responsibilities of the local platform forwarding service. At the same time, the data reissue business is separated to form a dedicated data reissue service. However, it is worth noting that the data reissue service itself does not directly perform data reissue operations. In actual operation, after completing data forwarding, the national and local platform forwarding services will write the results back to the database. The data reissue service will re-extract the data to be reissued from the database according to preset rules (for example, at 1:00 a.m. every day for 7 consecutive days) and place this data in the queue of each forwarding service. In this way, the original forwarding service will continue to forward the data based on the data in the queue to ensure data integrity and timeliness. In order to cope with traffic fluctuations, the system uses the Horizontal The PodAutoscaler (HPA) function dynamically adjusts the number of Pods based on the service load. In addition, the system is configured with a load balancing strategy, using the Nginx Ingress Controller or Istio's load balancing function to ensure that traffic between services is evenly distributed, thereby avoiding the problem of single point overload. This architectural design not only improves the flexibility and scalability of the system, but also ensures the accuracy and efficiency of data forwarding.

[0047] Step 3: Dynamic Scaling and Load Balancing

[0048] a. Configure automatic scaling: Use the automatic scaling feature of the data processing container to dynamically increase or decrease the number of forwarding service instances based on traffic and resource usage.

[0049] b. Implement load balancing: Configure a load balancing strategy within each service to ensure load balancing for each service instance and avoid single-point overload. In step three, the implementation of dynamic expansion depends on the Horizontal Pod Autoscaler (HPA) function of the data processing container. HPA monitors specified resource indicators and automatically adjusts the number of Pod replicas based on preset thresholds and strategies. When system traffic increases, causing resource utilization to rise and exceed the set threshold, HPA triggers Pod expansion operations and increases service instances to cope with traffic pressure; conversely, when traffic decreases and resource utilization decreases, HPA reduces service instances to release resources. This is achieved by combining the Service resources of the data processing container and an external load balancer. In the data processing container, the Service resource defines the access method and load balancing strategy of the Pod. By configuring the Service type to LoadBalancer and specifying the parameters of the external load balancer, automatic traffic distribution and load balancing are achieved.

[0050] Step 4: Monitoring and log management

[0051] a. Integrated monitoring system: Integrate a monitoring system into the data processing container to monitor the performance and resource usage of each forwarding service in real time.

[0052] b. Implement log aggregation and analysis: Use log aggregation and analysis tools to collect, store, and analyze the log data of the forwarding service. In step 4, use the Prometheus and Grafana monitoring components that come with the data processing container to achieve real-time monitoring and visualization of resources. By configuring Prometheus collection rules and Grafana dashboards, view system resource usage, service instance status, and performance indicators, including request response time and throughput. Use a centralized log collection and processing solution, including using ELKStack. By configuring Logstash or Fluentd log collection tools, log data scattered across various pods is uniformly collected into the Elasticsearch log storage system for centralized log storage, retrieval, and analysis.

[0053] Step 5: Continuous Optimization

[0054] a. Performance tuning: Regularly evaluate and optimize the performance of each forwarding service based on monitoring and log data.

[0055] b. Service upgrade: Based on the development of the new energy vehicle industry and changes in regulatory requirements, the functions and performance of the forwarding service are continuously updated and optimized. In step five, by configuring RBAC, defining roles and role bindings, and assigning different permissions to different users or services, fine-grained access control is achieved. In addition, the Secrets resource of the data processing container is used to securely store and manage sensitive information, including database passwords and API keys, and establish identity authentication and authorization mechanisms. For user access, multi-factor authentication technology is adopted, including combining multiple authentication methods such as username and password, SMS verification code, and biometrics to improve account security. For communication between services, two-way TLS authentication technology is used to ensure that communication between services is encrypted and authenticated to prevent man-in-the-middle attacks.

[0056] Working principle of the present invention:

[0057] The distributed architecture-based data forwarding method for new energy vehicles works by first selecting a data processing container (DPC) as the core platform for container orchestration and management. Based on this, Docker container images are built for each data forwarding service. These images encapsulate the required applications and their configurations, ensuring service consistency and portability. These built container images are then deployed into the DPC. Leveraging the automated deployment, scaling, and management capabilities provided by the DPC, this approach enables rapid deployment and efficient operation of data forwarding services.

[0058] During the service deployment phase, this approach designed a distributed architecture based on actual needs, encompassing a national platform forwarding service, multiple local platform forwarding services, and a data re-issuance service. Through a service splitting strategy, the single centralized forwarding service was broken down into multiple, more targeted service components, each deployed on different nodes of the data processing container. System resources were allocated based on the volume of vehicle data processed by each service and the pressure it faced, achieving optimal resource allocation and efficient utilization.

[0059] To achieve elastic service expansion and load balancing, this approach configures the data processing container's automatic scaling feature. This dynamically adjusts the number of forwarding service instances based on real-time changes in traffic and resource usage, ensuring the system maintains stable operation under varying loads. Furthermore, within each service, load balancing strategies such as round-robin and least connections are configured to achieve balanced traffic distribution across service instances, effectively preventing single point overloads.

[0060] To ensure stable system operation and timely problem detection and resolution, this approach also integrates a monitoring system to monitor the performance and resource usage of each forwarding service in real time within the data processing container. Furthermore, log aggregation and analysis tools are used to collect, store, and conduct in-depth analysis of forwarding service log data, providing robust data support for system operations and maintenance.

[0061] Finally, this approach emphasizes the importance of continuous optimization. By regularly evaluating and optimizing the performance of each forwarding service, and continuously updating and optimizing the forwarding service's functionality and performance based on the development of the new energy vehicle industry and evolving regulatory requirements, this approach ensures the system's continued growth and competitiveness. In summary, through key elements such as containerized deployment, service splitting, dynamic scaling, load balancing, monitoring and log management, and continuous optimization, this approach achieves efficient, stable, and scalable operation of the new energy vehicle data forwarding system.

[0062] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of the present invention. All changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A new energy vehicle data forwarding method based on a distributed architecture, characterized in that: The following steps are involved: Step 1: Containerized deployment a. Select a data processing container: First, determine whether to use a data processing container as the platform for container orchestration and management. b. Build container images: Build Docker container images for each data forwarding service, ensuring that the images contain applications and configurations. c. Deploy data: Deploy the built container image to the data processing container and use the data processing container's automated deployment, expansion, and management capabilities to deploy and run the service. Step 2: Split forwarding service a. Design service architecture: Based on actual needs, design a distributed architecture that includes national platform forwarding services, multiple local platform forwarding services, and data re-issuance services; b. Implement service splitting: Split the original single centralized forwarding service into multiple targeted services and deploy them separately into data processing containers; c. Resource allocation: Allocate system resources based on the amount of vehicle data and pressure to be processed by each service; Step 3: Dynamic Scaling and Load Balancing a. Configure automatic scaling: Use the automatic scaling feature of the data processing container to dynamically increase or decrease the number of forwarding service instances based on traffic and resource usage; b. Implement load balancing: Configure a load balancing strategy within each service to ensure load balancing for each service instance and avoid single point overload. Step 4: Monitoring and log management a. Integrated monitoring system: Integrate a monitoring system into the data processing container to monitor the performance and resource usage of each forwarding service in real time; b. Implement log aggregation and analysis: Use log aggregation and analysis tools to collect, store, and analyze log data from forwarding services; Step 5: Continuous Optimization a. Performance tuning: Regularly evaluate and optimize the performance of each forwarding service based on monitoring and log data; b. Service Upgrade: Continuously update and optimize the functionality and performance of the forwarding service based on the development of the new energy vehicle industry and changes in regulatory requirements; The NoSQL database stores vehicle data and forwarding results, the relational database stores forwarding configuration rules, and the data queue stores vehicle data; After completing data forwarding, the national and local platform forwarding services will write the results back to the NoSQL database. The data reissue service will re-extract the data that needs to be reissued from the NoSQL database according to preset rules and place this data in the queue of each forwarding service. In this way, the original forwarding service will continue to forward based on the vehicle data in the data queue.

2. The method for forwarding data of a new energy vehicle based on a distributed architecture according to claim 1, characterized in that: In step 1, a data processing container is selected as the container orchestration platform. Based on business requirements and technology stack characteristics, a data processing container distribution is selected, including GKE, EKS or self-built, and initial configuration is performed, which includes node planning, network policy setting, storage solution configuration, and security policy customization based on RBAC (role-based access control). A Docker image repository is selected to store and manage these images, including Docker Hub, Harbor, or a self-built private repository to meet security requirements. At the same time, a CI / CD pipeline is built, using Jenkins and GitLab CI automation tools to achieve automation of image building, standardization of testing processes, and automatic push of images. The data processing container includes a cloud service platform, a virtual machine, and a Kubernetes (K8S) platform.

3. The method for forwarding data of a new energy vehicle based on a distributed architecture according to claim 1, characterized in that: In step 2, the responsibilities of each forwarding service are clearly defined based on business logic and data flow. The national platform forwarding service handles data transmission to the national regulatory platform, while the local platform forwarding service is responsible for distributing data to local regulatory platforms. Through the Horizontal Pod Autoscaler (HPA) function of the data processing container, the number of pods is dynamically adjusted according to the load of the service to cope with traffic fluctuations. The load balancing strategy is configured, and the load balancing function of Nginx IngressController or Istio is used to ensure uniform distribution of traffic between services and avoid single point overload.

4. The method for forwarding data of a new energy vehicle based on a distributed architecture according to claim 1, characterized in that: In step three, the implementation of dynamic expansion depends on the Horizontal Pod Autoscaler (HPA) function of the data processing container. HPA monitors specified resource indicators and automatically adjusts the number of Pod copies according to preset thresholds and policies. When system traffic increases, causing resource utilization to rise and exceed the set threshold, HPA triggers the Pod expansion operation and increases service instances to cope with traffic pressure; conversely, when traffic decreases and resource utilization decreases, HPA reduces service instances to release resources. This is achieved by combining the Service resources of the data processing container and the external load balancer. In the data processing container, the Service resources define the access method and load balancing policy of the Pod. By configuring the Service type to LoadBalancer and specifying the parameters of the external load balancer, automatic traffic distribution and load balancing are achieved.

5. The method for forwarding data of a new energy vehicle based on a distributed architecture according to claim 1, characterized in that: In step 4, the Prometheus and Grafana monitoring components provided by the data processing container are used to implement real-time monitoring and visualization of resources. By configuring the Prometheus collection rules and the Grafana dashboard, the usage of system resources, the status of service instances, and performance indicators, including request response time and throughput, are viewed. A centralized log collection and processing solution is adopted, including the use of ELK Stack. By configuring Logstash or Fluentd log collection tools, the log data scattered in each Pod is uniformly collected into the Elasticsearch log storage system, realizing centralized storage, retrieval, and analysis of logs.

6. The method for forwarding data of a new energy vehicle based on a distributed architecture according to claim 1, characterized in that: In step five, by configuring RBAC, defining roles and role bindings, and assigning different permissions to different users or services, fine-grained access control is achieved. In addition, the Secrets resource of the data processing container is used to securely store and manage sensitive information, including database passwords and API keys, to establish an identity authentication and authorization mechanism. For user access, multi-factor authentication technology is adopted, including combining multiple authentication methods such as username and password, SMS verification code, and biometrics to improve account security. For communication between services, two-way TLS authentication technology is used to ensure that communication between services is encrypted and authenticated to prevent man-in-the-middle attacks.

Images