Train data transmission method and device

Abstract

The invention provides a train data transmission method and device, and the method comprises the steps: constructing a BS end and a CS end, the BS end is mainly used for visual display, data configuration issuing and management, and test result analysis report generation, and the CS end is mainly used for direct communication with various data collectors and data preprocessing; an efficient data transmission channel is established through a Kafka system, a BS end and a CS end are connected, the BS end issues data configuration to the CS end, and the CS end sends preprocessed data to a Kafka message queue according to the data configuration; and subscribing a corresponding theme from the Kafka by the BS end, and obtaining data for analysis and display. Compared with the prior art, the train data transmission method and device provided by the invention can effectively improve the data transmission efficiency and safety, reduce the development cost and improve the production efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of data testing, and in particular to a train data transmission method and device. Background Art

[0002] The testing phase is a crucial link in the development and production of high-speed trains. It is necessary to conduct comprehensive and accurate testing and evaluation of various performance indicators of the train, which involves a large number of different types of data collection, such as mechanical properties, electrical properties, operating parameters, etc. Traditional data collection and processing methods have many problems, such as poor real-time data transmission, inflexible system architecture that is difficult to meet diverse testing needs, and low data processing efficiency.

[0003] Therefore, it is necessary to provide a new train data transmission method and device to overcome the above-mentioned defects. Summary of the invention

[0004] The purpose of the present invention is to provide a novel train data transmission method and device, which can effectively improve data transmission efficiency and security, reduce development costs, and improve production efficiency.

[0005] In order to achieve the above object, the present invention provides a train data transmission method, comprising:

[0006] Build BS and CS. BS is mainly used for visualization, data configuration distribution and management, and test result analysis report generation. CS is mainly used for direct communication with various data collectors and data preprocessing.

[0007] An efficient data transmission channel is established through the Kafka system to connect the BS and CS ends. The BS end sends the data configuration to the CS end, and the CS end sends the pre-processed data to the Kafka message queue according to the data configuration; the BS end subscribes to the corresponding topic from Kafka and obtains data for analysis and display.

[0008] The present invention also provides a data transmission device, the device comprising:

[0009] The browser side is mainly connected to the server and Kafka system for visualization, data analysis and configuration distribution;

[0010] Server, which is mainly connected with the client and Kafka system for data storage and forwarding;

[0011] The client mainly connects to the server and Kafka system for data cleaning and data compression;

[0012] The Kafka system uses its distributed architecture and efficient storage mechanism to transmit data between the client and the browser, ensuring high data availability;

[0013] The data collector is mainly connected to the client and is used for data collection and testing.

[0014] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the train data transmission method are implemented.

[0015] The present invention also provides a computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the train data transmission method when executing the computer program.

[0016] Compared with related technologies, the present invention can acquire data in real time, reducing data transmission delays and errors. The BS+CS hybrid architecture enables the system to meet the needs of testers for remote test task management and data analysis, while ensuring efficient data collection and preprocessing at the test site. At the same time, the introduction of Kafka makes it easy to expand the system with new data collection sources and data processing modules, adapt to the continuous development and changes of testing technology, improve data transmission efficiency and security, reduce development costs, and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0018] Figure 1 This is a system architecture diagram of the train data transmission method of the present invention;

[0019] Figure 2 It is a flow chart of the train data transmission method of the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Technical term explanation:

[0022] The BS-side framework is the Browser / Server architecture;

[0023] The CS-side framework is the Client / Server architecture;

[0024] Kafka is a high-throughput distributed publish-subscribe messaging system.

[0025] The present invention designs a train data transmission method and device, by constructing a BS end and a CS end, the BS end is mainly used for visual display, data configuration distribution and management, and test result analysis report generation, and the CS end is mainly used for direct communication with various data collectors and data preprocessing;

[0026] An efficient data transmission channel is established through the Kafka system to connect the BS and CS ends. The BS end sends the data configuration to the CS end, and the CS end sends the pre-processed data to the Kafka message queue according to the data configuration; the BS end subscribes to the corresponding topic from Kafka and obtains data for analysis and display.

[0027] See also Figure 1 and Figure 2 , Figure 1 is a system architecture diagram of the method of the present invention, Figure 2 Detailed description of the method of the present invention is given below.

[0028] The present invention adopts BS+CS hybrid architecture and introduces Kafka as the middleware for data transmission and processing. The BS part is mainly used for functions such as visual display, configuration and management of test tasks for test data, and generation of analysis reports of test results. Testers can access the BS system on different terminal devices through a browser to facilitate remote operation and monitoring. The CS part focuses on direct communication and data preprocessing with various data collectors, and is installed on a local server or industrial computer close to the test site to ensure the real-time and stability of data collection.

[0029] The data collector integration system supports the access of various types of data collectors, including but not limited to sensors such as temperature sensors, pressure sensors, vibration sensors, data recorders, test instruments, etc. For different types of data collectors, corresponding drivers and interfaces are developed to ensure stable data communication with the CS end.

[0030] For example, for train axle temperature monitoring, axle temperature data is collected through temperature sensors, and the sensors send the data to a specific interface on the CS side according to the agreed communication protocol. The CS side receives the data in real time and performs simple validity verification, such as data range check, and then encapsulates the data into a format suitable for Kafka transmission and sends it to the Kafka queue.

[0031] Kafka connects the BS and CS to form an efficient data transmission channel. The data collected by the data collector is first sent to the CS, which performs preliminary format conversion and screening on the data before sending the data to the Kafka message queue. The BS subscribes to the corresponding topic from Kafka to obtain data for further analysis and display.

[0032] Data processing flow

[0033] CS-side data preprocessing: In addition to receiving and sending data to Kafka, the CS-side also performs some data preprocessing work. This includes data cleaning, removing outliers and noise data; data compression to reduce data transmission volume and storage space; and real-time analysis and early warning of some key data. For example, when it is detected that the temperature of a key component of the train exceeds the set threshold, the CS-side immediately issues an alarm and sends the relevant data to Kafka after special marking.

[0034] Kafka data transmission and distribution: Kafka is the data hub responsible for reliable data transmission and distribution. Different topics are set according to the type and purpose of the data. For example, train operation parameter data is set as one topic, and mechanical component test data is set as another topic. The CS side sends the preprocessed data to the Kafka queue of the corresponding topic, and the BS side subscribes to the data of the corresponding topic as needed. Kafka uses its distributed architecture and efficient storage mechanism to ensure fast data transmission and high availability.

[0035] BS-side data analysis and display: After the BS-side subscribes to data from Kafka, it conducts in-depth data analysis and display. Through data visualization technology, such as charts (line charts, bar charts, pie charts, etc.), dashboards, etc., the test data is intuitively presented to testers and related technical personnel. At the same time, the BS-side also has data analysis functions, such as data statistical analysis, trend analysis, correlation analysis, etc., to help users evaluate the performance of the train and discover potential problems. For example, by analyzing the acceleration data of the train over a period of time, the dynamic performance and running stability of the train can be evaluated.

[0036] Compared with the related art, the present invention can bring the following beneficial effects:

[0037] Improve the real-time and accuracy of data collection: The CS terminal directly communicates with the data collector, can obtain data in real time, and perform preliminary processing, reducing data transmission delays and errors. Through the integration of multiple data collectors, it can comprehensively and accurately collect various data of high-speed trains during the test process, providing reliable data support for train performance evaluation.

[0038] Enhance the flexibility and scalability of the system: The BS+CS hybrid architecture enables the system to meet the needs of testers for remote test task management and data analysis, while ensuring efficient data collection and preprocessing at the test site. At the same time, the introduction of Kafka makes it easy to expand the system with new data collection sources and data processing modules, and can adapt to the continuous development and changes of high-speed train testing technology.

[0039] Improve data processing efficiency and reliability: As an efficient distributed message queue system, Kafka can quickly transmit and process large amounts of test data, avoiding data blockage and loss. The layered design in the data processing process, including preprocessing on the CS side and in-depth analysis on the BS side, improves the efficiency and quality of data processing. Through data storage and backup mechanisms, data security and traceability are guaranteed, providing a guarantee for subsequent data analysis and troubleshooting.

[0040] Optimize test workflow and management: Through the test task management and scheduling function on the BS side, testers can easily create, configure and manage test tasks, monitor the task execution status in real time, and improve the efficiency and coordination of test work. The data analysis and report generation functions provided by the system help testers quickly evaluate train performance, discover potential problems, adjust test plans and improve train design in a timely manner, thereby accelerating the development and production of high-speed trains.

[0041] See also Figure 2 The present invention also provides a data transmission device comprising:

[0042] The browser side is mainly connected to the server and Kafka system for visualization, data analysis and configuration distribution;

[0043] Server, which is mainly connected with the client and Kafka system for data storage and forwarding;

[0044] The client is mainly connected to the server, data collection device and Kafka system for data cleaning and data compression;

[0045] The Kafka system uses its distributed architecture and efficient storage mechanism to transmit data between the client and the browser to ensure high availability of data.

[0046] The data collector is mainly connected to the client and is used for data collection and testing.

[0047] Select hardware equipment suitable for the high-speed train test environment, including data collectors, servers (for CS end and data storage), network equipment, etc. Data collectors should select products with high accuracy and good stability according to test requirements, and ensure their compatibility with the CS end. The configuration of the server should be selected according to the data processing volume and system performance requirements, including CPU, memory, storage capacity, etc. Network equipment should ensure the stability and bandwidth of the test site network to meet the needs of data transmission.

[0048] The data collector is installed at the corresponding test location of the train, such as inside the carriage, near key components, etc. The CS server is deployed in the control room close to the test site and connected to the data collector through a wired network. The BS server can be deployed in a data center or remote office, and communicates with the CS and Kafka through the Internet.

[0049] Develop software systems for the BS and CS sides. The BS side uses front-end development technologies (such as HTML5, CSS3, JavaScript, etc.) combined with back-end development frameworks (such as Node.js, Django, Java, etc.) to develop functions such as user interface design, test task management, data analysis and display. The CS side uses languages ​​such as C++ or Java to develop functions such as communication with data collectors, data preprocessing, and Kafka data transmission.

[0050] Install and configure the Kafka environment, including the deployment of Zookeeper and the cluster settings of Kafka. According to the data volume and processing requirements, rationally plan the number of Kafka topics, partitions, and replicas to ensure efficient data transmission and storage. Integrate the BS side, CS side, and Kafka, and realize data interaction and communication through corresponding APIs and interfaces.

[0051] Conduct functional testing, performance testing, and stability testing of the system. Functional testing includes verification of functions such as data collection, data processing, test task management, data analysis, and display to ensure the normal operation of various system functions. Performance testing mainly tests the performance of the system under large data volumes and high concurrency, such as data transmission rate, data processing efficiency, etc., and improves system performance by optimizing system parameters and algorithms. Stability testing runs the system for a long time to observe whether the system has problems such as memory leaks and data loss, and promptly repairs the problems found to ensure the stability and reliability of the system. Optimize and adjust the system based on the test results, and continuously improve the system functions and performance to meet the actual needs of high-speed train test data collection and processing.

[0052] Compared with related technologies, the present invention can acquire data in real time, reducing data transmission delays and errors. The BS+CS hybrid architecture enables the system to meet the needs of testers for remote test task management and data analysis, while ensuring efficient data collection and preprocessing at the test site. At the same time, the introduction of Kafka makes it easy to expand the system with new data collection sources and data processing modules, adapt to the continuous development and changes of testing technology, improve data transmission efficiency and security, reduce development costs, and improve production efficiency.

[0053] On the other hand, the present invention further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned train data transmission method are implemented.

[0054] As an extension of another aspect of the present invention, a computer terminal is also provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned train data transmission method when executing the computer program.

[0055] When the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are realized. Exemplarily, the computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0056] The computer terminal may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. It may include, but is not limited to, a processor and a memory. It may include more or fewer components, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, buses, etc.

[0057] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0058] The memory may be an internal storage unit, such as a hard disk or a memory. The memory may also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory may include both an internal storage unit and an external storage device. The memory is used to store the computer program and other programs and data. The memory may also be used to temporarily store data that has been output or is to be output.

[0059] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0060] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0061] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0062] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0064] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0065] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0066] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A train data transmission method, characterized in that: include: Build BS and CS. BS is mainly used for visualization, data configuration distribution and management, and test result analysis report generation. CS is mainly used for direct communication with various data collectors and data preprocessing. Establish an efficient data transmission channel through the Kafka system to connect the BS and CS. The BS sends the data configuration to the CS, and the CS sends the pre-processed data to the Kafka message queue according to the data configuration. The BS subscribes to the corresponding topic from Kafka and obtains data for analysis and display.

2. The train data transmission method according to claim 1, characterized in that: The BS system is mainly used for visualization, data configuration and management, and test result analysis report generation, including: The BS side uses front-end development technology combined with back-end development framework to achieve user interface design, test task management, data analysis and display; Through visualization technology, the test data can be intuitively presented to testers and related technical personnel; through data analysis functions, the performance of various indicators can be evaluated, potential problems can be discovered, and corresponding reports can be generated.

3. The train data transmission method according to claim 2, characterized in that: The CS end is mainly used for direct communication with various data collectors and data preprocessing, including: The CS side is developed using C++ or Java to achieve communication with the data collector, data preprocessing, and Kafka data transmission; Data preprocessing on the CS side includes data cleaning, which is mainly used to remove outliers and noise data; data compression, which is mainly used to reduce data transmission volume and storage space; as well as real-time analysis and early warning of data, and sending relevant data to Kafka after special marking.

4. The train data transmission method according to claim 3, characterized in that: The establishment of an efficient data transmission channel through the Kafka system to connect the BS and CS ends includes: As the data hub, the Kafka system is responsible for the reliable transmission and distribution of data, and sets different topics according to the type and purpose of the data; The CS end encapsulates the preprocessed data according to the format transmitted by the Kafka system and sends it to the Kafka system queue of the corresponding topic. The BS subscribes to the data of the corresponding topic as needed. The Kafka system uses its distributed architecture and efficient storage mechanism to ensure the fast transmission and high availability of the data subscribed by the BS.

5. A device for applying the train data transmission method according to any one of claims 1 to 4, characterized in that: include: The browser side is mainly connected to the server and Kafka system for visualization, data analysis and configuration distribution; Server, which is mainly connected with the client and Kafka system for data storage and forwarding; The client mainly connects to the server and Kafka system for data cleaning and data compression; The Kafka system uses its distributed architecture and efficient storage mechanism to transmit data between the client and the browser, ensuring high data availability; The data collector is mainly connected to the client and is used for data collection and testing.

6. The train data transmission method according to claim 5, characterized in that: The data collector includes one or more of a temperature sensor, a pressure sensor, a vibration sensor, a noise sensor, a data recorder and a data tester.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the train data transmission method according to any one of claims 1 to 4 are implemented.

8. A computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the train data transmission method according to any one of claims 1 to 4 are implemented.

Images