A data processing method, system, electronic device and medium

Abstract

The application discloses a data processing method, system, electronic equipment and medium, the data is communication data in a DDS real-time communication process, and the method comprises the following steps: step 1, setting the data to be counted, and generating an XML file; step 2, identifying and counting the data to be counted according to the generated XML file, and generating a statistical file; step 3, screening out valid data in the statistical file and arranging and comparing to obtain result data; and step 4, displaying the result data according to the set dimension. Through the method, system, electronic equipment and medium, the data concerned by the user in the DDS real-time communication process can be counted and clearly displayed, and it is convenient to troubleshoot problems in the DDS communication process.

Description

Technical Field

[0001] This invention relates to the field of distributed real-time communication technology, and in particular to a data processing method, system, electronic device, and medium. Background Technology

[0002] With the development of intelligent vehicles, Data Distribution Service (DDS), with its real-time performance, high reliability, open architecture, and decoupled publish / subscribe capabilities, is increasingly widely used in the field of autonomous vehicles. However, in practical applications, DDS often experiences packet loss and slow transmission speeds. In such cases, it is necessary to analyze the messages to determine the causes. However, current message analysis methods have two problems: first, message analysis is highly specialized, involving numerous elements and is quite challenging; second, the sheer volume of data in messages makes it difficult to intuitively statistically analyze and display key information, resulting in low analysis efficiency. Therefore, how to better statistically analyze and display communication data during DDS real-time communication to facilitate troubleshooting of problems in the DDS communication process is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0003] To address the limitations of existing technologies, this invention provides a data processing method, system, electronic device, and medium for statistically analyzing and processing data of interest to users during DDS real-time communication, facilitating troubleshooting of problems in the DDS communication process.

[0004] To achieve the above effects, the technical solution of the present invention is as follows:

[0005] The first aspect of this invention provides a data processing method, wherein the data is communication data during DDS real-time communication, comprising the following steps:

[0006] Step 1: The user selects the data to be analyzed in the settings module and generates an XML file;

[0007] Step 2: The statistics module identifies and performs statistics on the data to be analyzed based on the generated XML file, and generates a statistics file;

[0008] Step 3: The processing module filters out valid data from the statistics file, organizes and compares it, and obtains the result data;

[0009] Step 4: The storage module stores the result data.

[0010] Furthermore, the data to be counted in step 1 includes at least one of the following: number of entities, number of publishers, number of receivers, number of writers, number of readers, number of write packets, number of read packets, sending time, writing time, reading time, writer cache, and reader cache data.

[0011] Furthermore, step 1 specifically involves the user selecting the data to be statistically analyzed in the settings module. After the selection is completed, the selected data is generated into an XML file based on the user's selection.

[0012] Furthermore, the user selects the data to be statistically analyzed. After the selection is completed, an XML file is generated based on the user's selection, specifically as follows:

[0013] Based on the actual business scenarios and problems encountered during DDS real-time communication, users can select the data to be statistically analyzed from the communication data. After selecting the data, the selected data is generated into an XML file. Users can then place the XML file into a preset project. When the program runs, it selectively records important data based on the XML file to obtain the most important data.

[0014] Furthermore, step 2 specifically involves the statistics module identifying the important data to be counted based on the generated XML file, automatically writing and counting the most important data during the DDS real-time communication process, and generating a statistics file.

[0015] Furthermore, step 3 specifically involves: the processing module reading data from the statistical file; after reading the statistical file, filtering out valid data from the statistical file according to the user's settings to obtain the second most important data; after filtering, organizing and comparing the second most important data to form result data that is convenient for user analysis; and storing the result data.

[0016] The sorting and comparison process involves parsing the second most important data, analyzing whether the number of packets written and read is consistent, counting the number of lost packets, calculating the time consumption of each step, sorting the written and read data in chronological order, and calculating the size of each written and read data and the cache space.

[0017] Furthermore, after step 4, the user can set the display dimensions of the result data according to their actual needs and then display the result data on the page.

[0018] Specifically, the user can set the display dimensions of the result data according to actual needs. The user can display data arranged in chronological order, data arranged in chronological order of consumption time, and data arranged in chronological order of packet loss.

[0019] A second aspect of the present invention provides a data processing system, comprising a setting module, a statistics module, a processing module, a storage module, and a display module connected in sequence; wherein,

[0020] The setting module is used to select the data to be statistically analyzed; that is, the user selects the data to be statistically analyzed from the communication data of the setting module. After the data to be statistically analyzed is selected, the setting module automatically generates an XML file from the selected data. The user puts the XML file into a preset project. When the program runs, it selectively records important data according to the XML file to obtain the first important data. The setting module sends the first important data to the statistics module. The XML file is used by the statistics module to filter the important data that needs to be recorded.

[0021] The statistics module is used to send important data of concern to users in DDS real-time communication; that is, the statistics module is built on the DDS code, and selectively and automatically identifies the important data to be counted according to the XML file generated by the settings module. During the DDS real-time communication process, it automatically writes and counts the first important data, generates a statistics file, and sends the statistics file to the processing module.

[0022] The processing module is used to read data from the statistical file. After reading the statistical file, it filters out the valid data in the statistical file according to the user's settings to obtain the second important data. After filtering, the second important data is sorted and compared to form the result data that is convenient for the user to analyze. The result data is then sent and saved in the storage module.

[0023] The storage module is used to store the result data;

[0024] The display module is used to display the result data in the storage module.

[0025] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the data processing method by means of the computer program.

[0026] A fourth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein the computer program executes the data processing method when it is run.

[0027] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0028] This invention can automatically collect important data during real-time DDS communication based on user selections, and process and display the statistical results according to user settings. This allows users to observe the DDS communication process more clearly and intuitively, and analyze the causes of problems more accurately. Compared to current methods of manually analyzing messages, the results provided by this invention are more concise and the process is more efficient. Attached Figure Description

[0029] Figure 1This is a schematic diagram of a data processing method provided in an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the data processing method system provided in an embodiment of the present invention;

[0031] Explanation of reference numerals in the attached diagram: 10, Data processing system; 100, Settings module; 200, Statistics module; 300, Processing module; 400, Storage module; 500, Display module. Detailed Implementation

[0032] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0033] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0034] Before introducing the embodiments of the present invention, the relevant terms involved in the embodiments of the present invention are first defined as follows:

[0035] DDS: Data Distribution Service;

[0036] XML: Extensible Markup Language.

[0037] This embodiment proposes a data processing method, such as... Figure 1 As shown, it includes the following steps:

[0038] Step 1: The user selects the data to be analyzed in the settings module 100 and generates an XML file; specifically,

[0039] Based on the actual business scenarios and problems encountered during DDS real-time communication, users can select the data to be statistically analyzed from the communication data. After selecting the data to be statistically analyzed in the settings module 100, the selected data is generated into an XML file. Users can put the XML file into a preset project. When the program runs, it selectively records important data based on the XML file to obtain the most important data.

[0040] It should be noted that the program execution is the DDS real-time communication process.

[0041] In this embodiment, the data to be counted includes at least one of the following: number of entities, number of publishers, number of receivers, number of writers, number of readers, number of write packets, number of read packets, sending time, writing time, reading time, writer cache, and reader cache data.

[0042] Step 2: The statistics module 200 identifies and performs statistics on the data to be analyzed based on the generated XML file, and generates a statistics file; specifically:

[0043] The statistics module 200 identifies the important data to be counted based on the generated XML file, automatically writes and counts the most important data during the DDS real-time communication process, and generates a statistics file.

[0044] Step 3: The processing module 300 filters out valid data from the statistical file and organizes and compares it to obtain the result data; specifically: the processing module 300 reads data from the statistical file, and after reading the statistical file, it filters out valid data from the statistical file according to the user's settings to obtain the second most important data. After filtering, the second most important data is organized and compared to form result data that is convenient for users to analyze, and the result data is stored at the same time.

[0045] The sorting and comparison process involves parsing the second most important data, analyzing whether the number of packets written and read is consistent, counting the number of lost packets, calculating the time consumption of each step, sorting the written and read data in chronological order, and calculating the size of each written and read data and the cache space.

[0046] Step 4: The storage module 400 stores the result data;

[0047] In this embodiment, after step 4, the user further includes setting the display dimension of the result data according to actual needs and displaying the result data on the page.

[0048] In this embodiment, the user sets the display dimension of the result data according to actual needs. Specifically, the user can display data arranged in chronological order, data arranged in chronological order of consumption time, and data arranged in chronological order of packet loss.

[0049] Using the above method, users (including any software user) can quickly troubleshoot problems in the DDS communication process based on the obtained statistical results. The results are more intuitive, saving time and manpower. At the same time, the system of the present invention can be further optimized to better meet the needs of users.

[0050] In another embodiment, the present invention also provides a data processing system 10, see below. Figure 2 The data refers to communication data during the DDS real-time communication process, used for data statistics and display, and includes a setting module 100, a statistics module 200, a processing module 300, a storage module 400, and a display module 500 connected in sequence; wherein,

[0051] The setting module 100 is used to select the data to be statistically analyzed; that is, the user selects the data to be statistically analyzed from the communication data of the setting module 100. After the data to be statistically analyzed is selected, the setting module 100 automatically generates an XML file from the selected data. The user puts the XML file into a preset project. When the program runs, it selectively records important data according to the XML file to obtain the first important data. The setting module 100 sends the first important data to the statistics module 200. The XML file is used by the statistics module 200 to filter the important data to be recorded.

[0052] The statistics module 200 is used to send important data of concern to the user in DDS real-time communication; that is, the statistics module 200 is built on the basis of DDS code, and selectively and automatically identifies important data that needs to be counted according to the XML file generated by the setting module 100. During the DDS real-time communication process, it automatically writes and counts the first important data, generates a statistics file, and sends the statistics file to the processing module 300.

[0053] The processing module 300 is used to read data from the statistical file. After reading the statistical file, it filters out the valid data in the statistical file according to the user's settings to obtain the second important data. After filtering, the second important data is sorted and compared to form the result data that is convenient for the user to analyze. The result data is then sent and saved in the storage module 400.

[0054] The storage module 400 is used to store the result data;

[0055] The display module 500 is used to display the result data in the storage module 400.

[0056] In this embodiment, the storage module 400 can choose to save the result data in the form of a database, data table, etc.

[0057] In this embodiment, for the display module 500, the result data is used by the user to analyze the problem and determine the cause of the problem. The user can set the display dimension of the result data according to actual needs. After selecting the display dimension, the user can find the steps with the most time consumption and the steps with serious packet loss by analyzing the result data and sorting the data according to the number of packet loss, so as to find the cause of the problem more accurately and efficiently.

[0058] In another embodiment, the present invention also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause a vehicle to perform the data processing method as described in any of the above embodiments.

[0059] In this embodiment, the computer system suitable for implementing the electronic device of the present invention includes a Central Processing Unit (CPU), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or a program loaded from storage into random access memory (RAM), such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0060] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN (Local Area Network) cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0061] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs various functions defined in the system of the present invention.

[0062] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0063] In another embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer's processor, causes the computer to perform the data processing method provided in any of the foregoing embodiments. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0064] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0065] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A data processing method, wherein the data is communication data during DDS real-time communication, characterized in that, Includes the following steps: Step 1: The user selects the data to be analyzed in the settings module and generates an XML file; In this process, users select the data to be counted from the communication data based on the actual business scenarios and problems encountered during the DDS real-time communication process. After selecting the data to be counted, the selected data is generated into an XML file. Users put the XML file into a preset project. When the program runs, it selectively records important data based on the XML file to obtain the most important data. Step 2: The statistics module identifies and performs statistics on the data to be analyzed based on the generated XML file, and generates a statistics file; Step 3: The processing module filters out valid data from the statistics file, organizes and compares it, and obtains the result data; specifically: The processing module reads data from the statistical file. After reading the statistical file, it filters out the valid data from the statistical file according to the user's settings to obtain the second most important data. After filtering, the second most important data is sorted and compared to form the result data that is convenient for users to analyze. At the same time, the result data is stored. The sorting and comparison process involves parsing the second most important data, analyzing whether the number of packets written and read is consistent, counting the number of lost packets, counting the time consumption of each step, sorting the written and read data in chronological order, and calculating the size of each written and read data and the cache space, etc. Step 4: The storage module stores the result data; The data to be counted in step 1 includes at least one of the following: number of entities, number of publishers, number of receivers, number of writers, number of readers, number of write packets, number of read packets, sending time, writing time, reading time, writer cache, and reader cache data. Step 4 is followed by the user setting the display dimensions of the result data according to actual needs, and then displaying the result data on the page; Specifically, the user can set the display dimensions of the result data according to actual needs. The user can display data arranged in chronological order, data arranged in chronological order of consumption time, and data arranged in chronological order of packet loss.

2. The data processing method according to claim 1, characterized in that, Step 1 involves the user selecting the data to be analyzed in the settings module. After the selection is complete, an XML file is generated based on the user's selection.

3. The data processing method according to claim 1, characterized in that, Step 2 specifically involves the statistics module identifying the important data to be counted based on the generated XML file, automatically writing and counting the most important data during the DDS real-time communication process, and generating a statistics file.

4. A data processing system according to any one of the data processing methods in claims 1-3, characterized in that, It includes a settings module, a statistics module, a processing module, a storage module, and a display module connected in sequence; among them, The setting module is used to select the data to be statistically analyzed; that is, the user selects the data to be statistically analyzed from the communication data of the setting module. After the data to be statistically analyzed is selected, the setting module automatically generates an XML file from the selected data. The user puts the XML file into a preset project. When the program runs, it selectively records important data according to the XML file to obtain the first important data. The setting module sends the first important data to the statistics module. The XML file is used by the statistics module to filter the important data that needs to be recorded. The statistics module is used to send important data of concern to users in DDS real-time communication; that is, the statistics module is built on the DDS code, and selectively and automatically identifies the important data to be counted according to the XML file generated by the settings module. During the DDS real-time communication process, it automatically writes and counts the first important data, generates a statistics file, and sends the statistics file to the processing module. The processing module is used to read data from the statistical file. After reading the statistical file, it filters out the valid data in the statistical file according to the user's settings to obtain the second important data. After filtering, the second important data is sorted and compared to form the result data that is convenient for the user to analyze. The result data is then sent and saved in the storage module. The storage module is used to store the result data; The display module is used to display the result data in the storage module.

5. An electronic device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the data processing method according to any one of claims 1-3 through the computer program.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program, when executed, performs the data processing method according to any one of claims 1-3.

Images