Data processing method, device, equipment and medium

Abstract

The embodiments of the present application provide a method, apparatus, device and medium for data processing, the method comprising: obtaining data to be processed, wherein the data to be processed is received from a second device, or the data to be processed is generated by the first device; running a data processing script, and using the data processing script to perform data processing operations on the data to be processed to obtain target data, wherein the data processing script is automatically generated by a script generation program through a protocol configuration table, and the protocol configuration table is used to store communication information between the first device and the second device, and the communication information at least includes a correspondence between a telegram name and a telegram number; performing subsequent operations on the target data. Through some embodiments of the present application, it is possible to operate on the data to be processed through an automatically generated data processing script, thereby eliminating the need for developers to develop scripts and greatly improving production efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication data processing, and specifically to a method, apparatus, device, and medium for data processing. Background Art

[0002] In the communications field, devices on both sides often communicate with multiple data points. Developers must adapt to different data structures when developing, and the more data exchanged, the greater the workload. Related technologies require developers to develop processing logic for each field, comparing the protocol's data structure. This complexity makes it prone to errors. Furthermore, data additions, deletions, and modifications may be required at various stages of software development and release. These changes require developers to modify code, reducing productivity.

[0003] Therefore, how to improve production efficiency in the data processing process becomes a problem that needs to be solved. Summary of the Invention

[0004] Embodiments of the present application provide a method, apparatus, device, and medium for data processing. Through some embodiments of the present application, at least the data to be processed can be operated through automatically generated data processing scripts, thereby eliminating the need for developers to develop scripts and greatly improving production efficiency.

[0005] In a first aspect, the present application provides a data processing method, which is applied to a first device, and the method includes: obtaining data to be processed, wherein the data to be processed is received from a second device, or the data to be processed is generated by the first device; running a data processing script, and using the data processing script to perform data processing operations on the data to be processed to obtain target data, wherein the data processing script is automatically generated by a script generation program through a protocol configuration table, and the protocol configuration table is used to store communication information between the first device and the second device, and the communication information at least includes the correspondence between the telegram name and the telegram number; performing subsequent operations on the target data.

[0006] Therefore, unlike the method in the related art that requires developers to compare the data structure of the protocol and develop processing logic field by field, the embodiment of the present application processes the data to be processed by using the data processing script automatically generated by the protocol configuration table, which can replace the developer's writing of the data processing logic, thereby improving production efficiency and reducing the waste of human resources.

[0007] In combination with the first aspect, in one embodiment of the present application, before running the data processing script and using the data processing script to perform data processing operations on the data to be processed to obtain the target data, the method also includes: generating a search statement for searching for content corresponding to the data to be processed according to the i-th row in the protocol configuration table; generating a parsing statement for parsing the data to be processed according to the i-th row in the protocol configuration table, and generating an assembly statement for assembling the data to be processed according to the i-th row in the protocol configuration table; generating a sending statement for sending the target data, and generating a storage statement for storing the target data according to the i-th row in the protocol configuration table; repeating the above steps until the protocol configuration table is read, integrating the search statement, the parsing statement, the assembly statement, the sending statement and the storage statement into a program to obtain the data processing script.

[0008] Therefore, the embodiment of the present application generates search statements, parsing statements, assembly statements, sending statements and storage statements for the protocol configuration table line by line, and assembles them into data processing scripts, which can save developers time in writing scripts and thus improve production efficiency.

[0009] In combination with the first aspect, in one embodiment of the present application, the search statement, the parsing statement, the assembly statement, the sending statement and the storage statement are integrated into a program to obtain the data processing script, including: establishing a data search module, a data parsing storage module and a data assembly sending module; integrating the search statement into the data search module, integrating the parsing statement and the storage statement into the data parsing storage module, and integrating the assembly statement and the sending statement into the data assembly sending module to obtain the data processing script.

[0010] Therefore, by establishing modules with different functions, the embodiment of the present application can make corresponding calls according to functional requirements during the execution of the script, thereby improving the execution speed of the script.

[0011] In combination with the first aspect, in one embodiment of the present application, after obtaining the data processing script, the method further includes: when the protocol configuration table is updated, reading the updated protocol configuration table; re-running the script generation program to obtain the updated data processing script.

[0012] Therefore, the embodiment of the present application can automatically process the new configuration without the developer having to update the program by re-running the script generation program when there is an update to the protocol configuration table, thereby improving production efficiency.

[0013] In combination with the first aspect, in one embodiment of the present application, the data to be processed is received from a second device; the use of the data processing script to perform data processing operations on the data to be processed to obtain target data includes: finding the field length and field name related to the data to be processed through the data search module in the data processing script; parsing the data to be processed based on the field length through the data parsing and storage module in the data processing script to obtain the target data; the subsequent operations on the target data include: storing the target data in a corresponding data table based on the field name.

[0014] In combination with the first aspect, in one embodiment of the present application, the data to be processed is generated by the first device; the use of the data processing script to perform data processing operations on the data to be processed to obtain target data includes: finding the field length and field name related to the data to be processed through the data search module in the data processing script; assembling the data to be processed according to the field length, field name and field type through the data assembly and sending module in the data processing script to obtain the target data; the subsequent operation on the target data includes: sending the target data to the second device.

[0015] Therefore, the embodiment of the present application can realize corresponding calls according to functional requirements during the execution of the script through each module in the script, thereby improving the execution speed of the script.

[0016] In the second aspect, the present application provides a data processing device, which is applied to a first device, and the device includes: a data acquisition module, configured to acquire data to be processed, wherein the data to be processed is received from a second device, or the data to be processed is generated by the first device; a script running module, configured to run a data processing script, and use the data processing script to perform data processing operations on the data to be processed to obtain target data, wherein the data processing script is automatically generated by a script generation program through a protocol configuration table, and the protocol configuration table is used to store communication information between the first device and the second device, and the communication information at least includes the correspondence between the telegram name and the telegram number; a data operation module, configured to perform subsequent operations on the target data.

[0017] In combination with the second aspect, in one embodiment of the present application, the script generation module is configured to: generate a search statement for searching for content corresponding to the data to be processed according to the i-th row in the protocol configuration table; generate a parsing statement for parsing the data to be processed according to the i-th row in the protocol configuration table, and generate an assembly statement for assembling the data to be processed according to the i-th row in the protocol configuration table; generate a sending statement for sending the target data, and generate a storage statement for storing the target data according to the i-th row in the protocol configuration table; repeat the above steps until the protocol configuration table is read, integrate the search statement, the parsing statement, the assembly statement, the sending statement and the storage statement into a program, and obtain the data processing script.

[0018] In combination with the second aspect, in one embodiment of the present application, the script generation module is further configured to: establish a data search module, a data parsing and storage module, and a data assembly and sending module; integrate the search statement into the data search module, integrate the parsing statement and the storage statement into the data parsing and storage module, and integrate the assembly statement and the sending statement into the data assembly and sending module to obtain the data processing script.

[0019] In combination with the second aspect, in one embodiment of the present application, the script generation module is further configured to: when the protocol configuration table is updated, read the updated protocol configuration table; re-run the script generation program to obtain the updated data processing script.

[0020] In combination with the second aspect, in one embodiment of the present application, the data to be processed is received from a second device; the script execution module is further configured to: find the field length and field name related to the data to be processed through the data search module in the data processing script; parse the data to be processed based on the field length through the data parsing and storage module in the data processing script to obtain the target data; the subsequent operation on the target data includes: storing the target data in a corresponding data table based on the field name.

[0021] In combination with the second aspect, in one embodiment of the present application, the data to be processed is generated by the first device; the script running module is also configured to: find the field length and field name related to the data to be processed through the data search module in the data processing script; assemble the data to be processed according to the field length, field name and field type through the data assembly and sending module in the data processing script to obtain the target data; the subsequent operation on the target data includes: sending the target data to the second device.

[0022] In a third aspect, the present application provides an electronic device comprising: a processor, a memory and a bus; the processor is connected to the memory via the bus, the memory stores a computer program, and when the computer program is executed by the processor, it can implement the method described in any embodiment of the first aspect.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, the method described in any embodiment of the first aspect can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is one of the data processing scenario diagrams shown in an embodiment of the present application;

[0025] Figure 2 This is a second data processing scenario diagram shown in an embodiment of the present application;

[0026] Figure 3 This is one of the flow charts of a data processing method shown in an embodiment of the present application;

[0027] Figure 4 This is a second flow chart of a data processing method shown in an embodiment of the present application;

[0028] Figure 5 This is a third flowchart of a data processing method shown in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of a data processing device according to an embodiment of the present application;

[0030] Figure 7 This is a schematic diagram of the composition of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0032] The embodiments of the present application can be applied to scenarios where target data is obtained by performing operations such as parsing and storing data to be processed. In order to improve the problems in the background technology, in some embodiments of the present application, the data to be processed is processed by using a data processing script automatically generated by a protocol configuration table, which can replace the developer's writing of data processing logic, thereby improving production efficiency and reducing waste of human resources. For example, in some embodiments of the present application, after obtaining the data to be processed, the data processing script automatically generated by the protocol configuration table is run, and the data processing script is used to perform data processing operations on the data to be processed to obtain the target data.

[0033] The method steps in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0034] Figure 1 A data processing scenario diagram of some embodiments of the present application is provided, which includes a first device 110 and a second device 120. Specifically, the second device 120 sends data to be processed to the first device 110. After receiving the data to be processed, the first device 110 runs a data processing script and uses the data processing script to perform operations such as parsing the data to be processed to obtain target data, and then stores the target data.

[0035] Figure 2 A second data processing scenario diagram in some other embodiments of the present application is provided, which includes a first device 110 and a second device 120. Specifically, after the first device 110 generates data to be processed, it runs a data processing script, and uses the data processing script to combine the data to be processed and other operations to obtain target data, and then sends the target data to the second device 120.

[0036] In the field of computer communications, the TCP / IP protocol is the most basic protocol of the Internet. In a communication link based on the TCP / IP protocol, the essence of transmission is a data byte stream. The two communicating parties need to agree on an application layer protocol and construct or parse a byte stream according to a unified protocol to achieve the purpose of communication between the two parties. What is different from the embodiments of the present application is that in the related art, for a certain piece of data, developers need to refer to the data structure of the protocol and develop processing logic field by field. Due to the cumbersome content of the protocol, errors are prone to occur. In addition, there may be a need to add, delete, and modify data at different stages from development to release of the software, which involves changes. Developers need to modify the code, which reduces production efficiency. However, the present application processes the data to be processed by using a data processing script automatically generated by a protocol configuration table, which can replace the developer's writing of data processing logic, thereby improving production efficiency and reducing the waste of human resources.

[0037] The following will take the first device as an example to illustrate a data processing method performed by the first device in this application. It is understandable that the first device can be a server, a computer, or other device.

[0038] At least to solve the problems in the background technology, such as Figure 3 As shown, some embodiments of the present application provide a data processing method, the method comprising:

[0039] S310, obtaining data to be processed.

[0040] In one embodiment of the present application, the data to be processed is received by the first device from the second device. After receiving the data to be processed, the first device uses a data processing script to parse the data to be processed to obtain target data, and then stores the target data in the database of the first device.

[0041] For example, the first device receives a blast furnace iron-out message from the second device, and then runs a data processing script to parse the blast furnace iron-out message based on the field length and field name, obtain target data such as operation identifier, event time, and operation time, and store the parsed target data according to a pre-set path.

[0042] In another embodiment of the present application, the data to be processed is data generated by the first device. After the data processing script integrates the data to be processed, target data is obtained, and finally the target data is sent to the second device.

[0043] For example, the first device generates information about blast furnace iron production, including operation identification, event time, operation time, etc., and then runs a data processing script to combine the information about blast furnace iron production based on field length and field name, obtain target data, and send the target data to the second device.

[0044] S320: Run the data processing script, and use the data processing script to perform data processing operations on the data to be processed to obtain target data.

[0045] It should be noted that the data processing script is a program automatically generated by a protocol configuration table. The protocol configuration table is used to store communication information between the first device and the second device. The communication information at least includes the correspondence between the message name and the message number. For example, the message name is "blast furnace iron tapping information" and the message number is I3Z101.

[0046] As a specific embodiment of the present application, the protocol configuration table includes a data entry configuration table and a data structure configuration table, wherein the data entry configuration table is used to store the name of the message transmitted between the first device and the second device, the message identifier, and the table name of the first device storing the corresponding message. For example, the data entry configuration table is shown in Table 1:

[0047] Table 1 Data entry configuration table

[0048] Serial number Telegram Name Telegram number Table name 1 Blast furnace iron tapping information I3Z101 XX_RECV_FUR_SPT_INFO 2 Tank car iron information I3Z102 XX_RECV_FUR_TPC_INFO 3 Wind oxygen information I3Z103 XX_RECV_FUR_WIND_OXY 4 Rest and Recovery Plan I3Z104 XX_RECV_FUR_STOP_PLAN 5 Blast furnace tapping plan I3Z105 XX_RECV_FUR_CAST_PLAN

[0049] The data structure configuration table is the specific information corresponding to any row in the data entry configuration table. The data structure configuration table is used to store information such as field length, field name, and field description used to parse or combine data to be processed. For example, the data structure configuration table corresponding to the blast furnace iron tapping information in Table 1 is shown in Table 2:

[0050] Table 2 Blast furnace iron tapping information data structure configuration table

[0051]

[0052] It can be understood that the number of data structure configuration tables in this application is the same as the number of telegram names in the data entry configuration table. For example, if there are 5 telegram names in the data entry configuration table, then these 5 telegram names correspond to one data structure configuration table respectively. In other words, one telegram name corresponds to one data structure configuration table.

[0053] In one embodiment of the present application, before S320, a data processing script needs to be automatically generated based on the data entry configuration table and at least one data structure configuration table. The specific implementation process of generating the data processing script is as follows:

[0054] S1, generating a search statement for searching for content corresponding to the data to be processed according to the i-th row in the protocol configuration table.

[0055] That is to say, whether the data to be processed is combined to obtain the target data, or the data to be processed is parsed to obtain the target data, it is necessary to first find the corresponding field name, field length, and field description in the protocol configuration table. Therefore, this application automatically generates a search statement for finding the content corresponding to the data to be processed.

[0056] For example, the functions of the search statement include: generating a search statement for the correspondence between the telegram name and the telegram number based on the first row of the data entry configuration table, and generating a search statement for the field name, field description, field type, and field length based on the data structure configuration table corresponding to the first row. In other words, if the received data to be processed is "I3Z101", the search statement can be used to find the telegram name corresponding to "I3Z101" in the data entry configuration table as blast furnace iron tapping information. Then, the corresponding data structure matching table is used to find the field names and field lengths. Finally, the data to be processed is parsed based on the field names and field lengths to obtain the target data.

[0057] S2, generating a parsing statement for parsing the data to be processed according to the i-th row in the protocol configuration table, and generating an assembling statement for assembling the data to be processed according to the i-th row in the protocol configuration table.

[0058] That is to say, when the data to be processed is data generated by the first device, an assembly statement is required to assemble the data to be processed; when the data to be processed is data obtained by the first device from the second device, a parsing statement is required to parse the data to be processed.

[0059] For example, the functions of the parsing statement include: according to the first row in the data entry configuration table, finding the blast furnace iron-making information data structure configuration table, and then, according to the field length in the blast furnace iron-making information data structure configuration table, splitting the data to be processed, and matching the corresponding field description, field name and field type after the split to obtain the target data.

[0060] For example, the functions of the assembly statement include: according to the first row in the data entry configuration table, finding the blast furnace iron tapping information data structure configuration table, and then combining the data to be processed generated by the first device according to the field length in the blast furnace iron tapping information data structure configuration table (for example, the data to be processed are "DEAL_FLAG" and "WORK_TIME", then the above data to be processed are spliced according to the field length, that is, the first byte is "DEAL_FLAG", and the following 14 bytes are "WORK_TIME"), and matching the field description, field name and field type accordingly to obtain the target data.

[0061] S3, generating a sending statement for sending the target data, and generating a storage statement for storing the target data according to the i-th row in the protocol configuration table.

[0062] That is to say, in the case where the data to be processed is data generated by the first device, after the data to be processed is spliced to obtain the target data, the target data needs to be sent to the second device. Therefore, the present application generates a sending statement in the script for sending the target data to the second device. In the case where the data to be processed is data obtained by the first device from the second device, after the data to be processed is parsed to obtain the target data, the target data needs to be stored in the corresponding table (that is, the target data is stored in the table name in the data entry configuration table). Therefore, the present application generates a storage statement in the script for storing the target data in the corresponding table of the first device.

[0063] S4, repeat the above steps until the protocol configuration table is read, and integrate the search statement, parsing statement, assembly statement, sending statement and storage statement into a program to obtain a data processing script.

[0064] In other words, the present application uses the data entry configuration table in the protocol configuration table to find statements, and the data structure configuration table generates search statements, parsing statements, assembly statements, and storage statements line by line. After both the data entry configuration table and the data structure configuration table are read and generated, the statements are integrated into a program to obtain a data processing script. It is understood that the send statement is used to add the necessary message header information to the target data based on the message number and other information in the data entry configuration table after obtaining the target data and then send it.

[0065] Specifically, first, a data search module, a data parsing and storage module, and a data assembly and sending module are established. Then, the search statement is integrated into the data search module, the parsing statement and storage statement are integrated into the data parsing and storage module, and the assembly statement and sending statement are integrated into the data assembly and sending module to obtain a data processing script.

[0066] As a specific embodiment of the present application, when integrating various statements into a program, a data search module for data search, a data parsing and storage module for parsing and storing data, and a data assembly and sending module for data assembly are pre-established. The search statement is then placed in the data search module, the parsing and storage statements are placed in the data parsing and storage module, and the assembly and sending statements are placed in the data assembly and sending module, ultimately resulting in a data processing script.

[0067] As another specific embodiment of the present application, in the process of generating a data processing script, after pre-establishing a data search module, a data parsing and storage module, and a data assembly and sending module, a search statement is generated in the data search module, a parsing statement and a storage statement are generated in the data parsing and storage module, and an assembly statement and a sending statement are generated in the data assembly and sending module.

[0068] In one embodiment of the present application, after the data processing script is generated, when the protocol configuration table is updated, the updated protocol configuration table is read, and then the script generation program is re-run to obtain the updated data processing script.

[0069] That is to say, if there is an update to the protocol configuration table, this application only needs to rerun the current data processing script to update the data processing script, obtain the updated data processing script, and use the updated data processing script to process the data to be processed obtained after the update is completed.

[0070] After the data processing script is generated through the above embodiment, it is necessary to use the data processing script to perform data processing operations on the data to be processed to obtain target data.

[0071] In one embodiment of the present application, the data to be processed is received from the second device, and the specific steps include:

[0072] First, the field length and field name related to the data to be processed are found through the data search module in the data processing script.

[0073] That is, in this embodiment, the data to be processed is generated by the second device and then sent by the second device to the first device. After receiving the data to be processed, the first device first runs a data processing script. Then, through the data search module in the data processing script, using the search statement in the data search module, the telegram name corresponding to the telegram number is found in the data entry matching table. It can be understood that when the data to be processed is a telegram number, the search statement is used to find the telegram name corresponding to the telegram number. After obtaining the telegram name, the data structure configuration table corresponding to the telegram name can be obtained. For example, if the telegram name is blast furnace iron tapping information, then the corresponding blast furnace iron tapping information data structure configuration table is obtained. After finding the blast furnace iron tapping information data structure configuration table, the field length, field type, field description, and field name corresponding to each field are searched.

[0074] Then, the data to be processed is parsed based on the field length through the data parsing and storage module in the data processing script to obtain the target data.

[0075] That is to say, after the first device finds the relevant information, it uses the data parsing and storage module to split the acquired data to be processed according to the field length using the parsing statement to obtain multiple fields, and adds corresponding field names, field descriptions and field types to the multiple fields to obtain the target data.

[0076] Finally, execute S330 to perform subsequent operations on the target data.

[0077] Specifically, the target data is stored in the corresponding data table based on the field name. That is, the storage statement is used according to the table name in the data entry configuration table, and the target data is stored in the table corresponding to the table name.

[0078] In another embodiment of the present application, the data to be processed is generated by the first device, and the specific steps include:

[0079] First, the field length and field name related to the data to be processed are found through the data search module in the data processing script.

[0080] That is to say, in this embodiment, the data to be processed is generated by the first device, and then after the data to be processed is processed into target data, it is sent by the first device to the second device. After generating the data to be processed, the first device first runs the data processing script, and then, through the data search module in the data processing script, the search statement in the data search module is used to find the field description, field type and field length corresponding to the field name in the data structure configuration table. It can be understood that when the data to be processed is the field name and the field itself (for example: the operation identifier is a), the search statement is used to find the field description, field type and field length corresponding to the field name.

[0081] Then, the data to be processed is assembled according to the field length, field name and field type through the data assembly and sending module in the data processing script to obtain the target data.

[0082] That is to say, after the first device finds the relevant information, it uses the data assembly and sending module to splice multiple fields according to the field length using the assembly statement, and matches the corresponding telegram number and telegram name for the spliced multiple fields to obtain the target data.

[0083] Finally, execute S330 to perform subsequent operations on the target data.

[0084] Specifically, the target data is sent to the second device. That is, the target data obtained in this embodiment is sent to the second device, so that the second device performs subsequent data processing operations after receiving the target data.

[0085] The above describes the implementation process of a data processing method performed by the first device in the present application. The following describes a specific embodiment of a data processing method in the present application.

[0086] In the related art, a common development method is for software developers to process the relevant parts of the data structure according to the agreement between the two parties, and manually write the processing logic code for each data item and each field to achieve the purpose of building or parsing the byte stream. There are some automatic code generation technologies on the market, but for programming languages like C++ with complex syntax and flexible construction logic, building a set of code generators is labor-intensive and error-prone, which is not worth the effort. The Lua language is small and lightweight, with simple syntax, and is easy to embed in the C / C++ language. Therefore, it is widely used as an embedded script in the variable parts of the software. Parts involving network communication interfaces often require sending and receiving records. In conventional development solutions, developers cannot avoid the process of developing different codes according to different data structures to ultimately build SQL statements. Due to the different data structures, this process is relatively time-consuming.

[0087] In real-world applications, two systems that need to communicate are likely to share multiple data points. Developers must adapt to different data structures when developing, and the more data they interact with, the greater the workload. For a single piece of data, developers must develop processing logic field by field, following the protocol's data structure. This is cumbersome and prone to errors. Furthermore, data additions, deletions, and modifications may be required at various stages of software development and release. These changes require developers to modify code, making it difficult to create a stable, easy-to-use product without the help of dedicated developers.

[0088] In order to solve the above problems, this application proposes a method of using automatically generated data processing scripts, such as Figure 4 As shown, in the process of receiving the data to be processed, S401 starts, and then S402 C++ is executed to process the socket communication module based on the TPC / IP protocol, S403 C++ is embedded in the Lua virtual machine, and then, according to the data entry configuration table 404 and the data structure configuration table 405, they are input into the Lua script generation program 406 to generate a data processing script 410, and then S407 data processing script is executed to process the application layer protocol related logic to obtain the target data, and finally S408 is executed to execute the subsequent process for the target data, and S409 ends.

[0089] Specifically, a socket communication module based on the TCP / IP protocol is written in C++. This C++ module embeds a Lua virtual machine, which calls the Lua script in an event-triggered manner. Lua completes the processing logic related to the data structure and then calls the C++ module in an event-triggered manner to pass the processed data to the subsequent process. The C++ module is then compiled and solidified into an execution package.

[0090] The process of generating a script is to input a data entry configuration table and a data structure configuration table, and output a Lua script that can be embedded in C++ and run.

[0091] That is to say, if Figure 5 As shown, S501 starts, S502 writes the Lua framework code, S503 reads the memory configuration, that is, reads the data entry configuration table and the data structure configuration table, S504 configures whether the data has been read, if not, executes S505 to write the data processing logic statement, and then continues to execute S503, if so, executes S506 to execute the script generation program, S507 outputs the Lua script, and ends at S508.

[0092] Specifically, the table reading module is used to read the configuration table, and the text writing module is used to write Lua scripts. Then, based on the data processing flow, the Lua framework code is written to form a modular Lua framework. Next, the outer loop reads the data structure configuration table configuration, and the inner loop reads the data structure configuration table configuration and stores it in memory. Then, the configuration table information in memory is read in a loop and processed in a loop to fill the Lua script framework with content.

[0093] For example: Based on the message name and table name of the data structure configuration table, and based on the field name, field type, and field length of the data structure configuration table, SQL statements are generated to establish a data table for sending and receiving history. Based on the data structure configuration table, a table statement for the correspondence between the external data index (message number) and the internal data index is generated. Based on the data structure configuration table, a statement for parsing the byte stream of received data is generated. Based on the data structure configuration table, a statement for assembling the byte stream of sent data is generated. Based on the data structure configuration table and the data structure configuration table, an SQL statement for storing the history of sent and received data is generated to realize record storage. Execute the script generation program, output the Lua script file, deploy the Lua script in the deployment environment, and pass it through the application. When adding, deleting, or modifying the data structure, the new interface protocol data structure (i.e., the data entry configuration table and the data structure configuration table) is used as the input for the script generation program, execute the script generation program, output the Lua script file, deploy the Lua script in the deployment environment, and pass it through the application.

[0094] Therefore, the primary protection point of this application lies in embedding Lua scripts to process data structure-related logic when developing a network communication interface based on C++. Lua scripts are automatically generated using a script generator. The script generator's primary inputs are a data entry configuration table and a data structure configuration table. The script generator must first read all configurations into memory. The script generator then loops through the configurations in memory and generates script statements for the processing logic.

[0095] Therefore, this application only uses the Lua language to process some functions related to data structures, and does not change the other advantages of using C++ to develop network communication interfaces. Since the data structure-related parts are transferred to Lua scripts for processing, the C++ modules can be solidified and do not need to be frequently modified and recompiled, making software management simpler. Due to the simple syntax of the Lua language, the logic of generating Lua statements according to certain rules is relatively simple, and it is simple and feasible to build a script generation program. For systems with multiple interactive data, the workload of building a script generation program will not exceed the workload of directly writing Lua scripts. Under the existing C++-based network communication interface development model, the development workload is proportional to the amount of data. Using this application, the computer loop reads the data item configuration and the field item configuration, and automatically loops to generate Lua processing statements. The time required to develop a large number of data interfaces and generate scripts is negligible. Under the existing C++-based network communication interface development model, developers develop processing logic statements data item by data item and field by field. The work content is cumbersome and prone to errors, which increases the workload for subsequent testing. The present application solution uses a computer to read the configuration and output the processing logic, which will not make mistakes. In existing network communication interface development methods, developers rely on interface protocol data structure configuration tables. With this technical solution, the primary input data is also the interface protocol data structure configuration, which eliminates the need for additional work compared to conventional development models. Adding, deleting, or modifying data structures eliminates the need for developer involvement; anyone with basic computer knowledge can directly configure and use the interface. This frees up developers and significantly improves productivity.

[0096] A specific embodiment of a data processing method of the present application is described above, and an apparatus for the data processing method will be described below.

[0097] like Figure 6 As shown, some embodiments of the present application provide a data processing device 600 , which includes: a data acquisition module 610 , a script execution module 620 and a data operation module 630 .

[0098] The data acquisition module 610 is configured to acquire data to be processed, wherein the data to be processed is received from the second device, or the data to be processed is generated by the first device; the script running module 620 is configured to run a data processing script, and use the data processing script to perform data processing operations on the data to be processed to obtain target data, wherein the data processing script is automatically generated by the script generation program through the protocol configuration table, and the protocol configuration table is used to store communication information between the first device and the second device, and the communication information at least includes the correspondence between the telegram name and the telegram number; the data operation module 630 is configured to perform subsequent operations on the target data.

[0099] In one embodiment of the present application, the script generation module is configured to: generate a search statement for searching for content corresponding to the data to be processed according to the i-th row in the protocol configuration table; generate a parsing statement for parsing the data to be processed according to the i-th row in the protocol configuration table, and generate an assembly statement for assembling the data to be processed according to the i-th row in the protocol configuration table; generate a sending statement for sending the target data, and generate a storage statement for storing the target data according to the i-th row in the protocol configuration table; repeat the above steps until the protocol configuration table is read, integrate the search statement, the parsing statement, the assembly statement, the sending statement and the storage statement into a program, and obtain the data processing script.

[0100] In one embodiment of the present application, the script generation module is further configured to: establish a data search module, a data parsing and storage module, and a data assembly and sending module; integrate the search statement into the data search module, integrate the parsing statement and the storage statement into the data parsing and storage module, and integrate the assembly statement and the sending statement into the data assembly and sending module to obtain the data processing script.

[0101] In one embodiment of the present application, the script generation module is further configured to: when the protocol configuration table is updated, read the updated protocol configuration table; and re-run the script generation program to obtain the updated data processing script.

[0102] In one embodiment of the present application, the data to be processed is received from a second device; the script execution module 620 is further configured to: find the field length and field name related to the data to be processed through the data search module in the data processing script; parse the data to be processed based on the field length through the data parsing and storage module in the data processing script to obtain the target data; the subsequent operation on the target data includes: storing the target data in a corresponding data table based on the field name.

[0103] In one embodiment of the present application, the data to be processed is generated by the first device; the script running module 620 is also configured to: find the field length and field name related to the data to be processed through the data search module in the data processing script; assemble the data to be processed according to the field length, field name and field type through the data assembly and sending module in the data processing script to obtain the target data; the subsequent operation on the target data includes: sending the target data to the second device.

[0104] In the embodiments of this application, Figure 4 The modules shown can achieve Figure 1 、 Figure 2 and Figure 3 Each process in the method embodiment. Figure 4 The operations and / or functions of each module in Figure 1 、 Figure 2 and Figure 3 For details, please refer to the description in the above method embodiment. To avoid repetition, detailed description is appropriately omitted here.

[0105] like Figure 7 As shown, an embodiment of the present application provides an electronic device 700, including: a processor 710, a memory 720 and a bus 730, wherein the processor is connected to the memory via the bus, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement a method as described in any one of the above embodiments. For details, please refer to the description in the above method embodiments. To avoid repetition, detailed description is appropriately omitted here.

[0106] Among them, the bus is used to realize the direct connection and communication of these components. Among them, the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0107] The memory may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory stores computer-readable instructions. When the processor executes the computer-readable instructions, the method described in the above embodiment may be executed.

[0108] I understand. Figure 7 The structure shown is for illustration only and may also include Figure 7 More or fewer components than shown, or with Figure 7 Different configurations shown. Figure 7 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0109] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a server, any of the methods described in all the above-mentioned embodiments is implemented. For details, please refer to the description in the above-mentioned method embodiments. To avoid repetition, the detailed description is appropriately omitted here.

[0110] The foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0111] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data processing method, characterized in that: Applied to a first device, the method includes: Acquiring data to be processed, wherein the data to be processed is received from the second device, or the data to be processed is generated by the first device; Run a data processing script, and use the data processing script to perform a data processing operation on the data to be processed to obtain target data, wherein the data processing script is automatically generated by a script generation program through a protocol configuration table, the protocol configuration table is used to store communication information between the first device and the second device, the communication information at least including a correspondence between a message name and a message number, the protocol configuration table includes a data entry configuration table and a data structure configuration table, the data entry configuration table is used to store a message name and a message identifier transmitted between the first device and the second device, and a table name for storing the corresponding message on the first device, the data structure configuration table is used to store a field length, field name, and field description used for parsing the data to be processed or combining the data to be processed, and the number of the data structure configuration tables is the same as the number of message names in the data entry configuration table; performing subsequent operations on the target data; The subsequent operation on the target data includes: storing the target data in a corresponding data table based on a field name, sending the target data to the second device, and performing data processing on the target data through the second device.

2. The method according to claim 1, characterized in that Before running the data processing script and using the data processing script to perform data processing operations on the data to be processed to obtain target data, the method further includes: S1: generating a search statement for searching for content corresponding to the data to be processed according to the i-th row in the protocol configuration table; S2: generating a parsing statement for parsing the data to be processed according to the i-th row in the protocol configuration table, and generating an assembling statement for assembling the data to be processed according to the i-th row in the protocol configuration table; S3: Generate a sending statement for sending the target data, and generate a storage statement for storing the target data according to the i-th row in the protocol configuration table; Repeat the above steps S1-S3 until the protocol configuration table is read, integrate the search statement, the parsing statement, the assembly statement, the sending statement and the storage statement into a program to obtain the data processing script.

3. The method according to claim 2, characterized in that The step of integrating the search statement, the parsing statement, the assembling statement, the sending statement, and the storing statement into a program to obtain the data processing script includes: Establish data search module, data analysis and storage module and data assembly and sending module; The search statement is integrated into the data search module, the parsing statement and the storage statement are integrated into the data parsing storage module, and the assembling statement and the sending statement are integrated into the data assembling and sending module to obtain the data processing script.

4. The method according to claim 3, characterized in that After obtaining the data processing script, the method further includes: When the protocol configuration table is updated, reading the updated protocol configuration table; Re-run the script generation program to obtain an updated data processing script.

5. The method according to any one of claims 1 to 4, characterized in that The data to be processed is received from the second device; The step of performing a data processing operation on the data to be processed using the data processing script to obtain target data includes: Find the field length and field name related to the data to be processed through the data search module in the data processing script; Parsing the data to be processed based on the field length through the data parsing and storage module in the data processing script to obtain the target data; The subsequent operation on the target data includes: The target data is stored in a corresponding data table based on the field name.

6. The method according to any one of claims 1 to 4, characterized in that The data to be processed is generated by the first device; The step of performing a data processing operation on the data to be processed using the data processing script to obtain target data includes: Find the field length and field name related to the data to be processed through the data search module in the data processing script; The data to be processed is assembled according to the field length, field name and field type by the data assembly and sending module in the data processing script to obtain the target data; The subsequent operation on the target data includes: The target data is sent to the second device.

7. A data processing device, characterized in that: Applied to a first device, the apparatus includes: a data acquisition module, configured to acquire data to be processed, wherein the data to be processed is received from the second device, or the data to be processed is generated by the first device; a script execution module, configured to execute a data processing script and perform data processing operations on the data to be processed using the data processing script to obtain target data, wherein the data processing script is automatically generated by a script generation program through a protocol configuration table, the protocol configuration table being used to store communication information between the first device and the second device, the communication information at least including a correspondence between a message name and a message number, the protocol configuration table comprising a data entry configuration table and a data structure configuration table, the data entry configuration table being used to store a message name and message identifier transmitted between the first device and the second device, and a table name for storing the corresponding message on the first device, the data structure configuration table being used to store field lengths, field names, and field descriptions used for parsing or combining the data to be processed, and the number of the data structure configuration tables being the same as the number of message names in the data entry configuration table; a data operation module, configured to perform subsequent operations on the target data; The data operation module is specifically configured to: store the target data in a corresponding data table based on a field name, send the target data to the second device, and perform data processing on the target data through the second device.

8. The device according to claim 7, characterized in that The script generation module is configured to: S1: generating a search statement for searching for content corresponding to the data to be processed according to the i-th row in the protocol configuration table; S2: generating a parsing statement for parsing the data to be processed according to the i-th row in the protocol configuration table, and generating an assembling statement for assembling the data to be processed according to the i-th row in the protocol configuration table; S3: Generate a sending statement for sending the target data, and generate a storage statement for storing the target data according to the i-th row in the protocol configuration table; Repeat the above steps S1-S3 until the protocol configuration table is read, integrate the search statement, the parsing statement, the assembly statement, the sending statement and the storage statement into a program to obtain the data processing script.

9. An electronic device, characterized in that: include: processor, memory, and bus; The processor is connected to the memory via the bus, the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 6 can be implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed, can implement the method according to any one of claims 1 to 6.

Images