A message processing system and method based on Internet and data analysis technology

By abstracting the message structure into a data structure and adopting a modular design, the problem of code duplication and maintenance difficulty in the existing technology is solved, the flexibility and efficiency of message processing are achieved, and the development efficiency and scalability are improved.

CN118283146BActive Publication Date: 2024-11-05JINDIAN TECH DEV CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410413841.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-08
Publication Date
2024-11-05
Estimated Expiration
2044-04-08

AI Technical Summary

Technical Problem

When processing packets in the existing technology, developers need to write a lot of duplicate code, resulting in inefficient development and waste of resources, and the code base structure is chaotic, difficult to maintain and upgrade, which increases the difficulty of development and maintenance.

Method used

By abstracting the message structure into a data structure, defining the message area and data items, setting association rules, realizing the analysis, verification, encryption and signature of the message, adopting modular design, and configuring the interface to achieve customized configuration, reducing code Repeat and improve scalability.

Benefits of technology

It realizes the flexibility and efficiency of message processing, reduces code duplication, improves development efficiency and maintenance, and facilitates the expansion and modification of existing processing processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118283146B_ABST
    Figure CN118283146B_ABST
Patent Text Reader

Abstract

The invention discloses a message processing system and method based on the Internet and data analysis technology, and relates to the field of message data exchange. The structure of the message is abstracted into a message data structure; the original message data is analyzed, verified, encrypted and signed, and finally stored in a database. The invention abstracts the message structure into a data structure, reads the corresponding message data through a configuration interface, analyzes the message file, and then verifies the parsed message data to ensure the correctness and integrity of the data; assembles the internal data structure into message data for subsequent processing and transmission; encrypts and signs the assembled message data to ensure the confidentiality and integrity of the data; and finally stores the processed message data in a database. The modular design is adopted, and developers can flexibly define the message structure, analysis rules, verification rules, etc. according to actual needs without repeatedly writing codes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of message data exchange, and in particular, relates to a message processing system and method based on the Internet and data parsing technology. Background Art

[0002] A message is the basic unit of data exchange in a computer network. It usually consists of a message header and a message body. The message header contains some metadata information, such as the source address, destination address, protocol version, etc., which is used to indicate the sending and receiving information of the message; the message body contains the actual data to be transmitted; in network communication, when one host sends data to another host, it organizes the data into the form of a message and transmits it to the receiver through the network. After receiving the message, the receiver will parse the information in the message header and extract the data in the message body; messages can be transmitted using different protocols, such as HTTP, TCP, UDP, etc. Different protocols specify the format, organization and transmission rules of messages; messages play a very important role in network communication. They carry the transmission and exchange of data and ensure that data can be effectively transmitted and received in the network.

[0003] Chinese patent CN117643016A discloses message processing, including receiving a first message and forwarding a second message, wherein the second message includes a first segment routing header SRH and the first message, and the first SRH includes tag information, and the tag information indicates the encapsulation basis of the first SRH, so that the intermediate node can accurately obtain the original message.

[0004] In the prior art, when processing messages, developers need to write a large amount of duplicate code for different types of messages, which leads to low development efficiency and waste of resources. At the same time, the implementation method of each message processing requirement may be different, resulting in a chaotic code structure in the code base, which is difficult to maintain and upgrade. When the existing message processing process needs to be expanded or modified, the code of multiple modules may need to be modified, which increases the difficulty of development and maintenance. Summary of the invention

[0005] In view of the problems in the related technology, the present invention proposes a message processing system and method based on the Internet and data parsing technology to overcome the above-mentioned technical problems existing in the existing related technology.

[0006] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] The present invention is a message processing method based on the Internet and data analysis technology, comprising the following steps:

[0008] S1, abstracting the structure of the message into a message data structure; the message data structure includes regions constituting the message, obtaining a message region set; each message region in the message region set includes data items, obtaining a message data item matrix; association rules are set between each region of the message region set, obtaining a message association rule matrix;

[0009] S2, reading the original message from the outside into the memory to obtain the original message data; the outside includes a file; the original message data is parsed according to the message area set, the message data item matrix and the message association rule matrix to obtain the parsed message data; the parsed message data is filled into the message data structure to obtain the initial message data;

[0010] S3, performing integrity check and rule check on each row of data items in the initial message data, and obtaining a final check result after the check is completed; when the final check result is successful, converting the initial message data to obtain message file data; when the check result is failed, generating a failure feedback message according to the initial message data;

[0011] S4, encrypting and signing the message file data to obtain processed message file data; storing the processed message file data in a database; when receiving the encrypted message file data, decrypting the encrypted message file data and verifying the signature, and storing the message file data in the database after the decryption and signature verification are completed;

[0012] The message structure is abstracted into a data structure, that is, the entire message is divided into areas, and the data items of the message are placed in the areas. There is a correlation between the areas. By defining the meta information of the message structure, including which areas the message is composed of, which data items each area has, the data type and length of each data item, etc., in the subsequent message processing flow, the system can parse and process the message according to these definitions; then specify the message format, message encoding, message meta information, and message storage path through the configuration interface. The system will read the corresponding message data according to the configuration information and define the meta message information of the message parsing through the configuration interface. The system will parse the message file according to the message meta information and parse the message file into internal data structure for subsequent processing; verify the parsed message data to ensure the correctness and integrity of the data; define the meta information generated by the message through the configuration interface, including field mapping relationship, data type conversion, etc.; the system will assemble the internal data structure into message data according to these rules for subsequent processing and transmission; encrypt and sign the assembled message data to ensure the confidentiality and integrity of the data; select the signature algorithm and key through the configuration interface, and the system will sign the message data according to the configuration information; decrypt the received message data and verify the signature to ensure the confidentiality and integrity of the data, then encrypt the message data that needs to be kept confidential, and finally store the processed message data in the database.

[0013] Preferably, the S1 comprises the following steps:

[0014] S11, abstracting the structure of the message into a message data structure; the message data structure includes the areas constituting the message, and obtaining a message area set a={a1, a2, ..., a i ,...,a a′}, a i is the ith region divided in the message data structure, is the total number of regions divided in the message data structure;

[0015] S12. Each message area in the message area set contains a data item, and the data item includes a data type and a length, and a message data item matrix a″ is obtained as follows:

[0016]

[0017] Where: a″ ij Indicates the jth data item in the i-th area in the message data structure; a″′1, a″′2, ..., a″′ i , ..., a″′ a′ Respectively represent the total number of data items in the 1st, 2nd, ..., ith, ..., a′th area in the message data structure;

[0018] Association rules are set between each area of ​​the message area set, and a message association rule matrix b is obtained, which is as follows:

[0019]

[0020] Where: b ij Represents an association rule between the ith area and the jth area in the message data structure;

[0021] By defining the meta information of the message structure, including the areas of the message, the data items in each area, the data type and length of each data item, etc., in the subsequent message processing flow, the system can parse and process the message according to these definitions.

[0022] Preferably, S2 comprises the following steps:

[0023] S21, read the original message from the outside into the memory to obtain the original message data;

[0024] S22, parsing the original message data to obtain parsed sub-message data; according to the message area set a={a1, a2, ..., a i ,...,a a′}, the message data item matrix a″ and the message association rule matrix b divide the parsed sub-message data to obtain the parsed message data;

[0025] S23, filling the parsed message data into the message data structure to obtain initial message data;

[0026] By adopting the above solution to parse the original message data, the parsed message data is finally obtained, and the message file is parsed into an internal data structure to facilitate subsequent processing.

[0027] Preferably, the S22 comprises the following steps:

[0028] S221, reading the Header part of the original message data and parsing it to obtain a message sub-message; reading the Header part of the message sub-message to obtain the Header of the message sub-message;

[0029] S222. Parse the Header of the message sub-message to obtain the id value and length value of the message sub-message;

[0030] S223, set sub-message type set b'={b'1, b'2, ..., b' i ,...,b′ b″} and the corresponding set of parsing methods b″′={b″′1,b″′2,...,b″′ i ,...,b″′ b″}, b′ i is the i-th sub-message type, b″ is the total number of sub-message types, b″′ i b′ i The parsing method corresponding to the type sub-message; according to the id value of the message sub-message and the sub-message type set b′={b′1,b′2,...,b′ i ,...,b′ b″} Determine the type of the message sub-message and obtain the sub-message type;

[0031] S224, extract the length value of the message sub-message from the message sub-message to obtain the sub-message data; then according to the sub-message type and parsing method set b″′={b″′1,b″′2,...,b″′ i ,...,b″′ b″} Parsing the sub-message data to obtain parsed sub-message data;

[0032] S225, according to the message area set a={a1, a2, ..., a i ,...,a a′}, the message data item matrix a″ and the message association rule matrix b divide the parsed sub-message data to obtain the parsed message data;

[0033] By parsing the Header part of the original message data to obtain the Header of the message sub-message, and then parsing the Header of the message sub-message to obtain the id value and length value of the message sub-message, the type of the sub-message of the original message data is determined, and then the corresponding parsing method is used for parsing, thereby improving the accuracy and efficiency of the parsing.

[0034] Preferably, S3 comprises the following steps:

[0035] S31, using a cyclic redundancy check algorithm to perform integrity check on each row of data items in the initial message data, and obtaining a first check result after the check is completed;

[0036] S32. When the first verification result is successful, perform rule verification on each row of data items in the initial message data to obtain a second verification result; when the first verification result or the second verification result is failed, the final verification result is failed; when the first verification result and the second verification result are both successful, the final verification result is successful;

[0037] S33, when the final verification result is successful, converting the initial message data to obtain message file data; when the final verification result is failed, generating a failure feedback message according to the initial message data;

[0038] The parsed message data is checked for integrity and correctness to ensure the correctness and integrity of the data.

[0039] Preferably, the S31 comprises the following steps:

[0040] S311, set the data item set consisting of each row of data items in the initial message data to c = {c1, c2, ..., c i ,...,c c′},c i is the i-th data item in the initial message data, c′ is the total number of data items in the initial message data;

[0041] S312, obtain c={c1, c2, ..., c i ,...,c c′}, and obtain the data item bit column set c″={c″1,c″2,...,c″ i ,...,c″ c′},c″ i is the bit column of the i-th data item in the initial message data; and the data item bit column set c″={c″1,c″2,...,c″ i ,...,c″ c′}, each digit in each bit column is used as the coefficient of the polynomial, and the bit column polynomial set c″′={c″′1,c″′2,...,c″′ i ,...,c″′ c},c″′ i A polynomial formed by the bit sequence of the i-th data item in the initial message data;

[0042] S313, set the prepolynomial d, and set the bit sequence polynomial set c″′={c″′1, c″′2, ..., c″′ i ,...,c″′ c′} is divided by d to obtain the first remainder polynomial set d′={d′1,d′2,...,d′ i ,...,d′ c′}, set the second remainder polynomial set d″={d″1,d″2,...,d″ obtained by using the prepolynomial d as the divisor when the initial message data is complete i ,...,d″ c′}, when the first remainder polynomial set d′={d′1,d′2,...,d′ i ,...,d′ c′} and the second remainder polynomial set d″={d″1,d″2,...,d″ i ,...,d′ c′} are exactly the same, the initial message data is complete, and the first verification result is success; otherwise, the initial message data is incomplete, and the first verification result is failure;

[0043] By converting the data items in the initial message data into a polynomial, dividing the polynomial by the same polynomial, and then comparing whether the remainders are the same, it is possible to determine whether the data items in the initial message data have been changed, thereby verifying the integrity of the data items in the initial message data.

[0044] Preferably, the S32 comprises the following steps:

[0045] S321. When the first verification result is successful, perform a rule verification on each row of data items in the initial message data to obtain a second verification result; performing a rule verification on each row of data items in the initial message data includes the following steps:

[0046] S3211, set verification rule set d″′={d″′1,d″′2,...,d″′ i ,...,d″′ e},d″′ i is the i-th verification rule, and e is the total number of verification rules;

[0047] S3212, according to c={c1,c2,...,c i ,...,c c′} and data item verification rule set d″′={d″′1,d″′2,...,d″′ i ,...,d″′ e The data item verification rule matrix formed by the verification rules mounted on each data item in the setting is e′, as follows:

[0048]

[0049] Where: e′ ij For c = {c1, c2, ..., c i ,...,c c′}, the jth validation rule mounted on the i-th data item, e″ i For c = {c1, c2, ..., c i ,...,c c′The total number of validation rules mounted on the i-th data item in};

[0050] S3213, c={c1,c2,...,c i ,...,c c′} is substituted into the verification rules mounted therein to obtain the correctness verification result matrix e″′, as follows:

[0051]

[0052] Where: e″′ ij For c = {c1, c2, ..., c i ,...,c c′ The verification result of the i-th data item in} after being verified by the j-th verification rule mounted on it; the data item correctness verification result judgment rule is set as follows:

[0053] When e″′ i1 , e″′ i2 , ..., e″′ ij ,...and If all the values ​​of are successful, then c={c1,c2,...,c i ,...,c c′}, the verification result of the i-th data item in is successful, otherwise, c={c1,c2,...,c i ,...,c c′ The verification result of the i-th data item in} is failed;

[0054] According to the data item correctness check result judgment rule, the initial message data correctness check result set f = {f1, f2, ..., f i ,...,f c′}, f i is the correctness check result of the i-th data item; when f1, f2, ..., f i , ... and f c′ If all are successful, the second verification result is successful, otherwise, the second verification result is failed;

[0055] S322: When the first verification result or the second verification result is a failure, the final verification result is a failure; when the first verification result and the second verification result are both successful, the final verification result is a success;

[0056] Use custom expressions as validation rules for data items. Multiple validation rules can be attached to each data item. During validation, substitute the value of the data item obtained from message parsing into the expression, execute the expression, and obtain the validation result as the validation result of this data item.

[0057] Preferably, S4 comprises the following steps:

[0058] S41, encrypting and signing the message file data to obtain processed message file data; storing the processed message file data in a database;

[0059] S42, when the encrypted message file data is received, the encrypted message file data is decrypted and the signature is verified, and after the decryption and signature verification are completed, the message file data is stored in the database;

[0060] Select the decryption algorithm and key through the configuration interface, and the system will decrypt the encrypted message data according to the configuration information; then select the signature verification algorithm through the configuration interface, and the system will verify the signature of the received message data according to the configuration information. Finally, specify the mapping relationship between the message structure meta-information and the database table fields through the configuration interface. The system will generate the SQL statement for database insertion based on the mapping relationship information for the processed message data and store the message data in the database.

[0061] Preferably, the S41 comprises the following steps:

[0062] S411, set encryption algorithm set f′={f′1,f′2,...,f′ i ,...,f″′ f′} and signature algorithm set f″′={f″′1,f″′2,...,f″′ i ,...,f″′ g}, f′ i represents the i-th encryption algorithm in the encryption algorithm set, f″ represents the total number of encryption algorithms in the encryption algorithm set; f″′ i represents the i-th signature algorithm in the signature algorithm set, and g represents the total number of signature algorithms in the signature algorithm set;

[0063] S412, first from the encryption algorithm set f′={f′1,f′2,...,f′ i ,...,f′ f′} select an encryption algorithm to encrypt the message file data, and after the encryption is completed, the encrypted message file data is obtained; then select a signature algorithm set f″′={f″′1,f″′2,...,f″′ i ,...,f″′ g} select a signature algorithm to perform a signature operation on the encrypted message file data, and after the signature is completed, obtain the processed message file data;

[0064] Select the encryption algorithm through the configuration interface, and the system will encrypt the message data according to the configuration information.

[0065] A message processing system based on the Internet and data analysis technology, including a message structure definition module, a message reading module, a message analysis module, a message verification module, a message generation module, a message signature module, a message signature verification module, a message encryption module, a message decryption module and a message storage module;

[0066] The message structure definition module is used to abstract the structure of the message into a message data structure;

[0067] The message reading module is used to read the original message from the outside into the memory;

[0068] The message parsing module is used to parse the message data according to the message area set, the message data item matrix and the message association rule matrix;

[0069] The message verification module is used to perform integrity verification and rule verification on each row of data items in the initial message data;

[0070] The message generation module is used to convert the initial message data to obtain message file data;

[0071] The message signature module is used to perform a signature operation on the message file data;

[0072] The message signature verification module is used to verify the signature of the encrypted message file data;

[0073] The message encryption module is used to perform encryption operation on the message file data;

[0074] The message decryption module is used to decrypt the encrypted message file data;

[0075] The message storage module is used to store the processed message file data into a database.

[0076] The present invention has the following beneficial effects:

[0077] 1. In the present invention, by setting a message structure definition module, a message reading module, a message parsing module, a message verification module, a message generation module, a message signature module, a message verification module, a message encryption module, a message decryption module and a message storage module, the message structure is abstracted into a data structure, that is, the entire message is divided into regions, the data items of the message are placed in the regions, and there is a correlation between the regions. By defining the meta information of the message structure, including which regions the message is composed of, which data items each region has, the data type and length of each data item, etc., in this way, in the subsequent message processing flow, the system can parse and process the message according to these definitions; then specify the format, message encoding, message meta information, and message storage path of the message through the configuration interface, the system will read the corresponding message data according to the configuration information and define the meta message information of the message parsing through the configuration interface, the system will parse the message file according to the message meta information, and parse the message file into an internal data structure for subsequent processing; the parsed message data is verified to ensure that the data The system ensures the correctness and integrity of the message; defines the meta information generated by the message through the configuration interface, including field mapping relationship, data type conversion, etc.; the system will assemble the internal data structure into message data according to these rules for subsequent processing and transmission; encrypt and sign the assembled message data to ensure the confidentiality and integrity of the data; select the signature algorithm and key through the configuration interface, and the system will sign the message data according to the configuration information; decrypt and verify the signature of the received message data to ensure the confidentiality and integrity of the data, and then encrypt the message data that needs to be kept confidential, and finally store the processed message data in the database. It adopts modular design and realizes customized configuration of the message processing process through the configuration interface, so that developers can flexibly define the message structure, parsing rules, verification rules, etc. according to actual needs without repeatedly writing code. At the same time, it adopts modular design and has strong scalability. When the existing message processing process needs to be expanded or modified, it only needs to configure the corresponding functional modules without modifying the code of other modules.

[0078] 2. In the present invention, the Header part of the original message data is parsed to obtain the Header of the message sub-message, and then the Header of the message sub-message is parsed to obtain the ID value of the message sub-message and the length value of the message sub-message, the type of the sub-message of the original message data is determined, and then the corresponding parsing method is used for parsing, thereby improving the accuracy and efficiency of the parsing.

[0079] 3. In the present invention, by converting the data items in the initial message data into a polynomial, dividing the polynomial by the same polynomial, and then comparing whether the remainders are the same, it is possible to determine whether the data items in the initial message data have been changed, thereby verifying the integrity of the data items in the initial message data.

[0080] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.

[0082] Figure 1 The present invention is a flowchart of a message processing system based on the Internet and data analysis technology for processing messages. DETAILED DESCRIPTION

[0083] The following will be combined with the drawings in the embodiments of the invention to clearly and completely describe the technical solutions in the embodiments of the invention. Obviously, the described embodiments are only part of the embodiments of the invention, not all of the embodiments. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the invention.

[0084] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0085] The present invention is a message processing method based on the Internet and data analysis technology, comprising the following steps:

[0086] S1, abstracting the structure of the message into a message data structure; the message data structure includes regions constituting the message, obtaining a message region set; each message region in the message region set includes data items, obtaining a message data item matrix; association rules are set between each region of the message region set, obtaining a message association rule matrix;

[0087] The S1 comprises the following steps:

[0088] S11, abstracting the structure of the message into a message data structure; the message data structure includes the areas constituting the message, and obtaining a message area set a={a1, a2, ..., a i ,...,a a′}, a i is the ith region divided in the message data structure, is the total number of regions divided in the message data structure;

[0089] S12. Each message area in the message area set contains a data item, and the data item includes a data type and a length, and a message data item matrix a″ is obtained as follows:

[0090]

[0091] Where: a″ ij Indicates the jth data item in the i-th area in the message data structure; a″′1, a″′2, ..., a″′ i , ..., a″′ a Respectively represent the total number of data items in the 1st, 2nd, ..., ith, ..., a′th area in the message data structure;

[0092] Association rules are set between each area of ​​the message area set, and a message association rule matrix b is obtained, which is as follows:

[0093]

[0094] Where: b ij Represents an association rule between the ith area and the jth area in the message data structure;

[0095] S2, reading the original message from the outside into the memory to obtain the original message data; the outside includes a file; the original message data is parsed according to the message area set, the message data item matrix and the message association rule matrix to obtain the parsed message data; the parsed message data is filled into the message data structure to obtain the initial message data;

[0096] The S2 comprises the following steps:

[0097] S21, read the original message from the outside into the memory to obtain the original message data;

[0098] S22, parsing the original message data to obtain parsed sub-message data; according to the message area set a={a1, a2, ..., a i ,...,a a′}, the message data item matrix a″ and the message association rule matrix b divide the parsed sub-message data to obtain the parsed message data;

[0099] The S22 comprises the following steps:

[0100] S221, reading the Header part of the original message data and parsing it to obtain a message sub-message; reading the Header part of the message sub-message to obtain the Header of the message sub-message;

[0101] S222. Parse the Header of the message sub-message to obtain the id value and length value of the message sub-message;

[0102] S223, set sub-message type set b'={b'1, b'2, ..., b' i ,...,b′ b′} and the corresponding set of parsing methods b″′={b″′1,b″′2,...,b″′ i ,...,b″′ b″}, b′ i is the i-th sub-message type, b″ is the total number of sub-message types, b″′ i b′ i The parsing method corresponding to the type sub-message; according to the id value of the message sub-message and the sub-message type set b′={b′1,b′2,...,b′ i ,...,b′ b″} Determine the type of the message sub-message and obtain the sub-message type;

[0103] S224, extract the length value of the message sub-message from the message sub-message to obtain the sub-message data; then according to the sub-message type and parsing method set b″′={b″′1,b″′2,...,b″′ i ,...,b″′ b″} Parsing the sub-message data to obtain parsed sub-message data;

[0104] S225, according to the message area set a={a1, a2, ..., a i ,...,a a′}, the message data item matrix a″ and the message association rule matrix b divide the parsed sub-message data to obtain the parsed message data;

[0105] S23, filling the parsed message data into the message data structure to obtain initial message data;

[0106] S3, performing integrity check and rule check on each row of data items in the initial message data, and obtaining a final check result after the check is completed; when the final check result is successful, converting the initial message data to obtain message file data; when the check result is failed, generating a failure feedback message according to the initial message data;

[0107] The S3 comprises the following steps:

[0108] S31, using a cyclic redundancy check algorithm to perform integrity check on each row of data items in the initial message data, and obtaining a first check result after the check is completed;

[0109] The S31 comprises the following steps:

[0110] S311, set the data item set consisting of each row of data items in the initial message data to c = {c1, c2, ..., c i ,...,c c′},c i is the i-th data item in the initial message data, c′ is the total number of data items in the initial message data;

[0111] S312, obtain c={c1, c2, ..., c i ,...,c c′}, and obtain the data item bit column set c″={c″1,c″2,...,c″ i ,...,c″ c′},c″ i is the bit column of the i-th data item in the initial message data; and the data item bit column set c″={c″1,c″2,...,c″ i ,...,c″ c′}, each digit in each bit column is used as the coefficient of the polynomial, and the bit column polynomial set c″′={c″′1,c″′2,...,c″′ i ,...,c″′ c′},c″′ i A polynomial formed by the bit sequence of the i-th data item in the initial message data;

[0112] S313, set the prepolynomial d, and set the bit sequence polynomial set c″′={c″′1, c″′2, ..., c″′ i ,...,c″′ c′} is divided by d to obtain the first remainder polynomial set d′={d′1,d′2,...,d′ i ,...,d′ c′}, set the second remainder polynomial set d″={d″1,d″2,...,d″ obtained by using the prepolynomial d as the divisor when the initial message data is complete i ,...,d″ c′}, when the first remainder polynomial set d′={d′1,d′2,...,d′ i ,...,d′ c′} and the second remainder polynomial set d″={d″1,d″2,...,d″ i ,...,d″ c′} are exactly the same, the initial message data is complete, and the first verification result is success; otherwise, the initial message data is incomplete, and the first verification result is failure;

[0113] S32. When the first verification result is successful, perform rule verification on each row of data items in the initial message data to obtain a second verification result; when the first verification result or the second verification result is failed, the final verification result is failed; when the first verification result and the second verification result are both successful, the final verification result is successful;

[0114] The S32 comprises the following steps:

[0115] S321. When the first verification result is successful, perform a rule verification on each row of data items in the initial message data to obtain a second verification result; performing a rule verification on each row of data items in the initial message data includes the following steps:

[0116] S3211, set verification rule set d″′={d″′1,d″′2,...,d″′ i ,...,d″′ e},d″′ i is the i-th verification rule, and e is the total number of verification rules;

[0117] S3212, according to c={c1,c2,...,c i ,...,c c′} and data item verification rule set d″′={d″′1,d″′2,...,d″′ i ,...,d″′ e The data item verification rule matrix formed by the verification rules mounted on each data item in the setting is e′, as follows:

[0118]

[0119] Where: e′ ij For c = {c1, c2, ..., c i ,...,cc′}, the jth validation rule mounted on the i-th data item, e″ i For c = {c1, c2, ..., c i ,...,c c′ The total number of validation rules mounted on the i-th data item in};

[0120] S3213, c={c1,c2,...,c i ,...,c c′} is substituted into the verification rules mounted therein to obtain the correctness verification result matrix e″′, as follows:

[0121]

[0122] Where: e″′ ij For c = {c1, c2, ..., c i ,...,c c′ The verification result of the i-th data item in} after being verified by the j-th verification rule mounted on it; the data item correctness verification result judgment rule is set as follows:

[0123] When e″′ i1 , e″′ i2 , ..., e″′ ij ,...and If all the values ​​of are successful, then c={c1,c2,...,c i ,...,c c′}, the verification result of the i-th data item in is successful, otherwise, c={c1,c2,...,c i ,...,c c′ The verification result of the i-th data item in} is failed;

[0124] According to the data item correctness check result judgment rule, the initial message data correctness check result set f = {f1, f2, ..., f i ,...,f c′}, f i is the correctness check result of the i-th data item; when f1, f2, ..., f i , ... and f c′ If all are successful, the second verification result is successful, otherwise, the second verification result is failed;

[0125] S322: When the first verification result or the second verification result is a failure, the final verification result is a failure; when the first verification result and the second verification result are both successful, the final verification result is a success;

[0126] S33, when the final verification result is successful, converting the initial message data to obtain message file data; when the final verification result is failed, generating a failure feedback message according to the initial message data;

[0127] S4, encrypting and signing the message file data to obtain processed message file data; storing the processed message file data in a database; when receiving the encrypted message file data, decrypting the encrypted message file data and verifying the signature, and storing the message file data in the database after the decryption and signature verification are completed;

[0128] The S4 comprises the following steps:

[0129] S41, encrypting and signing the message file data to obtain processed message file data; storing the processed message file data in a database;

[0130] The S41 comprises the following steps:

[0131] S411, set encryption algorithm set f′={f′1,f′2,...,f′ i ,...,f′ f′} and signature algorithm set f″′={f″′1,f″′2,...,f″′ i ,...,f″′ g}, f′ i represents the i-th encryption algorithm in the encryption algorithm set, f″ represents the total number of encryption algorithms in the encryption algorithm set; f″′ i represents the i-th signature algorithm in the signature algorithm set, and g represents the total number of signature algorithms in the signature algorithm set;

[0132] S412, first from the encryption algorithm set f′={f′1,f′2,...,f′ i ,...,f′ f′} select an encryption algorithm to encrypt the message file data, and after the encryption is completed, the encrypted message file data is obtained; then select a signature algorithm set f″′={f″′1,f″′2,...,f″′ i ,...,f″′ g} select a signature algorithm to perform a signature operation on the encrypted message file data, and after the signature is completed, obtain the processed message file data;

[0133] S42: When the encrypted message file data is received, the encrypted message file data is decrypted and the signature is verified. After the decryption and signature verification are completed, the message file data is stored in the database.

[0134] A message processing system based on the Internet and data analysis technology, including a message structure definition module, a message reading module, a message analysis module, a message verification module, a message generation module, a message signature module, a message signature verification module, a message encryption module, a message decryption module and a message storage module;

[0135] The message structure definition module is used to abstract the structure of the message into a message data structure;

[0136] The message reading module is used to read the original message from the outside into the memory;

[0137] The message parsing module is used to parse the message data according to the message area set, the message data item matrix and the message association rule matrix;

[0138] The message verification module is used to perform integrity verification and rule verification on each row of data items in the initial message data;

[0139] The message generation module is used to convert the initial message data to obtain message file data;

[0140] The message signature module is used to perform a signature operation on the message file data;

[0141] The message signature verification module is used to verify the signature of the encrypted message file data;

[0142] The message encryption module is used to perform encryption operation on the message file data;

[0143] The message decryption module is used to decrypt the encrypted message file data;

[0144] The message storage module is used to store the processed message file data into a database.

[0145] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. A message processing method based on the Internet and data analysis technology, characterized in that: The following steps are involved: S1, abstracting the structure of the message into a message data structure; the message data structure includes regions constituting the message, obtaining a message region set; each message region in the message region set includes data items, obtaining a message data item matrix; association rules are set between each region of the message region set, obtaining a message association rule matrix; The S1 comprises the following steps: S11, abstracting the structure of the message into a message data structure; the message data structure includes the areas constituting the message, and obtaining a message area set a={a1, a2, ..., a i ,...,a a′ }, a i is the ith region divided in the message data structure, is the total number of regions divided in the message data structure; S12. Each message area in the message area set contains a data item, and the data item includes a data type and a length, and a message data item matrix a″ is obtained as follows: Among them: a i ' j ′ represents the jth data item in the i-th area in the message data structure; a1″′, a′2″, ..., a i ″′, ..., a′ a ″ ′ Respectively represent the total number of data items in the 1st, 2nd, ..., ith, ..., a′th area in the message data structure; Association rules are set between each area of ​​the message area set, and a message association rule matrix b is obtained, which is as follows: Where: b ij Represents an association rule between the ith area and the jth area in the message data structure; S2, reading the original message from the outside into the memory to obtain the original message data; the outside includes a file; the original message data is parsed according to the message area set, the message data item matrix and the message association rule matrix to obtain the parsed message data; the parsed message data is filled into the message data structure to obtain the initial message data; S3, performing integrity check and rule check on each row of data items in the initial message data, and obtaining a final check result after the check is completed; when the final check result is successful, converting the initial message data to obtain message file data; when the check result is failed, generating a failure feedback message according to the initial message data; S4. Encrypt and sign the message file data to obtain processed message file data; store the processed message file data in a database; when the encrypted message file data is received, decrypt the encrypted message file data and verify the signature, and after the decryption and signature verification are completed, store the message file data in the database.

2. The message processing method based on the Internet and data analysis technology according to claim 1 is characterized in that: The S2 comprises the following steps: S21, read the original message from the outside into the memory to obtain the original message data; S22, parsing the original message data to obtain parsed sub-message data; according to the message area set a={a1, a2, ..., a i ,...,a a′ }, the message data item matrix a″ and the message association rule matrix b divide the parsed sub-message data to obtain the parsed message data; S23, filling the parsed message data into the message data structure to obtain initial message data.

3. The message processing method based on the Internet and data analysis technology according to claim 2 is characterized in that: The S22 comprises the following steps: S221, reading the Header part of the original message data and parsing it to obtain a message sub-message; reading the Header part of the message sub-message to obtain the Header of the message sub-message; S222. Parse the Header of the message sub-message to obtain the id value and length value of the message sub-message; S223, set sub-message type set b'={b'1, b'2, ..., b' i ,...,b′ b″ } and the corresponding set of parsing methods b″′={b″′1,b″′2,...,b″′ i ,...,b″′ b′ }, b′ i is the i-th sub-message type, b″ is the total number of sub-message types, b″′ i b′ i The parsing method corresponding to the type sub-message; according to the id value of the message sub-message and the sub-message type set b′={b′1,b′2,...,b′ i ,...,b′ b″ } Determine the type of the message sub-message and obtain the sub-message type; S224, extract the length value of the message sub-message from the message sub-message to obtain the sub-message data; then according to the sub-message type and parsing method set b″′={b″′1,b″′2,...,b″′ i ,...,b″′ b″ } Parsing the sub-message data to obtain parsed sub-message data; S225, according to the message area set a={a1, a2, ..., a i ,...,a a′ }, the message data item matrix a″ and the message association rule matrix b divide the parsed sub-message data to obtain the parsed message data.

4. The message processing method based on the Internet and data analysis technology according to claim 1 is characterized in that: The S3 comprises the following steps: S31, using a cyclic redundancy check algorithm to perform integrity check on each row of data items in the initial message data, and obtaining a first check result after the check is completed; S32. When the first verification result is successful, perform rule verification on each row of data items in the initial message data to obtain a second verification result; when the first verification result or the second verification result is failed, the final verification result is failed; when the first verification result and the second verification result are both successful, the final verification result is successful; S33. When the final verification result is success, convert the initial message data to obtain message file data; when the final verification result is failure, generate a failure feedback message according to the initial message data.

5. The message processing method based on the Internet and data analysis technology according to claim 4 is characterized in that: The S31 comprises the following steps: S311, set the data item set consisting of each row of data items in the initial message data to c = {c1, c2, ..., c i ,...,c c′ },c i is the i-th data item in the initial message data, c′ is the total number of data items in the initial message data; S312, obtain c={c1, c2, ..., c i ,...,c c′ }, and obtain the data item bit column set c″={c″1,c″2,...,c″ i ,...,c″ c′ },c″ i is the bit column of the i-th data item in the initial message data; and the data item bit column set c″={c″1,c″2,...,c″ i ,...,c″ c′ }, each digit in each bit column is used as the coefficient of the polynomial, and the bit column polynomial set c″′={c″′1,c″′2,...,c″′ i ,...,c″′ c′ },c″′ i A polynomial formed by the bit sequence of the i-th data item in the initial message data; S313, set the prepolynomial d, and set the bit sequence polynomial set c″′={c″′1, c″′2, ..., c″′ i ,...,c″′ c′ } is divided by d to obtain the first remainder polynomial set d′={d′1,d′2,...,d′ i ,...,d′ c′ }, set the second remainder polynomial set d″={d″1,d″2,...,d″ obtained by using the prepolynomial d as the divisor when the initial message data is complete i ,...,d″ c′ }, when the first remainder polynomial set d′={d′1,d′2,...,d′ i ,...,d′ c′ } and the second remainder polynomial set d″={d″1,d″2,...,d″ i ,...,d″ c′ } are exactly the same, the initial message data is complete and the first verification result is success; otherwise, the initial message data is incomplete and the first verification result is failure.

6. The message processing method based on the Internet and data analysis technology according to claim 5 is characterized in that: The S32 comprises the following steps: S321. When the first verification result is successful, perform a rule verification on each row of data items in the initial message data to obtain a second verification result; performing a rule verification on each row of data items in the initial message data includes the following steps: S3211, set verification rule set d″′={d″′1,d″′2,...,d″′ i ,...,d″′ e },d″′ i is the i-th verification rule, and e is the total number of verification rules; S3212, according to c={c1,c2,...,c i ,...,c c′ } and data item verification rule set d″′={d″′1,d″′2,...,d″′ i ,...,d″′ e The data item verification rule matrix formed by the verification rules mounted on each data item in the setting is e′, as follows: Where: e′ ij For c = {c1, c2, ..., c i ,...,c c′ }, the jth validation rule mounted on the i-th data item, e″ i For c = {c1, c2, ..., c i ,...,c c′ The total number of validation rules mounted on the i-th data item in}; S3213, c={c1,c2,...,c i ,...,c c′ } is substituted into the verification rules mounted therein to obtain the correctness verification result matrix e″′, as follows: Where: e″′ ij For c = {c1, c2, ..., c i ,...,c c′ The verification result of the i-th data item in} after being verified by the j-th verification rule mounted on it; the data item correctness verification result judgment rule is set as follows: When e″′ i1 , e″′ i2 , ..., e″′ ij ,...and If all the values ​​of are successful, then c={c1,c2,...,c i ,...,c c′ }, the verification result of the i-th data item in is successful, otherwise, c={c1,c2,...,c i ,...,c c′ The verification result of the i-th data item in} is failed; According to the data item correctness check result judgment rule, the initial message data correctness check result set f = {f1, f2, ..., f i ,...,f c′ }, f i is the correctness check result of the i-th data item; when f1, f2, ..., f i , ... and f c′ If all are successful, the second verification result is successful, otherwise, the second verification result is failed; S322: When the first verification result or the second verification result is a failure, the final verification result is a failure; when the first verification result and the second verification result are both successful, the final verification result is a success.

7. The message processing method based on the Internet and data analysis technology according to claim 1 is characterized in that: The S4 comprises the following steps: S41, encrypting and signing the message file data to obtain processed message file data; storing the processed message file data in a database; S42: When the encrypted message file data is received, the encrypted message file data is decrypted and the signature is verified. After the decryption and signature verification are completed, the message file data is stored in the database.

8. The message processing method based on the Internet and data analysis technology according to claim 7 is characterized in that: The S41 comprises the following steps: S411, set encryption algorithm set f′={f′1,f′2,...,f′ i ,...,f′ f″ } and signature algorithm set f″′={f″′1,f″′2,...,f″′ i ,...,f″′ g }, f i ′ represents the i-th encryption algorithm in the encryption algorithm set, and f″ represents the total number of encryption algorithms in the encryption algorithm set; f i ″′ represents the i-th signature algorithm in the signature algorithm set, and g represents the total number of signature algorithms in the signature algorithm set; S412, first from the encryption algorithm set f′={f1′,f′2,...,f i ′,...,f′ f″ } selects an encryption algorithm to encrypt the message file data, and after the encryption is completed, the encrypted message file data is obtained; then selects an encryption algorithm from the signature algorithm set f″′={f1″′,f″′2,...,f i ″′,...,f″′ g }A signature algorithm is selected to perform a signature operation on the encrypted message file data, and after the signature is completed, the processed message file data is obtained.

9. A system for implementing the message processing method based on the Internet and data analysis technology as claimed in any one of claims 1 to 8, characterized in that: It includes a message structure definition module, a message reading module, a message parsing module, a message checking module, a message generating module, a message signing module, a message signature verification module, a message encryption module, a message decryption module and a message storage module; The message structure definition module is used to abstract the structure of the message into a message data structure; The message reading module is used to read the original message from the outside into the memory; The message parsing module is used to parse the message data according to the message area set, the message data item matrix and the message association rule matrix; The message verification module is used to perform integrity verification and rule verification on each row of data items in the initial message data; The message generation module is used to convert the initial message data to obtain message file data; The message signature module is used to perform a signature operation on the message file data; The message signature verification module is used to verify the signature of the encrypted message file data; The message encryption module is used to perform encryption operation on the message file data; The message decryption module is used to decrypt the encrypted message file data; The message storage module is used to store the processed message file data into a database.