A consumption record and analysis method, device and equipment based on matrix

Abstract

The present invention discloses a consumption record and analysis method, device and equipment based on a matrix. The method comprises: starting a consumption program, pulling messages in a message middleware and generating a message set; initializing a matrix according to the number of messages in the message set; sequentially processing the messages in the message set, and recording the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message; if the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix; if so, executing the matrix analysis action according to the analysis instruction.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a matrix-based recording method, device and equipment. Background Art

[0002] After consuming the message in the message middleware, the consumer performs corresponding business processing. During this process, the following situations may occur: 1. After the message is processed by the business in the consumer system, it will correspond to multiple processing results due to the different element values ​​in each message. It is necessary to regularly count the number of messages corresponding to each processing result within a specific time and the message element characteristics corresponding to the specific processing result; 2. During the processing process, there may be exceptions due to program defects or lack of business logic, or the message may not correspond to any established processing results. At this time, the message is considered "lost", that is, the business system did not process the message correctly; 3. The correct processing of the message requires preconditions. Only when specific preconditions are met can the message be processed correctly with business logic, so "backtracking" consumption of specific messages will occur to ensure that the message is processed correctly when the preconditions are met.

[0003] At present, consumer applications generally record messages by recording the message record itself, a brief description of the message processing results, and unstructured information such as exceptions. The current conventional solutions have the following disadvantages: 1. It does not support the above-mentioned "backtracking" scenario; 2. It does not support the above-mentioned "statistical" scenario; 3. It does not support the above-mentioned "verification" scenario; 4. Since each message will be printed in the log, it will cause excessive log volume, disk and I / O pressure; 5. Problem discovery has a lag and does not support active detection; 6. Unstructured record logs are not friendly to further log-based follow-up processing (remediation when problems occur, problem location when necessary, etc.). Summary of the invention

[0004] The present invention provides a matrix-based consumption record and analysis method, device and equipment, which solves the problem of difficulty in analyzing messages processed in a message system.

[0005] A consumption record and analysis method based on a matrix, comprising:

[0006] Start the consumer program, pull messages from the message middleware and generate a message collection;

[0007] Initializing a matrix according to the number of messages in the message set;

[0008] Processing the messages in the message set in sequence, and recording the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message;

[0009] If the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix;

[0010] If yes, the matrix analysis action is performed according to the analysis instruction.

[0011] In one embodiment of the present invention, the recording of the current message offset information at the corresponding position in the matrix according to the processing order and processing result of the current message specifically includes: determining the storage row of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage column of the current message offset information in the matrix according to the processing result of the current message; or determining the storage column of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage row of the current message offset information in the matrix according to the processing result of the current message; locating the specific storage position of the current message offset information in the matrix according to the storage row and the storage column, and recording the current message offset information in the storage position; wherein, if the processing result of the current message is a program exception, the negative value of the offset of the current message is recorded in the corresponding storage position.

[0012] In one embodiment of the present invention, the method also includes: if the processing result of the current message is a program exception, the negative value of the offset of the current message is recorded in the corresponding storage location; if the processing result of the current message is a business logic exception, the offset of the current message is recorded in the corresponding storage location.

[0013] In one embodiment of the present invention, the initializing the matrix according to the number of messages in the message set specifically includes: determining the number of rows of the matrix according to the number of messages in the message set; determining the number of columns of the matrix according to the number of message processing results; or determining the number of columns of the matrix according to the number of messages in the message set; determining the number of rows of the matrix according to the number of message processing results.

[0014] In one embodiment of the present invention, the matrix is ​​initialized according to the number of messages in the message set, specifically including: if the number of messages in the message set is M, then the number of rows or columns of the matrix is ​​determined to be M; if the number of normal processing results of the message business logic is N, then the number of rows or columns of the matrix is ​​determined to be N+1; wherein, the processing result of the N+1th column or the N+1th row corresponds to a program exception result and / or a business logic exception result; wherein, if the number of messages in the message set M is the number of rows of the matrix, then the number of message processing results N+1 is the number of columns of the matrix; conversely, if M is the number of columns, then N+1 is the number of rows.

[0015] In one embodiment of the present invention, the matrix analysis action is performed according to the analysis instruction, specifically including: if the processing result that does not meet the precondition is the corresponding i-th processing result in the matrix; determining the offset recorded in the row or column corresponding to the i-th processing result; regularly backtracking consumption according to the offset; or locating the message in the message middleware according to the offset, and regularly querying whether the message has met the precondition, and if so, consuming it again.

[0016] In one embodiment of the present invention, the matrix analysis action is performed according to the analysis instruction, specifically including: obtaining the message distribution of each processing result; or determining the offset stored in the row or column corresponding to a specific processing result; pulling the message from the message middleware according to the offset; performing feature analysis on the message to obtain the message features under the specific processing result.

[0017] In one embodiment of the present invention, a matrix analysis action is performed according to an analysis instruction, specifically including: the processing result includes a program exception result and a business logic exception result; determining the offset information in the row or column corresponding to the program exception result and / or the business logic exception result; determining a unique exception result of the corresponding message according to the offset information; and performing corresponding exception processing according to the exception result.

[0018] A consumption recording and analysis device based on a matrix, comprising:

[0019] The message acquisition module is used to start the consumer program, pull messages from the message middleware and generate message collections;

[0020] A recording module, used for initializing a matrix according to the number of messages in the message set; sequentially processing the messages in the message set, and recording the current message offset information to a corresponding position in the matrix according to the processing order and processing result of the current message;

[0021] The analysis module is used to determine whether to perform matrix analysis on the matrix if the messages in the message set have been processed; if so, execute the matrix analysis action according to the analysis instruction.

[0022] A consumption recording and analysis device based on a matrix, comprising:

[0023] at least one processor; and,

[0024] A memory connected to the at least one processor via a bus; wherein,

[0025] The memory stores instructions executable by the at least one processor, and the instructions are executed to implement:

[0026] Start the consumer program, pull messages from the message middleware and generate a message collection;

[0027] Initializing a matrix according to the number of messages in the message set;

[0028] Processing the messages in the message set in sequence, and recording the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message;

[0029] If the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix;

[0030] If yes, the matrix analysis action is performed according to the analysis instruction.

[0031] A non-volatile storage medium stores computer executable instructions, wherein the computer executable instructions are executed by a processor to implement the following steps:

[0032] Start the consumer program, pull messages from the message middleware and generate a message collection;

[0033] Initializing a matrix according to the number of messages in the message set;

[0034] Processing the messages in the message set in sequence, and recording the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message;

[0035] If the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix;

[0036] If yes, the matrix analysis action is performed according to the analysis instruction.

[0037] The present invention provides a matrix-based consumption record and analysis method, device and equipment, which at least have the following beneficial effects:

[0038] 1. Support "backtracking" scenarios

[0039] By matching the message offset with the processing result of the business logic, it supports "backtracking" and consuming the message corresponding to the offset in the specific processing result under specific conditions or periodically.

[0040] 2. Support "statistics" scenario

[0041] By corresponding the message offset and the processing results of the business logic, it supports statistics and analysis in multiple dimensions, such as the distribution of message processing results and multi-dimensional feature analysis of messages with specific processing results.

[0042] 3. Support "check" scenario

[0043] The offset of program exceptions or business logic exception messages that occur during business logic processing is recorded in the matrix, such as in the last column, so that it is convenient to check whether there are abnormal situations such as message omissions in the business processing process, and perform corresponding subsequent processing according to the recorded message offset after the abnormal situation is detected.

[0044] 4. Matrix analysis flexible switch

[0045] By configuring the switch of matrix analysis, you can flexibly decide whether to perform matrix analysis. By configuring the analysis cycle and supporting matrix analysis for specific batches, you can save resources while meeting the analysis requirements.

[0046] 5. Problem discovery is proactive and timely

[0047] Since the completion of message collection processing is used as the trigger action for matrix analysis, it can effectively ensure proactive and timely discovery of problems and reduce the impact of delayed problem solving.

[0048] 6. Reduce log printing

[0049] The body of each message will not be printed in the log, which can greatly reduce the amount of logs in the consumer system, reduce disk usage and I / O pressure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0051] Figure 1 A schematic diagram of a consumption record and analysis method based on a matrix provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a matrix-based consumption record and analysis solution process provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a matrix-based consumption recording and analysis device provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of a matrix-based consumption recording and analysis device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be described clearly and completely in conjunction with the specific embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] It should be noted that it is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present invention can be combined with other embodiments without conflict. Unless otherwise defined, the technical terms or scientific terms involved in the present invention should be the usual meanings understood by people with ordinary skills in the technical field to which the present invention belongs. The words "one", "a", "a", "the" and the like involved in the present invention do not indicate a quantitative limitation and may represent the singular or plural. The terms "including", "comprising", "having" and any of their variations involved in the present invention are intended to cover non-exclusive inclusions; the terms "first", "second", "third", etc. involved in the present invention are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0057] After consuming the message in the message middleware, the consumer performs the corresponding business processing. In this process, the consumer program needs to have multiple features such as "backtracking", "statistics", and "verification" to ensure that each message can be processed correctly. In order to ensure the realization of the above scenario, the present invention provides a message recording scheme. The evaluation of the advantages and disadvantages of the message recording mechanism scheme is mainly based on whether the scheme can support as many processing scenarios as possible with the simplest structure while minimizing the additional workload. The present invention proposes a matrix-based consumption recording and analysis method, device and equipment, which ensures that each message can be correctly processed through a structured recording method that supports "backtracking", "statistics", "verification" and other features, and that the consumption system has regular multi-dimensional statistics and analysis, active and effective problem discovery and other features. The following is a detailed explanation.

[0058] The following is an introduction to the terms involved in the embodiments of the present disclosure:

[0059] Explanation of terms (explanation of terms that should be listed):

[0060] Message middleware: It is a supporting software system based on queue and message passing technology, which provides synchronous or asynchronous, reliable message transmission for application systems in a network environment. It has many advantages such as asynchronous performance improvement, reduced coupling, and traffic peak shaving, and is widely used in information systems.

[0061] Consumer: The party that receives messages. Consumers connect to the message middleware and receive messages, and then perform corresponding business logic processing.

[0062] Topic: Messages in the message middleware are classified by topic. Producers are responsible for sending messages to specific topics, while consumers are responsible for subscribing to topics and consuming.

[0063] Partition: A topic is a logical concept, which can be subdivided into multiple partitions. A partition belongs to a single topic, and partitions are often referred to as topic partitions. Different partitions under the same topic contain different messages. Partitions can be regarded as appendable log files at the storage level. When messages are appended to partition log files, they are assigned a specific offset.

[0064] Offset: It is the unique identifier of a message in a partition. The message middleware uses it to ensure the order of messages within the partition. Consumers use the offset to determine a specific message in a partition when consuming. The message middleware-related API supports consuming specific messages according to the offset.

[0065] Normal processing result of business logic: the result obtained after normal consumption processing.

[0066] Abnormal results: including program abnormal results and business logic abnormal results.

[0067] Processing result: A general term for the sum of all normal processing results and abnormal results of all business logic.

[0068] Figure 1 A schematic diagram of a consumption record and analysis method based on a matrix provided by an embodiment of the present invention may include the following steps:

[0069] S110: Start the consumer program, pull messages from the message middleware and generate a message set.

[0070] Specifically, Figure 2 The figure shows a flow chart of a matrix-based consumption record and analysis solution provided by an embodiment of the present invention. After the consumption program is started in the consumption system, messages are pulled from the message middleware, and a specific number of M messages can be pulled at a time to obtain a message set including M messages.

[0071] S120: Initialize the matrix according to the number of messages in the message set.

[0072] In one embodiment of the present invention, a matrix is ​​initialized according to the number of messages in a message set, specifically including: determining the number of rows of the matrix according to the number of messages in the message set; determining the number of columns of the matrix according to the number of message processing results; or determining the number of columns of the matrix according to the number of messages in the message set; determining the number of rows of the matrix according to the number of message processing results.

[0073] Furthermore, if the number of messages in the message set is M, the number of rows or columns of the matrix is ​​determined to be M; if the number of normal processing results of the message business logic is N, the number of rows or columns of the matrix is ​​determined to be N+1; wherein, the processing result of the N+1th column or the N+1th row corresponds to the program exception result and / or the business logic exception result; wherein, if the number of messages in the message set M is the number of rows of the matrix, the number of message processing results N+1 is the number of columns of the matrix; conversely, if M is the number of columns, then N+1 is the number of rows.

[0074] Specifically, after a message is pulled, the matrix A is initialized. The size of the matrix A is M*(N+1) or (N+1)*M, and the value of each element in the matrix is ​​initialized to 0. In this embodiment, the M*(N+1) matrix is ​​used as an example for explanation, and the (N+1)*M matrix is ​​the same. Among them, M is the number of messages pulled, as the number of rows of the matrix, N is all possible normal processing results of the established business logic (excluding abnormal results), and the length of each row of the matrix is ​​N+1, that is, the number of matrix columns is N+1. The purpose of setting the N+1th column is to record the offset information of the message when the program is abnormal and the business logic is abnormal in the N+1th column.

[0075] S130: Process the messages in the message set sequentially, and record the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message.

[0076] In one embodiment of the present invention, the current message offset information is recorded in a corresponding position in a matrix according to the processing order and processing result of the current message, specifically including: determining the storage row of the current message offset information in the matrix according to the order of the current message in a message set; determining the storage column of the current message offset information in the matrix according to the processing result of the current message; or determining the storage column of the current message offset information in the matrix according to the order of the current message in a message set; determining the storage row of the current message offset information in the matrix according to the processing result of the current message; locating the specific storage position of the current message offset information in the matrix according to the storage row and storage column, and recording the current message offset information in the storage position.

[0077] Specifically, the message undergoing business logic processing in the message set is taken as the current message. Assume the index of the current message in the message set fetched this time is i (the index in the message set starts from 0, 0 <= i < M). The offset of the current message is recorded in A[i][j] according to the result of business logic processing (0 <= j <= N, j starts from 0, and it can be equal to N because the column length of the matrix is N + 1), indicating that the i-th message in the message set falls into the j-th result after business logic processing.

[0078] In an embodiment of the present invention, if the processing result of the current message is a program exception, the negative value of the offset of the current message is recorded in the corresponding storage location; if the processing result of the current message is a business logic exception, the offset of the current message is recorded in the corresponding storage location.

[0079] Specifically, if a program exception occurs during the message processing, the negative number corresponding to the offset of the message is recorded in A[i][N]; if the message does not correspond to any normal business logic processing result (that is, a logic exception occurs) after business logic processing, the offset of the message is recorded in A[i][N].

[0080] For example, 10 messages are fetched from the message middleware at one time to form a message set, and the number of all normal business logic processing results (excluding exception results) existing in the consumption system is 3. Then the matrix is 10 * (3 + 1), that is, a 10 * 4 matrix. (This is only a simplified embodiment for easy explanation. In fact, the number of fetched messages can be several thousand, tens of thousands, or even millions, and the number of processing results is set according to business requirements).

[0081] The first message has a processing result of 0, and its offset 333 is recorded in row 0 and column 0;

[0082] The second message has a processing result of 1, and its offset 334 is recorded in row 1 and column 1;

[0083] The third message has a processing result of 2, and its offset 335 is recorded in row 2 and column 2;

[0084] The fourth message has a processing result of 1, and its offset 336 is recorded in row 3 and column 1;

[0085] The fifth message has a processing result of 1, and its offset 337 is recorded in row 4 and column 1;

[0086] The sixth message has a processing result of 2, and its offset 338 is recorded in row 5 and column 2;

[0087] The seventh message has a program exception, and its negative offset -339 is recorded in row 6 and column 3;

[0088] The eighth message is the processing result 2, and its offset 340 is recorded in row 7 and column 2;

[0089] The ninth message is a business logic exception, and its offset 341 is recorded in row 8 and column 3;

[0090] The tenth message is a business logic exception, and its offset 342 is recorded in row 9 and column 3.

[0091] Table 1 shows a schematic diagram of the format of matrix storage offset information:

[0092]

[0093]

[0094] Table 1

[0095] By setting the offset of the matrix to store messages, we avoid printing the message body of each message in the log, greatly reducing the log volume of the consumer system, reducing disk usage and I / O pressure.

[0096] S140: If the messages in the message set have been processed, determine whether to perform matrix analysis on the matrix.

[0097] Specifically, after each message in the message set completes business logic processing and processing result recording, the subscript of the message in the message set is incremented to iteratively process the next message of the current message. After the M messages pulled this time have been processed, it is determined whether matrix analysis is required for the current message set.

[0098] Furthermore, the matrix analysis action can be set as a switch action. If the staff believes that matrix analysis is not necessary, they can select "Turn off matrix analysis". Then, after the current message set processing is completed, the consumer system will skip the matrix analysis and directly enter the next message set pull and subsequent processing steps; if the staff believes that matrix analysis is necessary, they can select "Turn on matrix analysis". Then, after the current message set processing is completed, the consumer system will start to enter and execute the relevant matrix analysis actions.

[0099] Furthermore, in order to ensure a balance between resources and efficiency, matrix analysis can be set up to be performed periodically, for example, a matrix analysis is performed after every n message sets are consumed. If the consumption quality requirement is high, a matrix analysis is performed after every message set is consumed. The number n can be set according to consumption demand; or a random matrix analysis can be set up, etc.

[0100] Furthermore, the matrix analysis action can be performed by opening up another non-main thread to ensure the consumption efficiency of the consumer system and prevent message backlogs caused by matrix analysis.

[0101] By using the completion of message collection processing as the trigger action for matrix analysis, it can effectively ensure proactive and timely discovery of problems and reduce the impact of delayed problem solving. By setting the matrix analysis switch, you can flexibly configure whether to perform matrix analysis and support analysis of matrices for specific batches.

[0102] S150: If yes, perform matrix analysis according to the analysis instruction.

[0103] In one embodiment of the present invention, a matrix analysis action is performed according to an analysis instruction, specifically including: if the processing result that does not meet the precondition is the corresponding i-th processing result in the matrix; determining the offset recorded in the row or column corresponding to the i-th processing result; periodically backtracking consumption according to the offset; or locating the message in the message middleware according to the offset, periodically querying whether the message has met the precondition, and if so, consuming it again.

[0104] Specifically, assuming that a message will fall into the i-th result only when a certain precondition is not met, the offset in the matrix A[x][i] (the column with the subscript i in the matrix) is checked, and the offset falling in the column is subjected to multiple regular "backtracking" consumption or "backtracking" consumption after the precondition is met. For example, based on the above embodiment, processing result 1 is the consumption result when the message does not meet a specific precondition. At this time, "backtracking" consumption can be performed according to the offset in the column corresponding to processing result 1 in the matrix.

[0105] For example, in a payment scenario, the consumption system will pull the user's order and other payment information from the message middleware for deductions. However, if the user has "insufficient balance", the deduction conditions are not met and the deduction cannot be successful. In this case, it is necessary to regularly trace back to obtain the order for deduction until the deduction is successful; or monitor whether the order meets the preconditions, that is, after monitoring that the user balance meets the precondition of "sufficient balance", trace back to the message middleware again according to the offset information to pull the order and perform the deduction action.

[0106] By making the message offset correspond to the processing result, it supports "backtracking" and consuming the message corresponding to the offset in the specific processing result under specific conditions or periodically.

[0107] In one embodiment of the present invention, a matrix analysis action is performed according to an analysis instruction, specifically including: obtaining the message distribution of each processing result; or determining the offset stored in the row or column corresponding to a specific processing result; pulling the message from the message middleware according to the offset; performing feature analysis on the message to obtain the message features under the specific processing result.

[0108] Specifically, multi-dimensional statistics are performed on the matrix, such as the number of messages corresponding to each processing result (that is, the number of offset information corresponding to each processing result, the number of offset information is equal to the number of messages), and the characteristics of the messages in a specific processing result (such as the characteristics of the messages falling in a specific processing result).

[0109] For example, based on the above embodiment, the corresponding number of processing results of the 10 messages in the current message set is distributed as 1:3:3:3; if the staff needs to analyze the first processing result, they can re-consume the message itself corresponding to the offset (334, 336, 337) in the processing result 1, and analyze specific elements in the message. For example, in the payment scenario, the number of users who have successfully paid are from Shanxi, the number of users from Guangdong, etc.

[0110] By matching the message offset with the normal processing result of the business logic, it supports statistics and analysis in multiple dimensions, such as the distribution of message processing results and multi-dimensional feature analysis of messages with specific processing results.

[0111] In one embodiment of the present invention, a matrix analysis action is performed according to an analysis instruction, specifically including: processing results including program exception results and business logic exception results; determining offset information in rows or columns corresponding to the program exception results and / or business logic exception results; determining a unique exception result of the corresponding message based on the offset information; and performing corresponding exception processing based on the exception results.

[0112] Specifically, since the message offset is recorded in the Nth column only when a program exception (recorded as a negative number of the offset) or a business logic exception (recorded as the offset itself) occurs, when performing an exception check on consumption, directly check the offset information in the matrix A[x][N] (the Nth column of all rows in the matrix) and analyze the last column of the matrix to determine the unique exception result. If there is no offset in the last column (that is, all values ​​in the last column are 0), it means that the current batch has been processed correctly; if there is a negative number, it means that the message corresponding to its absolute value has caused a program exception; if there is a positive number, it means that the corresponding message has no corresponding logical result in our current consumption system, and it is necessary to analyze whether the current business logic analysis is missing.

[0113] By recording the offset information of program exceptions or logic exception messages that occur during business logic processing in the last column of the matrix, it is convenient to check whether there are abnormal situations such as message omissions in the business processing process, and perform corresponding subsequent processing based on the recorded message offset after detecting abnormal situations.

[0114] After the matrix analysis is completed, continue to pull messages from the message middleware to perform business logic and record them. In different consumption scenarios, matrix analysis actions and business logic processing can be executed sequentially or in parallel. For example, when time requirements are not strict, the matrix analysis time is negligible for consumption time. At this time, the matrix analysis program and business logic processing program can be executed sequentially in a single thread, and consumption requests can be queued to process, thereby saving computing resources; when multiple consumers consume at the same time, in order to further improve the processing speed, multiple threads can be opened to execute different business logic processing programs and matrix analysis actions respectively, and efficiency can be improved through multi-threaded parallel processing.

[0115] Most of the embodiments of the present invention are explained using the number of messages in a message set as the rows of the matrix and the number of message processing results as the columns of the matrix. If the number of messages in a message set is used as the columns of the matrix and the number of message processing results is used as the rows of the matrix, the same technical principles should be used and the same technical effects should be produced.

[0116] The above is a consumption record and analysis method based on a matrix provided by an embodiment of the present invention. Based on the same inventive idea, an embodiment of the present invention also provides a corresponding consumption record and analysis device based on a matrix, such as Figure 3 shown.

[0117] The message acquisition module 302 is used to start the consumer program, pull messages in the message middleware and generate a message set; the recording module 304 is used to initialize the matrix according to the number of messages in the message set; sequentially process the messages in the message set, and record the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message; the analysis module 306 is used to determine whether to perform matrix analysis on the matrix if the messages in the message set have been processed; if so, execute the matrix analysis action according to the analysis instruction.

[0118] The embodiment of the present invention also provides a corresponding matrix-based consumption recording and analysis device, such as Figure 4 As shown, including:

[0119] At least one processor 402, a communication interface 404, a memory 406, and a communication bus 408; wherein the processor 402, the communication interface 404, and the memory 406 communicate with each other through the communication bus 408; the processor 402 may call the logic instructions stored in the memory 406, so that at least one processor 402 executes:

[0120] Start the consumer program, pull messages from the message middleware and generate a message set; initialize the matrix according to the number of messages in the message set; process the messages in the message set sequentially, and record the current message offset information to the corresponding position in the matrix according to the processing order and processing results of the current messages; if the messages in the message set have been processed, determine whether to perform matrix analysis on the matrix; if so, execute the matrix analysis action according to the analysis instruction.

[0121] Based on the same idea, some embodiments of the present invention also provide a medium corresponding to the above method.

[0122] Some embodiments of the present invention provide a storage medium storing computer executable instructions, wherein the computer executable instructions are executed by a processor to implement the following steps:

[0123] Start the consumer program, pull messages from the message middleware and generate a message set; initialize the matrix according to the number of messages in the message set; process the messages in the message set sequentially, and record the current message offset information to the corresponding position in the matrix according to the processing order and processing results of the current messages; if the messages in the message set have been processed, determine whether to perform matrix analysis on the matrix; if so, execute the matrix analysis action according to the analysis instruction.

[0124] The various embodiments of the present invention are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0125] The devices and media provided in the embodiments of the present invention correspond one-to-one to the methods, and therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0126] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or method. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or method including the elements.

[0127] The above are only embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made on the basis of the present invention. Therefore, these modifications or improvements made on the basis of not departing from the spirit of the present invention all belong to the scope of protection claimed in the present invention.

Claims

1. A consumption record and analysis method based on a matrix, characterized in that: include: Start the consumer program, pull messages from the message middleware and generate a message collection; Initializing a matrix according to the number of messages in the message set, including: if the number of messages in the message set is M, determining the number of rows or columns of the matrix to be M; if the number of normal processing results of the message business logic is N, determining the number of rows or columns of the matrix to be N+1; wherein the processing result of the N+1th column or the N+1th row corresponds to a program abnormal result and / or a business logic abnormal result; wherein, if the number of messages in the message set M is the number of rows of the matrix, the number of message processing results N+1 is the number of columns of the matrix; conversely, if M is the number of columns, then N+1 is the number of rows; The messages in the message set are processed sequentially, and the current message offset information is recorded to the corresponding position in the matrix according to the processing order and processing result of the current message, including: determining the storage row of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage column of the current message offset information in the matrix according to the processing result of the current message; or determining the storage column of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage row of the current message offset information in the matrix according to the processing result of the current message; locating the specific storage position of the current message offset information in the matrix according to the storage row and the storage column, and recording the current message offset information in the storage position; If the processing result of the current message is a program exception, the negative value of the offset of the current message is recorded in the corresponding storage location; if the processing result of the current message is a business logic exception, the offset of the current message is recorded in the corresponding storage location; If the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix; If so, the matrix analysis action is performed according to the analysis instruction, including: if the processing result that does not meet the precondition is the corresponding i-th processing result in the matrix; determine the offset recorded in the row or column corresponding to the i-th processing result; regularly trace back the consumption according to the offset; or locate the message in the message middleware according to the offset, and regularly query whether the message has met the precondition. If so, consume it again.

2. The method according to claim 1, characterized in that The matrix analysis operation is performed according to the analysis instruction, specifically including: Get the message distribution of each processing result; or Determine the offset stored in the row or column corresponding to a particular processing result; Pull the message from the message middleware according to the offset; Perform feature analysis on the message to obtain message features under the specific processing result.

3. The method according to claim 1, characterized in that Execute matrix analysis actions according to the analysis instructions, including: The processing results include program abnormality results and business logic abnormality results; Determine the offset information in the row or column corresponding to the program abnormal result and / or the business logic abnormal result; Determine a unique abnormal result of the corresponding message according to the offset information; Execute corresponding exception processing according to the exception result.

4. A consumption recording and analysis device based on a matrix, characterized in that: include: The message acquisition module is used to start the consumer program, pull messages from the message middleware and generate message collections; A recording module, used to initialize a matrix according to the number of messages in the message set, including: if the number of messages in the message set is M, then determining the number of rows or columns of the matrix to be M; if the number of normal message business logic processing results is N, then determining the number of rows or columns of the matrix to be N+1; wherein, the processing result of the N+1th column or the N+1th row corresponds to a program abnormal result and / or a business logic abnormal result; wherein, if the number of messages in the message set M is the number of rows of the matrix, then the number of message processing results N+1 is the number of columns of the matrix; conversely, if M is the number of columns, then N+1 is the number of rows; sequentially processing the messages in the message set, and recording the current message offset information to the corresponding position in the matrix according to the processing order and processing result of the current message, including: according to the order of the current message in the message set Determine the storage row of the current message offset information in the matrix; determine the storage column of the current message offset information in the matrix according to the processing result of the current message; or determine the storage column of the current message offset information in the matrix according to the order of the current message in the message set; determine the storage row of the current message offset information in the matrix according to the processing result of the current message; locate the specific storage position of the current message offset information in the matrix according to the storage row and the storage column, and record the current message offset information in the storage position; if the processing result of the current message is a program exception, record the negative value of the offset of the current message in the corresponding storage position; if the processing result of the current message is a business logic exception, record the offset of the current message in the corresponding storage position; The analysis module is used to determine whether to perform matrix analysis on the matrix if the messages in the message set have been processed; if so, execute the matrix analysis action according to the analysis instruction, including: if the processing result that does not meet the precondition is the corresponding i-th processing result in the matrix; determine the offset recorded in the row or column corresponding to the i-th processing result; regularly trace back the consumption according to the offset; or locate the message in the message middleware according to the offset, and regularly query whether the message has met the precondition, and if so, consume it again.

5. A consumption recording and analysis device based on a matrix, characterized in that: include: at least one processor; as well as, A memory connected to the at least one processor via a bus; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed to implement: Start the consumer program, pull messages from the message middleware and generate a message collection; Initializing a matrix according to the number of messages in the message set, including: if the number of messages in the message set is M, determining the number of rows or columns of the matrix to be M; if the number of normal processing results of the message business logic is N, determining the number of rows or columns of the matrix to be N+1; wherein the processing result of the N+1th column or the N+1th row corresponds to a program abnormal result and / or a business logic abnormal result; wherein, if the number of messages in the message set M is the number of rows of the matrix, the number of message processing results N+1 is the number of columns of the matrix; conversely, if M is the number of columns, then N+1 is the number of rows; The messages in the message set are processed sequentially, and the current message offset information is recorded to the corresponding position in the matrix according to the processing order and processing result of the current message, including: determining the storage row of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage column of the current message offset information in the matrix according to the processing result of the current message; or determining the storage column of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage row of the current message offset information in the matrix according to the processing result of the current message; locating the specific storage position of the current message offset information in the matrix according to the storage row and the storage column, and recording the current message offset information in the storage position; If the processing result of the current message is a program exception, the negative value of the offset of the current message is recorded in the corresponding storage location; if the processing result of the current message is a business logic exception, the offset of the current message is recorded in the corresponding storage location; If the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix; If so, the matrix analysis action is performed according to the analysis instruction, including: if the processing result that does not meet the precondition is the corresponding i-th processing result in the matrix; determine the offset recorded in the row or column corresponding to the i-th processing result; regularly trace back the consumption according to the offset; or locate the message in the message middleware according to the offset, and regularly query whether the message has met the precondition. If so, consume it again.

6. A non-volatile storage medium storing computer executable instructions, characterized in that: The computer executable instructions are executed by a processor to implement the following steps: Start the consumer program, pull messages from the message middleware and generate a message collection; Initializing a matrix according to the number of messages in the message set, including: if the number of messages in the message set is M, determining the number of rows or columns of the matrix to be M; if the number of normal processing results of the message business logic is N, determining the number of rows or columns of the matrix to be N+1; wherein the processing result of the N+1th column or the N+1th row corresponds to a program abnormal result and / or a business logic abnormal result; wherein, if the number of messages in the message set M is the number of rows of the matrix, the number of message processing results N+1 is the number of columns of the matrix; conversely, if M is the number of columns, then N+1 is the number of rows; The messages in the message set are processed sequentially, and the current message offset information is recorded to the corresponding position in the matrix according to the processing order and processing result of the current message, including: determining the storage row of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage column of the current message offset information in the matrix according to the processing result of the current message; or determining the storage column of the current message offset information in the matrix according to the order of the current message in the message set; determining the storage row of the current message offset information in the matrix according to the processing result of the current message; locating the specific storage position of the current message offset information in the matrix according to the storage row and the storage column, and recording the current message offset information in the storage position; If the processing result of the current message is a program exception, the negative value of the offset of the current message is recorded in the corresponding storage location; if the processing result of the current message is a business logic exception, the offset of the current message is recorded in the corresponding storage location; If the messages in the message set have been processed, determining whether to perform matrix analysis on the matrix; If so, the matrix analysis action is performed according to the analysis instruction, including: if the processing result that does not meet the precondition is the corresponding i-th processing result in the matrix; determine the offset recorded in the row or column corresponding to the i-th processing result; regularly trace back the consumption according to the offset; or locate the message in the message middleware according to the offset, and regularly query whether the message has met the precondition. If so, consume it again.

Images