Message processing method and device based on pressure test, equipment and storage medium

Abstract

The invention belongs to the field of data processing and finance, and relates to a message processing method based on a pressure test, which comprises the following steps: constructing a message body according to service attributes and request content data extracted from a pressure test request; when it is judged that the processing load of the message body exceeds a preset threshold value according to the historical processing data, the message features of the message body are added to a message construction rule; extracting a message body key feature from the pressure measurement data and the online monitoring data to adjust a message construction rule; generating a message according to the new message construction rule, and initiating a message request; dynamically adjusting the number of cluster nodes according to the concurrency, obtaining messages corresponding to the message requests based on the processing sequence, and distributing the messages to the cluster nodes according to a load balancing algorithm. The invention further provides a message processing device and equipment based on the pressure test and a storage medium. In addition, the invention also relates to a block chain technology, and request content data can be stored in a block chain. The message body is dynamically constructed, and the message production speed is controlled.

Description

Technical Field

[0001] The present application relates to the fields of data processing and financial technology, and in particular to a message processing method, device, equipment and storage medium based on stress testing. Background Art

[0002] With the rapid development of Internet technology, the financial sector has an increasing demand for efficient, stable and scalable technical architecture. In the construction of financial systems, the application of middleware (Message Queue, MQ for short) has become more and more extensive, especially in key scenarios such as system decoupling, asynchronous communication, peak shaving and valley filling, and data order assurance. These scenarios not only require MQ to efficiently process large amounts of data, but also place extremely high demands on consumer processing performance to ensure low latency and high reliability of financial transactions.

[0003] In the actual operation of financial services, the performance of MQ message consumers is directly related to the stability and response speed of the system. Therefore, it is particularly important to conduct a comprehensive stress test on MQ message consumers before the application goes online. Stress testing is designed to simulate extreme conditions such as high concurrency and large data volumes in actual business scenarios to evaluate the system's carrying capacity and performance bottlenecks.

[0004] However, traditional open source stress testing tools, such as jmeter, have obvious limitations when performing stress testing on mainstream MQs such as Rocketmq. Specifically, jmeter lacks a universal Rocketmq message sending plug-in, which makes it difficult for testers to dynamically and automatically construct message bodies in a data-driven manner, and it is also impossible to dynamically configure message sending traffic. This defect greatly limits the flexibility and accuracy of stress testing.

[0005] In specific stress testing scenarios in the financial field, it is often necessary to simulate real online traffic scenarios to be closer to the actual business environment. This requires the ability to accurately control the production and sending speed of messages to prevent producers from sending messages too quickly, resulting in a large backlog of messages in the MQ middleware. If the message backlog is serious, it will not only affect the performance evaluation results of consumers, but may also pose a potential threat to the stability and security of the financial system.

[0006] In addition, traditional stress testing tools also have shortcomings in detecting message sending results. It is often impossible to automatically detect and provide feedback on whether the message sent by the producer is successful or failed, which is not conducive to stress testers to quickly determine the results of message sending, thus affecting test efficiency and accuracy. Summary of the invention

[0007] The purpose of the embodiments of the present application is to propose a message processing method, device, equipment and storage medium based on stress testing, so as to solve the technical problem that the related technology cannot dynamically and automatically construct the message body, and cannot realize the dynamic configuration of the message sending speed flow, thereby limiting the flexibility and accuracy of the stress test.

[0008] In order to solve the above technical problems, the embodiment of the present application provides a message processing method based on stress testing, which adopts the following technical solution:

[0009] Receive a stress test request, and extract request content data in the stress test request;

[0010] Obtaining a preset message body structure template, extracting business attributes of the message body structure template, and constructing a message body according to the business attributes and the request content data;

[0011] Acquire historical processing data corresponding to the message body, determine whether a processing load of the message body exceeds a preset threshold value according to the historical processing data, and add a message feature of the message body to a message construction rule when the processing load exceeds the preset threshold value;

[0012] Obtaining stress testing data and online monitoring data of the message body, extracting key features of the message body from the stress testing data and the online monitoring data, adjusting the message construction rule according to the key features, and obtaining a new message construction rule;

[0013] According to the new message construction rule, a message is generated according to the message body, the message is transmitted to the message middleware, and a message request is initiated;

[0014] Obtaining the concurrency of the message request, dynamically adjusting the number of distributed cluster nodes according to the concurrency, and determining the processing order according to the priority parameter of the message request;

[0015] The message corresponding to the message request is obtained from the message middleware based on the processing order, and the message is distributed to each of the distributed cluster nodes for processing according to a load balancing algorithm.

[0016] In order to solve the above technical problems, the embodiment of the present application also provides a message processing device based on stress testing, which adopts the following technical solution:

[0017] An extraction module, used for receiving a stress test request and extracting request content data in the stress test request;

[0018] A construction module, used to obtain a preset message body structure template, extract the business attributes of the message body structure template, and construct a message body according to the business attributes and the request content data;

[0019] A judgment module, used for obtaining historical processing data corresponding to the message body, judging whether the processing load of the message body exceeds a preset threshold value according to the historical processing data, and adding the message feature of the message body to the message construction rule when the processing load exceeds the preset threshold value;

[0020] A rule adjustment module, used to obtain stress test data and online monitoring data of the message body, extract key features of the message body from the stress test data and the online monitoring data, and adjust the message construction rule according to the key features to obtain a new message construction rule;

[0021] A generation module, used to generate a message according to the message body in accordance with the new message construction rule, transmit the message to the message middleware, and initiate a message request;

[0022] A node adjustment module, used to obtain the concurrency of the message request, dynamically adjust the number of distributed cluster nodes according to the concurrency, and determine the processing order according to the priority parameter of the message request;

[0023] A distribution module is used to obtain the message corresponding to the message request from the message middleware based on the processing order, and distribute the message to each of the distributed cluster nodes for processing according to a load balancing algorithm.

[0024] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0025] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the message processing method based on stress testing as described above when executing the computer-readable instructions.

[0026] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0027] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the message processing method based on stress testing as described above are implemented.

[0028] Compared with the prior art, this application has the following beneficial effects:

[0029] The present application provides a message processing method based on stress testing, which extracts request content data in a stress testing request, and constructs a message body according to the request content data and a preset message body structure template, thereby realizing dynamic and automatic construction of the message body and improving the flexibility of message body construction; when the processing load of the message body exceeds a preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress testing data and the online monitoring data, and the message production speed is controlled by adjusting the message construction rules to realize automatic configuration of the message production speed, thereby avoiding a large backlog of messages in the message middleware, improving system performance, stability and data reliability, and also improving the flexibility and accuracy of the stress test; according to the new message construction rules, a message is generated and transmitted to the message middleware, a message request is initiated, and the message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0032] Figure 2 is a flowchart of an embodiment of a message processing method based on stress testing according to the present application;

[0033] Figure 3 is a structural diagram of an embodiment of a message processing device based on stress testing according to the present application;

[0034] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0036] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0038] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0039] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0040] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0041] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0042] It should be noted that the message processing method based on stress testing provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the message processing device based on stress testing is generally arranged in the server / terminal device.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0044] Continue to refer Figure 2 , shows a flowchart of an embodiment of a message processing method based on stress testing according to the present application, comprising the following steps:

[0045] Step S201, receiving a stress test request, and extracting request content data in the stress test request.

[0046] Stress Testing is a software testing method that evaluates the performance, stability, reliability, and fault tolerance of a system by simulating the operation of the system under extreme load conditions.

[0047] In this embodiment, the message processing method based on the stress test is executed on the electronic device (eg Figure 1 The server / terminal device shown in the figure can stress test the request through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0048] The stress test request contains business data and data required for stress testing. The data required for stress testing include response time, throughput, concurrent users, resource utilization, error rate, performance inflection point, recovery time, etc. Response time refers to the time interval from the system receiving a request to returning a response, including server processing time, network transmission time, and client waiting time; throughput refers to the number of requests processed by the system per unit time; concurrent users refer to the number of users who send requests to the system at the same time; resource utilization includes CPU utilization, memory usage, disk I / O, etc.; error rate refers to the proportion of requests that have errors or failures during the stress test; performance inflection point refers to the load level at which system performance begins to decline significantly; recovery time refers to the time required for the system to return to normal after the stress test ends.

[0049] After obtaining the stress test request, extract the request content data in the stress test request for subsequent stress testing.

[0050] It should be emphasized that in order to further ensure the privacy and security of the requested content data, the above-mentioned requested content data can also be stored in a node of a blockchain.

[0051] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0052] Step S202, obtaining a preset message body structure template, extracting the business attributes of the message body structure template, and constructing a message body according to the business attributes and the request content data.

[0053] In this embodiment, the message body structure template is pre-set and stored in the message template library, and the message body structure template includes business attributes such as message field definition, field type and value range, and message fields, etc. The data required for the stress test is prepared by using file storage, and multiple fields can be expanded horizontally.

[0054] Specifically, multiple message body structure templates are obtained from a pre-established message template library, and the template business attributes and corresponding template message fields of each message body structure template are extracted; the request business attributes in the request content data are extracted, and the template logical relationship between the template business attributes is obtained, and the request logical relationship of the request business attributes is determined according to the template logical relationship; the target message field that matches the request field of the request content data is screened from the template message field, and the field content of the request field is filled into the corresponding target message field to generate corresponding message content fragments; the message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

[0055] Among them, the template business attributes are obtained, and it is determined which business attributes can form a contextual logical connection or a thinking chain to form a logical relationship. According to the logical relationship, the logical order between the business attributes can be determined. The request business attributes in the request content data are extracted, and the request logical relationship between the request business attributes is obtained according to the logical relationship between the business attributes of the message body structure template. According to the request logical relationship, the connection relationship between the target message fields is determined, and the field content of the request field is filled into the corresponding target message field to generate the corresponding message content fragments. The message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

[0056] It should be noted that for some stress testing scenarios that require the uniqueness of the message body field key, it can be used in conjunction with a random function to construct a unique message body without duplication.

[0057] In some optional implementations, the field type of the target message field is obtained to form a field type list; the field type corresponding to the field content of the request field is obtained to obtain the field type to be filled; the field type to be filled is matched with the field type of the corresponding field in the field type list; if the field types are consistent, the field content is filled into the corresponding target message field; if the field types are inconsistent, the field type of the field content is converted to match the field type of the target message field and then filled, until all target message fields are filled, and a filled message body is generated; the message body is output for use in subsequent business processes.

[0058] Constructing the message body of the request content data through a preset message body structure template can improve the flexibility of the message body construction, increase the response speed, and thus improve the efficiency and accuracy of the stress test.

[0059] Step S203, obtaining historical processing data corresponding to the message body, and determining whether the processing load of the message body exceeds a preset threshold value based on the historical processing data. When the processing load exceeds the preset threshold value, the message features of the message body are added to the message construction rules.

[0060] In this embodiment, historical processing data refers to multiple data records or instances of a message body that has been processed in the past. Specifically, a clustering algorithm is used to perform cluster analysis on the historical processing data to obtain processing data sets of different message categories; the average processing time and average resource consumption of the processing data set of each message category are statistically analyzed, and the average processing time and average resource consumption are used as the preset load threshold of the message category; the message body is parsed to obtain the target message category to which the message body belongs, and the target preset load threshold is obtained according to the target message category; the processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed the target preset load threshold.

[0061] The processing load includes the processing time and resource consumption of the message body. The K-means clustering algorithm is used to set a preset number of clusters. The historical processing data is divided into a preset number of message categories by calculating the Euclidean distance between the historical processing data. For each message category, the average value of the message body processing time and the CPU, memory and other resource consumption is calculated. For example, if the average processing time of class A is 150ms, the CPU occupancy rate is 35%, and the memory occupancy is 20MB, then it is used as the preset load threshold of class A messages.

[0062] Parse the message body and extract its message features such as message length and format. Determine the message category to which the message body belongs by calculating the Euclidean distance with the center point of each message category. For example, if the message body belongs to Category A, the preset load threshold of Category A is used as a reference to monitor the processing status of the message body in real time.

[0063] By determining the message category of the message body based on historical processing data, and determining the corresponding target preset load threshold based on the category of the message body, and performing real-time monitoring based on the target preset load threshold, more accurate and objective thresholds can be obtained, and the bottleneck of message processing can be discovered in time, so as to dynamically adjust resource allocation, improve system stability, and optimize service quality.

[0064] In this embodiment, the processing load exceeding the preset threshold value indicates that the processing time of the message body exceeds the average processing time and / or the resource consumption of the message body exceeds the average resource consumption. When the processing load exceeds the preset threshold value, the message characteristics of the message body are added to the message construction rules to optimize the subsequent processing of similar messages. The message construction rules include message content design, message sending rules, message reception and verification rules, etc., wherein the message content design includes business scenario simulation, data format and size, and message key field filling, etc.; the message sending rules include the sending frequency, the sending method (synchronous sending or asynchronous sending), and the retry mechanism, etc.; the message reception and verification rules include the receiving method, verification rules, and error handling, etc.

[0065] In some optional implementations, before the step of obtaining the historical processing data corresponding to the message body, the method further includes:

[0066] Input the message body into the trained decision tree model to determine whether the message body exceeds the limit; the decision tree model uses the features of the message body as nodes and whether it exceeds the limit as the classification basis;

[0067] If the limit is exceeded, the processing priority of the message body is adjusted; if the limit is not exceeded, the step of obtaining the historical processing data corresponding to the message body is executed.

[0068] Specifically, all the message body data that has been processed historically is obtained, the message features in the message body data are extracted, and the over-limit labels of the message body data are obtained to form sample data; a decision tree model is generated by training using a decision tree algorithm such as the ID3 algorithm, the C5 algorithm, or the CART algorithm, such as the root node for processing time and resource consumption, the child node for message format, message length, and message category, and the leaf node for whether the limit is exceeded. The path of the trained decision tree model is used to determine whether the message body is likely to exceed the limit. If the message length is greater than 1,000 characters and the format is JSON, there is a high probability of exceeding the limit. The system automatically raises its priority, starts early warning monitoring, and takes corresponding measures in a timely manner.

[0069] The decision tree model is used to predict the over-limit of the message body before processing it, thereby improving efficiency, reducing the risk of failure and saving resources.

[0070] Step S204, obtaining stress testing data and online monitoring data of the message body, extracting key features of the message body from the stress testing data and the online monitoring data, adjusting the message construction rules according to the key features, and obtaining new message construction rules.

[0071] In this embodiment, stress testing data is obtained from the stress testing platform, and online monitoring data is obtained from the online monitoring system. The stress testing data and the online monitoring data are merged to obtain merged data, and the merged data is preprocessed. Outliers and noise data are filtered out by cleaning to obtain preprocessed data.

[0072] Extract key features of the message body from the preprocessed data, such as throughput, response time, etc., and adjust the message construction rules based on the key features. The adjustment method can be to modify the field value of the message body, add or delete fields, adjust the length of the message body, adjust the sending frequency of the message body, etc., to obtain new message construction rules.

[0073] Step S205: Generate a message according to the message body in accordance with the new message construction rule, transmit the message to the message middleware, and initiate a message request.

[0074] In this embodiment, a new message construction rule is used to generate messages to achieve flow control and dynamically control the message flow sending speed. By controlling the message production speed, it is possible to avoid the message production speed being faster than the message consumption speed, so that a large number of messages will not be accumulated in the message middleware, thereby improving the accuracy of consumer performance evaluation.

[0075] Step S206, obtaining the concurrency of the message request, dynamically adjusting the number of distributed cluster nodes according to the concurrency, and determining the processing order according to the priority parameters of the message request.

[0076] Specifically, the number of message requests and response time are collected, the number of requests per unit time is calculated based on the number of message requests and the response time, and the current concurrency is obtained; the mapping relationship between the preset concurrency threshold interval and the number of nodes is obtained; if the current concurrency is lower than the lowest threshold of the preset concurrency threshold interval, the number of distributed cluster nodes is reduced; if the current concurrency is higher than the highest threshold of the preset concurrency threshold interval, the number of distributed cluster nodes is increased; if the current concurrency is within the preset concurrency threshold interval, the current number of distributed cluster nodes is maintained unchanged.

[0077] For example, the number of message requests in one second is 1200 times, and the average response time is 50 milliseconds. The current concurrency is 1200 requests / second*0.05 seconds=60.

[0078] The mapping relationship between the preset concurrency threshold range and the number of nodes is: when the concurrency is less than or equal to 30, the number of nodes is 2; when the concurrency is between 31 and 90, the number of nodes is 3; when the concurrency is greater than 90, the number of nodes is 4. Assuming that the current concurrency is 60, which is between 31 and 90, the current number of nodes remains unchanged at 3.

[0079] In this embodiment, the message requests currently waiting to be processed are sorted according to the priority parameter carried in the message request. For example, if the priority parameter is between 0 and 10, the smaller the value, the higher the priority. If the priority parameters are the same, the message requests are sorted according to the order of arrival time, and the request that arrives first has a higher priority. The sorted message requests are stored in a priority queue.

[0080] By dynamically adjusting the number of distributed cluster nodes according to the degree of concurrency, high concurrent requests can be achieved while improving the overall performance and stability of the cluster.

[0081] Step S207, obtaining the message corresponding to the message request from the message middleware based on the processing order, and distributing the message to each distributed cluster node for processing according to the load balancing algorithm.

[0082] In this embodiment, according to the processing order, the message corresponding to the message request is obtained from the message middleware, and the message is distributed to each distributed cluster node for processing according to the load balancing algorithm. Specifically, the message features of the message request are extracted, and the hash value of the message features is calculated using the consistent hashing algorithm; the target cluster node corresponding to the message request is determined based on the hash value; the message corresponding to the message request is sent to the target cluster node, and the message is processed by the target cluster node.

[0083] Among them, the consistent hashing algorithm is used as the load balancing algorithm, and the message identifier can be used as the message feature to calculate the hash value of the message identifier. After the hash value is calculated, the number of nodes is modulo and the message corresponding to the message request is distributed to the corresponding cluster node. After the cluster node receives the message, it will execute the corresponding business logic or task to process these messages.

[0084] By distributing messages through processing order and load balancing algorithm, we can ensure the consistency of business logic, improve system reliability, optimize resource utilization, increase system throughput, and enhance system scalability.

[0085] The present application realizes dynamic and automatic construction of the message body and improves the flexibility of the message body construction by extracting the request content data in the stress test request and constructing the message body according to the request content data and the preset message body structure template; when the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and the online monitoring data; the message production speed is controlled by adjusting the message construction rules, and the automatic configuration of the message production speed is realized, so as to avoid a large backlog of messages in the message middleware, improve the system performance, stability and data reliability, and also improve the flexibility and accuracy of the stress test; according to the new message construction rules, the message is generated and transmitted to the message middleware, and a message request is initiated, and the message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system.

[0086] In some optional implementations, after the step of distributing the message corresponding to the message request to each distributed cluster node for processing according to the load balancing algorithm, the step further includes:

[0087] Obtain message processing feedback results, compare the message processing feedback results with expected results, and filter out error message requests.

[0088] Record the processing log of message requests, including the arrival time, priority parameters, processing time, processing results and other information of the message requests, for subsequent performance analysis and troubleshooting.

[0089] By obtaining the message processing feedback results, it is possible to automatically detect and feedback whether the message sent by the producer is successful or failed, which helps stress testers quickly determine the results of message sending, and promptly discover and handle potential performance bottlenecks and failures, thereby improving test efficiency and accuracy.

[0090] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0092] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0093] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a message processing device based on stress testing. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0094] like Figure 3As shown, the message processing device 300 based on stress testing described in this embodiment includes: an extraction module 301, a construction module 302, a judgment module 303, a rule adjustment module 304, a generation module 305, a node adjustment module 306 and a distribution module 307. Among them:

[0095] The extraction module 301 is used to receive a stress test request and extract request content data in the stress test request;

[0096] The construction module 302 is used to obtain a preset message body structure template, extract the business attributes of the message body structure template, and construct a message body according to the business attributes and the request content data;

[0097] The judgment module 303 is used to obtain the historical processing data corresponding to the message body, and judge whether the processing load of the message body exceeds the preset threshold according to the historical processing data. If the processing load exceeds the preset threshold, the message feature of the message body is added to the message construction rule;

[0098] The rule adjustment module 304 is used to obtain the stress test data and online monitoring data of the message body, extract the key features of the message body from the stress test data and the online monitoring data, and adjust the message construction rule according to the key features to obtain a new message construction rule;

[0099] The generation module 305 is used to generate a message according to the message body according to the new message construction rule, transmit the message to the message middleware, and initiate a message request;

[0100] The node adjustment module 306 is used to obtain the concurrency of the message request, dynamically adjust the number of distributed cluster nodes according to the concurrency, and determine the processing order according to the priority parameter of the message request;

[0101] The distribution module 307 is used to obtain the message corresponding to the message request from the message middleware based on the processing order, and distribute the message to each of the distributed cluster nodes for processing according to the load balancing algorithm.

[0102] It should be emphasized that in order to further ensure the privacy and security of the requested content data, the above-mentioned requested content data can also be stored in a node of a blockchain.

[0103] The above-mentioned stress test-based message processing device 300 extracts the request content data in the stress test request, and constructs the message body according to the request content data and the preset message body structure template, so as to realize dynamic and automatic construction of the message body and improve the flexibility of the message body construction; when the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and the online monitoring data. The message production speed is controlled by adjusting the message construction rules, and the automatic configuration of the message production speed is realized, so as to avoid a large backlog of messages in the message middleware, improve the system performance, stability and data reliability, and also improve the flexibility and accuracy of the stress test; according to the new message construction rules, the message is generated and transmitted to the message middleware, and a message request is initiated. The message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system.

[0104] In some optional implementations of this embodiment, the construction module 302 is further used to:

[0105] Acquire multiple message body structure templates from a pre-established message template library, and extract template business attributes and corresponding template message fields of each of the message body structure templates;

[0106] Extracting the request service attributes in the request content data, and obtaining the template logical relationship between the template service attributes, and determining the request logical relationship of the request service attributes according to the template logical relationship;

[0107] Filtering a target message field that matches a request field of the request content data from the template message field, filling the field content of the request field into the corresponding target message field, and generating a corresponding message content fragment;

[0108] The message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

[0109] Constructing the message body of the request content data through a preset message body structure template can improve the flexibility of the message body construction, increase the response speed, and thus improve the efficiency and accuracy of the stress test.

[0110] In some optional implementations of this embodiment, the determination module 303 is further configured to:

[0111] Using a clustering algorithm to perform cluster analysis on the historical processing data to obtain processing data sets of different message categories;

[0112] Statistically analyzing the average processing time and average resource consumption of the processing data set of each message category, and using the average processing time and the average resource consumption as a preset load threshold of the message category;

[0113] Parsing the message body, obtaining a target message category to which the message body belongs, and obtaining a target preset load threshold according to the target message category;

[0114] The processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed a target preset load threshold.

[0115] By determining the message category of the message body based on historical processing data, and determining the corresponding target preset load threshold based on the category of the message body, and performing real-time monitoring based on the target preset load threshold, more accurate and objective thresholds can be obtained, and the bottleneck of message processing can be discovered in time, so as to dynamically adjust resource allocation, improve system stability, and optimize service quality.

[0116] In some optional implementations, the message processing apparatus 300 based on stress testing further includes a decision module, which is used to:

[0117] Input the message body into a trained decision tree model to determine whether the message body exceeds the limit; wherein the decision tree model uses the features of the message body as nodes and whether it exceeds the limit as a classification basis;

[0118] If the limit is exceeded, the processing priority of the message body is adjusted; if the limit is not exceeded, the step of obtaining the historical processing data corresponding to the message body is executed.

[0119] The decision tree model is used to predict the over-limit of the message body before processing it, thereby improving efficiency, reducing the risk of failure and saving resources.

[0120] In some optional implementations of this embodiment, the node adjustment module 306 is further used to:

[0121] Collect the number of message requests and the response time, calculate the number of requests per unit time according to the number of message requests and the response time, and obtain the current concurrency;

[0122] Obtain the mapping relationship between the preset concurrency threshold interval and the number of nodes;

[0123] If the current concurrency is lower than the lowest threshold of the preset concurrency threshold interval, reducing the number of distributed cluster nodes;

[0124] If the current concurrency is higher than the highest threshold of the preset concurrency threshold interval, increase the number of distributed cluster nodes;

[0125] If the current concurrency is within the preset concurrency threshold range, the current number of distributed cluster nodes is maintained unchanged.

[0126] By dynamically adjusting the number of distributed cluster nodes according to the degree of concurrency, high concurrent requests can be achieved while improving the overall performance and stability of the cluster.

[0127] In some optional implementations of this embodiment, the distribution module 307 is further used to:

[0128] Extracting a message feature of the message request, and calculating a hash value of the message feature using a consistent hashing algorithm;

[0129] Determine the target cluster node corresponding to the message request according to the hash value;

[0130] The message corresponding to the message request is sent to the target cluster node, and the message is processed by the target cluster node.

[0131] By distributing messages through processing order and load balancing algorithm, we can ensure the consistency of business logic, improve system reliability, optimize resource utilization, increase system throughput, and enhance system scalability.

[0132] In some optional implementations, the stress test-based message processing apparatus 300 further includes a feedback module for obtaining a message processing feedback result, comparing the message processing feedback result with an expected result, and screening out error message requests.

[0133] By obtaining the message processing feedback results, it is possible to automatically detect and feedback whether the message sent by the producer is successful or failed, which helps stress testers quickly determine the results of message sending, and promptly discover and handle potential performance bottlenecks and failures, thereby improving test efficiency and accuracy.

[0134] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0135] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0136] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0137] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a message processing method based on stress testing, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0138] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the message processing method based on stress testing.

[0139] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0140] By extracting the request content data in the stress test request and constructing the message body according to the request content data and the preset message body structure template, the message body can be dynamically and automatically constructed to improve the flexibility of the message body construction; when the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and online monitoring data. The message production speed is controlled by adjusting the message construction rules to realize automatic configuration of the message production speed, avoid a large backlog of messages in the message middleware, improve system performance, stability and data reliability, and also improve the flexibility and accuracy of the stress test; according to the new message construction rules, messages are generated and transmitted to the message middleware, and a message request is initiated. The message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing system pressure, and improving the stability and security of the system.

[0141] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the message processing method based on stress testing as described above.

[0142] By extracting the request content data in the stress test request and constructing the message body according to the request content data and the preset message body structure template, the message body can be dynamically and automatically constructed to improve the flexibility of the message body construction; when the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and online monitoring data. The message production speed is controlled by adjusting the message construction rules to realize automatic configuration of the message production speed, avoid a large backlog of messages in the message middleware, improve system performance, stability and data reliability, and also improve the flexibility and accuracy of the stress test; according to the new message construction rules, messages are generated and transmitted to the message middleware, and a message request is initiated. The message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing system pressure, and improving the stability and security of the system.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0144] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A message processing method based on stress testing, characterized in that: The steps include: Receive a stress test request, and extract request content data in the stress test request; Obtaining a preset message body structure template, extracting business attributes of the message body structure template, and constructing a message body according to the business attributes and the request content data; Acquire historical processing data corresponding to the message body, determine whether a processing load of the message body exceeds a preset threshold value according to the historical processing data, and add a message feature of the message body to a message construction rule when the processing load exceeds the preset threshold value; Obtaining stress testing data and online monitoring data of the message body, extracting key features of the message body from the stress testing data and the online monitoring data, adjusting the message construction rule according to the key features, and obtaining a new message construction rule; According to the new message construction rule, a message is generated according to the message body, the message is transmitted to the message middleware, and a message request is initiated; Obtaining the concurrency of the message request, dynamically adjusting the number of distributed cluster nodes according to the concurrency, and determining the processing order according to the priority parameter of the message request; The message corresponding to the message request is obtained from the message middleware based on the processing order, and the message is distributed to each of the distributed cluster nodes for processing according to a load balancing algorithm.

2. The message processing method based on stress testing according to claim 1 is characterized in that: The steps of obtaining a preset message body structure template, extracting the service attributes of the message body structure template, and constructing a message body according to the service attributes and the request content data include: Acquire multiple message body structure templates from a pre-established message template library, and extract template business attributes and corresponding template message fields of each of the message body structure templates; Extracting the request service attributes in the request content data, and obtaining the template logical relationship between the template service attributes, and determining the request logical relationship of the request service attributes according to the template logical relationship; Filtering a target message field that matches a request field of the request content data from the template message field, filling the field content of the request field into the corresponding target message field, and generating a corresponding message content fragment; The message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

3. The message processing method based on stress testing according to claim 1 is characterized in that: The step of judging whether the processing load of the message body exceeds a preset threshold according to the historical processing data comprises: Using a clustering algorithm to perform cluster analysis on the historical processing data to obtain processing data sets of different message categories; Statistically analyzing the average processing time and average resource consumption of the processing data set of each message category, and using the average processing time and the average resource consumption as a preset load threshold of the message category; Parsing the message body, obtaining a target message category to which the message body belongs, and obtaining a target preset load threshold according to the target message category; The processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed a target preset load threshold.

4. The message processing method based on stress testing according to claim 1 is characterized in that: Before the step of obtaining the historical processing data corresponding to the message body, the method further includes: Input the message body into a trained decision tree model to determine whether the message body exceeds the limit; wherein the decision tree model uses the features of the message body as nodes and whether it exceeds the limit as a classification basis; If the limit is exceeded, the processing priority of the message body is adjusted; if the limit is not exceeded, the step of obtaining the historical processing data corresponding to the message body is executed.

5. The message processing method based on stress testing according to claim 1 is characterized in that: The step of obtaining the concurrency of the message request and dynamically adjusting the number of distributed cluster nodes according to the concurrency includes: Collect the number of message requests and the response time, calculate the number of requests per unit time according to the number of message requests and the response time, and obtain the current concurrency; Obtain the mapping relationship between the preset concurrency threshold interval and the number of nodes; If the current concurrency is lower than the lowest threshold of the preset concurrency threshold interval, reducing the number of distributed cluster nodes; If the current concurrency is higher than the highest threshold of the preset concurrency threshold interval, increase the number of distributed cluster nodes; If the current concurrency is within the preset concurrency threshold range, the current number of distributed cluster nodes is maintained unchanged.

6. The message processing method based on stress testing according to claim 1 is characterized in that: The step of distributing the message corresponding to the message request to each of the distributed cluster nodes for processing according to the load balancing algorithm includes: Extracting a message feature of the message request, and calculating a hash value of the message feature using a consistent hashing algorithm; Determine the target cluster node corresponding to the message request according to the hash value; The message corresponding to the message request is sent to the target cluster node, and the message is processed by the target cluster node.

7. The message processing method based on stress testing according to claim 1 is characterized in that: After the step of distributing the message corresponding to the message request to each of the distributed cluster nodes for processing according to the load balancing algorithm, the step further includes: Obtain message processing feedback results, compare the message processing feedback results with expected results, and filter out error message requests.

8. A message processing device based on stress testing, characterized in that: include: An extraction module, used for receiving a stress test request and extracting request content data in the stress test request; A construction module, used to obtain a preset message body structure template, extract the business attributes of the message body structure template, and construct a message body according to the business attributes and the request content data; A judgment module, used for obtaining historical processing data corresponding to the message body, judging whether the processing load of the message body exceeds a preset threshold value according to the historical processing data, and adding the message feature of the message body to the message construction rule when the processing load exceeds the preset threshold value; A rule adjustment module, used to obtain stress test data and online monitoring data of the message body, extract key features of the message body from the stress test data and the online monitoring data, and adjust the message construction rule according to the key features to obtain a new message construction rule; A generation module, used to generate a message according to the message body in accordance with the new message construction rule, transmit the message to the message middleware, and initiate a message request; A node adjustment module, used to obtain the concurrency of the message request, dynamically adjust the number of distributed cluster nodes according to the concurrency, and determine the processing order according to the priority parameter of the message request; A distribution module is used to obtain the message corresponding to the message request from the message middleware based on the processing order, and distribute the message to each of the distributed cluster nodes for processing according to a load balancing algorithm.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the message processing method based on stress testing as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the message processing method based on stress testing according to any one of claims 1 to 7 are implemented.

Images