Message processing method, device, equipment and storage medium based on stress testing

Abstract

The present application belongs to the field of data processing and finance, and relates to a message processing method based on stress testing, including constructing a message body according to business attributes and request content data extracted from a stress test request; when it is determined based on historical processing data that the processing load of the message body exceeds a preset threshold, the message characteristics of the message body are added to the message construction rules; the key characteristics of the message body are extracted from stress testing data and online monitoring data to adjust the message construction rules; according to the new message construction rules, messages are generated and message requests are initiated; the number of cluster nodes is dynamically adjusted according to the concurrency, the message corresponding to the message request is obtained based on the processing order, and distributed to the cluster nodes according to the load balancing algorithm. The present application also provides a message processing device, equipment and storage medium based on stress testing. In addition, the present application also relates to blockchain technology, and the request content data can be stored in the blockchain. The present application realizes the dynamic construction of the message body and controls the message production speed.

Description

Technical Field

[0001] The present application relates to the fields of data processing and financial technology, and in particular to a message processing method, apparatus, device and storage medium based on stress testing. Background Art

[0002] With the rapid development of internet technology, the financial sector is increasingly demanding efficient, stable, and scalable technical architectures. The use of middleware (Message Queue, or MQ) in financial system construction is becoming increasingly widespread, particularly in key scenarios such as system decoupling, asynchronous communication, peak load shaving, and data order assurance. These scenarios not only require MQ to efficiently process large amounts of data, but also place extremely high demands on consumer processing performance to ensure low latency and high reliability of financial transactions.

[0003] In the actual operation of financial services, the performance of MQ message consumers is directly related to system stability and responsiveness. Therefore, it is particularly important to conduct comprehensive stress testing on MQ message consumers before the application is launched. Stress testing aims to simulate extreme conditions such as high concurrency and large data volumes in real business scenarios to assess the system's load capacity and performance bottlenecks.

[0004] However, traditional open-source stress testing tools, such as JMeter, have significant limitations when performing stress testing on mainstream MQs like RocketMQ. Specifically, JMeter lacks a universal RocketMQ message sending plugin. This prevents testers from conveniently and automatically constructing message bodies through data-driven methods, nor does it allow for dynamic configuration of message sending traffic. This limitation significantly limits the flexibility and accuracy of stress testing.

[0005] In specific stress testing scenarios within the financial sector, it's often necessary to simulate real-world online traffic scenarios to more closely mirror actual business environments. This requires precise control over the speed of message production and delivery to prevent producers from sending messages too quickly, leading to a massive message backlog in the MQ middleware. Severe message backlogs can not only affect performance evaluation results for consumers but also pose a potential threat to the stability and security of the financial system.

[0006] Furthermore, traditional stress testing tools also have shortcomings when it comes to detecting message sending results. They often cannot automatically detect and provide feedback on whether a message sent by a producer succeeds or fails, making it difficult for stress testers to quickly determine the results of message sending, thus affecting testing efficiency and accuracy. Summary of the Invention

[0007] The purpose of the embodiments of the present application is to propose a message processing method, device, equipment and storage medium based on stress testing to solve the technical problem that the relevant technology cannot dynamically and automatically construct the message body, nor can it realize the dynamic configuration of the message sending speed and flow, which limits the flexibility and accuracy of the stress test.

[0008] In order to solve the above technical problems, the embodiment of the present application provides a message processing method based on stress testing, which adopts the following technical solutions:

[0009] Receive a stress test request, and extract request content data from the stress test request;

[0010] Obtaining a preset message body structure template, extracting business attributes of the message body structure template, and constructing a message body according to the business attributes and the request content data;

[0011] Obtaining historical processing data corresponding to the message body, determining whether a processing load of the message body exceeds a preset threshold based on the historical processing data, and adding a message feature of the message body to a message construction rule when the processing load exceeds the preset threshold;

[0012] Obtaining stress testing data and online monitoring data of the message body, extracting key features of the message body from the stress testing data and the online monitoring data, and adjusting the message construction rule according to the key features to obtain a new message construction rule;

[0013] Generate a message based on the message body according to the new message construction rule, transmit the message to the message middleware, and initiate a message request;

[0014] Obtaining the concurrency of the message request, dynamically adjusting the number of distributed cluster nodes according to the concurrency, and determining the processing order according to the priority parameter of the message request;

[0015] The message corresponding to the message request is obtained from the message middleware based on the processing order, and the message is distributed to each of the distributed cluster nodes for processing according to a load balancing algorithm.

[0016] In order to solve the above technical problems, the embodiment of the present application further provides a message processing device based on stress testing, which adopts the following technical solution:

[0017] An extraction module is used to receive a stress test request and extract request content data in the stress test request;

[0018] A construction module, configured to obtain a preset message body structure template, extract the service attributes of the message body structure template, and construct a message body according to the service attributes and the request content data;

[0019] a judgment module, configured to obtain historical processing data corresponding to the message body, determine whether a processing load of the message body exceeds a preset threshold based on the historical processing data, and add a message feature of the message body to a message construction rule if the processing load exceeds the preset threshold;

[0020] a rule adjustment module, configured to obtain stress testing data and online monitoring data of the message body, extract key features of the message body from the stress testing data and the online monitoring data, and adjust the message construction rules according to the key features to obtain new message construction rules;

[0021] A generation module, configured to generate a message according to the message body in accordance with the new message construction rule, transmit the message to the message middleware, and initiate a message request;

[0022] A node adjustment module is used to obtain the concurrency of the message request, dynamically adjust the number of distributed cluster nodes according to the concurrency, and determine the processing order according to the priority parameter of the message request;

[0023] A distribution module is used to obtain the message corresponding to the message request from the message middleware based on the processing order, and distribute the message to each of the distributed cluster nodes for processing according to a load balancing algorithm.

[0024] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0025] The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the processor executes the computer-readable instructions, the steps of the message processing method based on stress testing are implemented.

[0026] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0027] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the message processing method based on stress testing described above are implemented.

[0028] Compared with the prior art, this application has the following beneficial effects:

[0029] The present application provides a message processing method based on stress testing, which extracts request content data from a stress testing request and constructs a message body according to the request content data and a preset message body structure template, thereby realizing dynamic and automatic construction of the message body and improving the flexibility of the message body construction; when the processing load of the message body exceeds a preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress testing data and the online monitoring data. By adjusting the message construction rules to control the message production speed, the automatic configuration of the message production speed is realized, and a large backlog of messages in the message middleware is avoided, thereby improving the system performance, stability and data reliability, and also improving the flexibility and accuracy of the stress test; according to the new message construction rules, a message is generated and transmitted to the message middleware, a message request is initiated, and the message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0032] Figure 2 is a flowchart of an embodiment of a message processing method based on stress testing according to the present application;

[0033] Figure 3 This is a structural diagram of an embodiment of a message processing device based on stress testing according to the present application;

[0034] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0036] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0038] like Figure 1 As shown, system architecture 100 may include a terminal device 101, a network 102, and a server 103. Terminal device 101 may be a laptop computer 1011, a tablet computer 1012, or a mobile phone 1013. Network 102 is a medium for providing a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0039] The user can use the terminal device 101 to interact with the server 103 via the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0040] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0041] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0042] It should be noted that the message processing method based on stress testing provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the message processing device based on stress testing is generally set in the server / terminal device.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0044] Continue to refer Figure 2 , shows a flowchart of an embodiment of a message processing method based on stress testing according to the present application, comprising the following steps:

[0045] Step S201: receiving a stress test request and extracting request content data in the stress test request.

[0046] Stress testing is a software testing method that evaluates the system's performance, stability, reliability, and fault tolerance by simulating the system's operation under extreme load conditions.

[0047] In this embodiment, the message processing method based on the stress test is run on the electronic device (eg Figure 1 The server / terminal device shown in the figure can stress test the request via a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other wireless connection methods currently known or developed in the future.

[0048] The stress test request contains business data and data required for stress testing. The data required for stress testing includes response time, throughput, concurrent users, resource utilization, error rate, performance inflection point, recovery time, etc. Response time refers to the time interval from when the system receives a request to when it returns a response, including server processing time, network transmission time, and client waiting time; throughput refers to the number of requests processed by the system per unit time; concurrent users refers to the number of users who simultaneously send requests to the system; resource utilization includes CPU utilization, memory usage, disk I / O, etc.; error rate refers to the proportion of requests that encounter errors or fail during the stress test; performance inflection point refers to the load level at which system performance begins to significantly decline; recovery time refers to the time required for the system to return to normal after the stress test.

[0049] After receiving the stress test request, extract the request content data in the stress test request for subsequent stress testing.

[0050] It should be emphasized that in order to further ensure the privacy and security of the requested content data, the above-mentioned requested content data can also be stored in a node of a blockchain.

[0051] The blockchain referred to in this application is a new application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0052] Step S202: obtaining a preset message body structure template, extracting the service attributes of the message body structure template, and constructing a message body according to the service attributes and the request content data.

[0053] In this embodiment, the message body structure template is pre-set and stored in the message template library. It contains business attributes such as message field definitions, field types, and value ranges, as well as message fields. The data required for stress testing is prepared using file storage, which can be expanded horizontally across multiple fields.

[0054] Specifically, multiple message body structure templates are obtained from a pre-established message template library, and the template business attributes and corresponding template message fields of each message body structure template are extracted; the request business attributes in the request content data are extracted, and the template logical relationship between the template business attributes is obtained, and the request logical relationship of the request business attributes is determined according to the template logical relationship; the target message field that matches the request field of the request content data is filtered from the template message field, and the field content of the request field is filled into the corresponding target message field to generate corresponding message content fragments; the message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

[0055] The template business attributes are obtained, and it is determined which business attributes can form a contextual logical connection or a chain of thought to form a logical relationship. The logical order between the business attributes can be determined based on the logical relationship. The request business attributes in the request content data are extracted, and the request logical relationship between the request business attributes is obtained according to the logical relationship between the business attributes of the message body structure template. The connection relationship between the target message fields is determined based on the request logical relationship. The field content of the request field is filled into the corresponding target message field to generate the corresponding message content fragments. The message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

[0056] It should be noted that for some stress testing scenarios that require the uniqueness of the message body field key, it can be combined with a random function to construct a unique message body without duplication.

[0057] In some optional implementations, the field type of the target message field is obtained to form a field type list; the field type corresponding to the field content of the request field is obtained to obtain the field type to be filled; the field type to be filled is matched with the field type of the corresponding field in the field type list; if the field types are consistent, the field content is filled into the corresponding target message field; if the field types are inconsistent, the field type of the field content is converted to match the field type of the target message field and then filled, until all target message fields are filled, and a filled message body is generated; the message body is output for use in subsequent business processes.

[0058] Constructing the message body of the request content data through a preset message body structure template can improve the flexibility of the message body construction, increase the response speed, and thus improve the efficiency and accuracy of the stress test.

[0059] Step S203, obtaining historical processing data corresponding to the message body, and judging whether the processing load of the message body exceeds a preset threshold value based on the historical processing data. When the processing load exceeds the preset threshold value, the message features of the message body are added to the message construction rules.

[0060] In this embodiment, historical processing data refers to multiple data records or instances of message bodies being processed in the past. Specifically, a clustering algorithm is used to perform cluster analysis on the historical processing data to obtain processing data sets for different message categories. The average processing time and average resource consumption of the processing data sets for each message category are statistically analyzed, and the average processing time and average resource consumption are used as the preset load threshold for the message category. The message body is parsed to obtain the target message category to which the message body belongs, and the target preset load threshold is determined based on the target message category. The processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed the target preset load threshold.

[0061] The processing load includes the message body processing time and resource consumption. Using the K-means clustering algorithm, a preset number of clusters is set. The historical processing data is divided into a preset number of message categories by calculating the Euclidean distance between them. For each message category, the average message body processing time, CPU usage, memory usage, and other resource consumption are calculated. For example, if the average processing time for Category A is 150ms, CPU usage is 35%, and memory usage is 20MB, these values ​​are used as the preset load threshold for Category A messages.

[0062] Parse the message body and extract its message features such as message length and format. Determine the message category to which the message body belongs by calculating the Euclidean distance with the center point of each message category. For example, if the message body belongs to Category A, the preset load threshold of Category A is used as a reference to monitor the processing status of the message body in real time.

[0063] By determining the message category of the message body based on historical processing data, and determining the corresponding target preset load threshold based on the category of the message body, and performing real-time monitoring based on the target preset load threshold, more accurate and objective thresholds can be obtained, and bottlenecks in message processing can be discovered in a timely manner, so as to dynamically adjust resource allocation, improve system stability, and optimize service quality.

[0064] In this embodiment, the processing load exceeding the preset threshold value indicates that the processing time of the message body exceeds the average processing time and / or the resource consumption of the message body exceeds the average resource consumption. When the processing load exceeds the preset threshold value, the message characteristics of the message body are added to the message construction rules to optimize the subsequent processing of similar messages. The message construction rules include message content design, message sending rules, message reception and verification rules, etc., wherein the message content design includes business scenario simulation, data format and size, and message key field filling, etc.; the message sending rules include sending frequency, sending method (synchronous sending or asynchronous sending), and retry mechanism, etc.; the message reception and verification rules include receiving method, verification rules, and error handling, etc.

[0065] In some optional implementations, before the step of obtaining the historical processing data corresponding to the message body, the method further includes:

[0066] Input the message body into the trained decision tree model to determine whether the message body exceeds the limit. The decision tree model uses the features of the message body as nodes and whether it exceeds the limit as the classification basis.

[0067] If the limit is exceeded, the processing priority of the message body is adjusted; if the limit is not exceeded, the step of obtaining the historical processing data corresponding to the message body is executed.

[0068] Specifically, all historically processed message data is obtained, message features are extracted from the message data, and over-limit labels for the message data are obtained to form sample data. A decision tree model is trained using a decision tree algorithm such as the ID3, C5, or CART algorithms. For example, the root node represents processing time and resource consumption, the child nodes represent message format, message length, and message category, and the leaf nodes represent whether the message exceeds the limit. The trained decision tree model's path determines whether the message body is likely to exceed the limit. If the message is longer than 1000 characters and is formatted as JSON, there is a high probability of exceeding the limit. The system automatically raises its priority, initiates early warning monitoring, and takes appropriate measures in a timely manner.

[0069] The decision tree model is used to predict message body overruns before processing them, thereby improving efficiency while reducing failure risks and saving resources.

[0070] Step S204 , obtaining stress test data and online monitoring data of the message body, extracting key features of the message body from the stress test data and online monitoring data, adjusting the message construction rules according to the key features, and obtaining new message construction rules.

[0071] In this embodiment, stress testing data is obtained from the stress testing platform, and online monitoring data is obtained from the online monitoring system. The stress testing data and the online monitoring data are merged to obtain merged data. The merged data is preprocessed, and outliers and noise data are filtered out by cleaning to obtain preprocessed data.

[0072] Extract key features of the message body from the preprocessed data, such as throughput and response time. Adjust the message construction rules based on the key features. The adjustment methods can be to modify the field values ​​of the message body, add or delete fields, adjust the length of the message body, adjust the sending frequency of the message body, etc., to obtain new message construction rules.

[0073] Step S205 : Generate a message based on the message body according to the new message construction rule, transmit the message to the message middleware, and initiate a message request.

[0074] In this embodiment, new message construction rules are used to generate messages, thereby achieving flow control and dynamically controlling the message flow rate. By controlling the message production rate, it is possible to prevent message production from being faster than message consumption, preventing a large backlog of messages in the message middleware, and improving the accuracy of consumer performance evaluation.

[0075] Step S206 , obtaining the concurrency of the message request, dynamically adjusting the number of distributed cluster nodes according to the concurrency, and determining the processing order according to the priority parameters of the message request.

[0076] Specifically, the number of message requests and response time are collected, the number of requests per unit time is calculated based on the number of message requests and the response time, and the current concurrency is obtained; the mapping relationship between the preset concurrency threshold interval and the number of nodes is obtained; if the current concurrency is lower than the lowest threshold of the preset concurrency threshold interval, the number of distributed cluster nodes is reduced; if the current concurrency is higher than the highest threshold of the preset concurrency threshold interval, the number of distributed cluster nodes is increased; if the current concurrency is within the preset concurrency threshold interval, the current number of distributed cluster nodes is maintained unchanged.

[0077] For example, if the number of message requests in a certain second is 1200 and the average response time is 50 milliseconds, the current concurrency is 1200 requests / second*0.05 seconds=60.

[0078] The mapping relationship between the preset concurrency threshold range and the number of nodes is: when the concurrency is less than or equal to 30, the number of nodes is 2; when the concurrency is between 31 and 90, the number of nodes is 3; and when the concurrency is greater than 90, the number of nodes is 4. Assuming that the current concurrency is 60, which is between 31 and 90, the current number of nodes remains unchanged at 3.

[0079] In this embodiment, message requests currently awaiting processing are sorted based on the priority parameter carried in the message requests. For example, if the priority parameter is between 0 and 10, the smaller the value, the higher the priority. If the priority parameters are the same, the message requests are sorted based on the time of arrival, with the request arriving earlier receiving a higher priority. The sorted message requests are then stored in a priority queue.

[0080] By dynamically adjusting the number of distributed cluster nodes according to the degree of concurrency, high concurrent requests can be achieved while improving the overall performance and stability of the cluster.

[0081] Step S207 , obtaining the message corresponding to the message request from the message middleware based on the processing order, and distributing the message to each distributed cluster node for processing according to the load balancing algorithm.

[0082] In this embodiment, according to the processing order, the message corresponding to the message request is obtained from the message middleware, and the message is distributed to each distributed cluster node for processing according to the load balancing algorithm. Specifically, the message features of the message request are extracted, and the hash value of the message features is calculated using the consistent hashing algorithm; the target cluster node corresponding to the message request is determined based on the hash value; the message corresponding to the message request is sent to the target cluster node, and the message is processed by the target cluster node.

[0083] Among them, the consistent hashing algorithm is used as the load balancing algorithm, and the message identifier can be used as the message feature to calculate the hash value of the message identifier. After the hash value is calculated, the modulus is taken by the number of nodes, and the message corresponding to the message request is distributed to the corresponding cluster node. After the cluster node receives the message, it will execute the corresponding business logic or task to process these messages.

[0084] Distributing messages through processing order and load balancing algorithms can ensure the consistency of business logic, improve system reliability, optimize resource utilization, increase system throughput, and enhance system scalability.

[0085] This application extracts the request content data in the stress test request, and constructs the message body according to the request content data and the preset message body structure template, so as to realize dynamic and automatic construction of the message body and improve the flexibility of the message body construction; when the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and the online monitoring data. By adjusting the message construction rules to control the message production speed, the automatic configuration of the message production speed is realized, and a large number of messages are avoided from being accumulated in the message middleware, thereby improving the system performance, stability and data reliability, and also improving the flexibility and accuracy of the stress test; according to the new message construction rules, messages are generated and transmitted to the message middleware, and a message request is initiated. The message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system.

[0086] In some optional implementations, after the step of distributing the message corresponding to the message request to each distributed cluster node for processing according to the load balancing algorithm, the step further includes:

[0087] Obtain message processing feedback results, compare them with expected results, and filter out error message requests.

[0088] Record the message request processing log, including the message request arrival time, priority parameters, processing time, processing results and other information, for subsequent performance analysis and troubleshooting.

[0089] By obtaining the message processing feedback results, it is possible to automatically detect and provide feedback on whether the message sent by the producer is successful or failed, which helps stress testers quickly determine the results of message sending, promptly discover and handle potential performance bottlenecks and failures, thereby improving test efficiency and accuracy.

[0090] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0091] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0092] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0093] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a message processing device based on stress testing. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0094] like Figure 3As shown, the message processing device 300 based on stress testing in this embodiment includes: an extraction module 301, a construction module 302, a judgment module 303, a rule adjustment module 304, a generation module 305, a node adjustment module 306 and a distribution module 307. Among them:

[0095] The extraction module 301 is used to receive a stress test request and extract request content data in the stress test request;

[0096] The construction module 302 is used to obtain a preset message body structure template, extract the business attributes of the message body structure template, and construct a message body according to the business attributes and the request content data;

[0097] The judgment module 303 is used to obtain historical processing data corresponding to the message body, and determine whether the processing load of the message body exceeds a preset threshold based on the historical processing data. If the processing load exceeds the preset threshold, the message feature of the message body is added to the message construction rule;

[0098] The rule adjustment module 304 is configured to obtain stress testing data and online monitoring data of the message body, extract key features of the message body from the stress testing data and the online monitoring data, and adjust the message construction rules according to the key features to obtain new message construction rules;

[0099] The generation module 305 is used to generate a message according to the message body in accordance with the new message construction rule, transmit the message to the message middleware, and initiate a message request;

[0100] The node adjustment module 306 is used to obtain the concurrency of the message request, dynamically adjust the number of distributed cluster nodes according to the concurrency, and determine the processing order according to the priority parameter of the message request;

[0101] The distribution module 307 is used to obtain the message corresponding to the message request from the message middleware based on the processing order, and distribute the message to each of the distributed cluster nodes for processing according to the load balancing algorithm.

[0102] It should be emphasized that in order to further ensure the privacy and security of the requested content data, the above-mentioned requested content data can also be stored in a node of a blockchain.

[0103] The above-mentioned stress test-based message processing device 300 extracts the request content data in the stress test request, and constructs the message body according to the request content data and the preset message body structure template, thereby realizing dynamic and automatic construction of the message body and improving the flexibility of the message body construction; when the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and the online monitoring data. By adjusting the message construction rules to control the message production speed, the automatic configuration of the message production speed is realized, and a large number of messages are avoided from being accumulated in the message middleware, thereby improving the system performance, stability and data reliability, and also improving the flexibility and accuracy of the stress test; according to the new message construction rules, messages are generated and transmitted to the message middleware, a message request is initiated, and the message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages are processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system.

[0104] In some optional implementations of this embodiment, the construction module 302 is further configured to:

[0105] Acquire multiple message body structure templates from a pre-established message template library, and extract template business attributes and corresponding template message fields of each message body structure template;

[0106] Extracting the request service attributes from the request content data, obtaining the template logical relationship between the template service attributes, and determining the request logical relationship of the request service attributes according to the template logical relationship;

[0107] Filtering a target message field that matches the request field of the request content data from the template message field, filling the field content of the request field into the corresponding target message field, and generating a corresponding message content fragment;

[0108] The message content fragments are spliced ​​according to the request logical relationship to form a complete message body.

[0109] Constructing the message body of the request content data through a preset message body structure template can improve the flexibility of the message body construction, increase the response speed, and thus improve the efficiency and accuracy of the stress test.

[0110] In some optional implementations of this embodiment, the determination module 303 is further configured to:

[0111] Performing cluster analysis on the historical processing data using a clustering algorithm to obtain processing data sets of different message categories;

[0112] Statistically analyzing the average processing time and average resource consumption of the processing data set of each message category, and using the average processing time and the average resource consumption as a preset load threshold of the message category;

[0113] Parsing the message body to obtain a target message category to which the message body belongs, and obtaining a target preset load threshold according to the target message category;

[0114] The processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed a target preset load threshold.

[0115] By determining the message category of the message body based on historical processing data, and determining the corresponding target preset load threshold based on the category of the message body, and performing real-time monitoring based on the target preset load threshold, more accurate and objective thresholds can be obtained, and bottlenecks in message processing can be discovered in a timely manner, so as to dynamically adjust resource allocation, improve system stability, and optimize service quality.

[0116] In some optional implementations, the stress test-based message processing apparatus 300 further includes a decision module configured to:

[0117] Input the message body into a trained decision tree model to determine whether the message body exceeds the limit; wherein the decision tree model uses the characteristics of the message body as nodes and whether it exceeds the limit as the classification basis;

[0118] If the limit is exceeded, the processing priority of the message body is adjusted; if the limit is not exceeded, the step of obtaining the historical processing data corresponding to the message body is executed.

[0119] The decision tree model is used to predict message body overruns before processing them, thereby improving efficiency while reducing failure risks and saving resources.

[0120] In some optional implementations of this embodiment, the node adjustment module 306 is further configured to:

[0121] Collect the number of message requests and the response time, calculate the number of requests per unit time based on the number of message requests and the response time, and obtain the current concurrency;

[0122] Get the mapping relationship between the preset concurrency threshold interval and the number of nodes;

[0123] If the current concurrency is lower than the lowest threshold of the preset concurrency threshold interval, reducing the number of distributed cluster nodes;

[0124] If the current concurrency is higher than the highest threshold of the preset concurrency threshold interval, increase the number of distributed cluster nodes;

[0125] If the current concurrency is within the preset concurrency threshold range, the current number of distributed cluster nodes is maintained unchanged.

[0126] By dynamically adjusting the number of distributed cluster nodes according to the degree of concurrency, high concurrent requests can be achieved while improving the overall performance and stability of the cluster.

[0127] In some optional implementations of this embodiment, the distribution module 307 is further configured to:

[0128] Extracting a message feature of the message request, and calculating a hash value of the message feature using a consistent hashing algorithm;

[0129] Determine the target cluster node corresponding to the message request according to the hash value;

[0130] The message corresponding to the message request is sent to the target cluster node, and the target cluster node processes the message.

[0131] Distributing messages through processing order and load balancing algorithms can ensure the consistency of business logic, improve system reliability, optimize resource utilization, increase system throughput, and enhance system scalability.

[0132] In some optional implementations, the stress-test-based message processing apparatus 300 further includes a feedback module for obtaining a message processing feedback result, comparing the message processing feedback result with an expected result, and screening out error message requests.

[0133] By obtaining the message processing feedback results, it is possible to automatically detect and provide feedback on whether the message sent by the producer is successful or failed, which helps stress testers quickly determine the results of message sending, promptly discover and handle potential performance bottlenecks and failures, thereby improving test efficiency and accuracy.

[0134] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0135] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0136] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0137] The memory 41 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for a message processing method based on stress testing. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0138] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or process data, such as executing computer-readable instructions for the stress test-based message processing method.

[0139] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0140] By extracting the request content data in the stress test request and constructing the message body according to the request content data and the preset message body structure template, the message body can be dynamically and automatically constructed, thereby improving the flexibility of the message body construction. When the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and online monitoring data. By adjusting the message construction rules to control the message production speed, the automatic configuration of the message production speed is realized, and a large backlog of messages in the message middleware is avoided, thereby improving the system performance, stability and data reliability, and also improving the flexibility and accuracy of the stress test. According to the new message construction rules, messages are generated and transmitted to the message middleware, and a message request is initiated. The message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages can be processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system.

[0141] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the message processing method based on stress testing as described above.

[0142] By extracting the request content data in the stress test request and constructing the message body according to the request content data and the preset message body structure template, the message body can be dynamically and automatically constructed, thereby improving the flexibility of the message body construction. When the processing load of the message body exceeds the preset threshold, the message characteristics of the message body are added to the message construction rules, and the message construction rules are adjusted according to the stress test data and online monitoring data. By adjusting the message construction rules to control the message production speed, the automatic configuration of the message production speed is realized, and a large backlog of messages in the message middleware is avoided, thereby improving the system performance, stability and data reliability, and also improving the flexibility and accuracy of the stress test. According to the new message construction rules, messages are generated and transmitted to the message middleware, and a message request is initiated. The message distribution strategy is adjusted by calculating the concurrency of the message request, so that the messages can be processed in a timely and efficient manner while having high concurrency, thereby improving the response speed, reducing the system pressure, and improving the stability and security of the system.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0144] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A message processing method based on stress testing, characterized in that: The steps include: Receive a stress test request, and extract request content data from the stress test request; Obtaining a preset message body structure template, extracting business attributes of the message body structure template, and constructing a message body according to the business attributes and the request content data; Obtaining historical processing data corresponding to the message body, determining whether a processing load of the message body exceeds a preset threshold based on the historical processing data, and adding a message feature of the message body to a message construction rule when the processing load exceeds the preset threshold; Obtaining stress testing data and online monitoring data of the message body, extracting key features of the message body from the stress testing data and the online monitoring data, and adjusting the message construction rule according to the key features to obtain a new message construction rule; Generate a message based on the message body according to the new message construction rule, transmit the message to the message middleware, and initiate a message request; Collect the number of message requests and response time, calculate the number of requests per unit time based on the number of message requests and the response time, obtain the current concurrency, dynamically adjust the number of distributed cluster nodes based on the concurrency, and determine the processing order based on the priority parameters of the message requests; Obtaining a message corresponding to the message request from the message middleware based on the processing order, and distributing the message to each of the distributed cluster nodes for processing according to a load balancing algorithm; The step of determining whether the processing load of the message body exceeds a preset threshold according to the historical processing data includes: Performing cluster analysis on the historical processing data using a clustering algorithm to obtain processing data sets of different message categories; Statistically analyzing the average processing time and average resource consumption of the processing data set of each message category, and using the average processing time and the average resource consumption as a preset load threshold of the message category; Parsing the message body to obtain a target message category to which the message body belongs, and obtaining a target preset load threshold according to the target message category; The processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed a target preset load threshold.

2. The message processing method based on stress testing according to claim 1, characterized in that: The steps of obtaining a preset message body structure template, extracting the service attributes of the message body structure template, and constructing the message body according to the service attributes and the request content data include: Acquire multiple message body structure templates from a pre-established message template library, and extract template business attributes and corresponding template message fields of each message body structure template; Extracting the request service attributes from the request content data, obtaining the template logical relationship between the template service attributes, and determining the request logical relationship of the request service attributes according to the template logical relationship; Filtering a target message field that matches the request field of the request content data from the template message field, filling the field content of the request field into the corresponding target message field, and generating a corresponding message content fragment; The message content fragments are spliced ​​together according to the request logical relationship to form a complete message body.

3. The message processing method based on stress testing according to claim 1, characterized in that: Before the step of obtaining the historical processing data corresponding to the message body, the method further includes: Input the message body into a trained decision tree model to determine whether the message body exceeds the limit; wherein the decision tree model uses the characteristics of the message body as nodes and whether it exceeds the limit as the classification basis; If the limit is exceeded, the processing priority of the message body is adjusted; if the limit is not exceeded, the step of obtaining the historical processing data corresponding to the message body is executed.

4. The message processing method based on stress testing according to claim 1, characterized in that: The step of dynamically adjusting the number of distributed cluster nodes according to the concurrency includes: Get the mapping relationship between the preset concurrency threshold interval and the number of nodes; If the current concurrency is lower than the lowest threshold of the preset concurrency threshold interval, reducing the number of distributed cluster nodes; If the current concurrency is higher than the highest threshold of the preset concurrency threshold interval, increase the number of distributed cluster nodes; If the current concurrency is within the preset concurrency threshold range, the current number of distributed cluster nodes is maintained unchanged.

5. The message processing method based on stress testing according to claim 1, characterized in that: The step of distributing the message corresponding to the message request to each of the distributed cluster nodes for processing according to the load balancing algorithm includes: Extracting a message feature of the message request, and calculating a hash value of the message feature using a consistent hashing algorithm; Determine the target cluster node corresponding to the message request according to the hash value; The message corresponding to the message request is sent to the target cluster node, and the target cluster node processes the message.

6. The message processing method based on stress testing according to claim 1, characterized in that: After the step of distributing the message corresponding to the message request to each of the distributed cluster nodes for processing according to the load balancing algorithm, the method further includes: Obtain message processing feedback results, compare the message processing feedback results with expected results, and filter out error message requests.

7. A message processing device based on stress testing, characterized in that: include: An extraction module is used to receive a stress test request and extract request content data in the stress test request; A construction module, configured to obtain a preset message body structure template, extract the service attributes of the message body structure template, and construct a message body according to the service attributes and the request content data; a judgment module, configured to obtain historical processing data corresponding to the message body, determine whether a processing load of the message body exceeds a preset threshold based on the historical processing data, and add a message feature of the message body to a message construction rule if the processing load exceeds the preset threshold; a rule adjustment module, configured to obtain stress testing data and online monitoring data of the message body, extract key features of the message body from the stress testing data and the online monitoring data, and adjust the message construction rules according to the key features to obtain new message construction rules; A generation module, configured to generate a message according to the message body in accordance with the new message construction rule, transmit the message to the message middleware, and initiate a message request; A node adjustment module is used to collect the number of message requests and response time, calculate the number of requests per unit time based on the number of message requests and the response time, obtain the current concurrency, dynamically adjust the number of distributed cluster nodes based on the concurrency, and determine the processing order based on the priority parameters of the message requests; A distribution module, configured to obtain a message corresponding to the message request from the message middleware based on the processing order, and distribute the message to each of the distributed cluster nodes for processing according to a load balancing algorithm; The judgment module is further used to: Performing cluster analysis on the historical processing data using a clustering algorithm to obtain processing data sets of different message categories; Statistically analyzing the average processing time and average resource consumption of the processing data set of each message category, and using the average processing time and the average resource consumption as a preset load threshold of the message category; Parsing the message body to obtain a target message category to which the message body belongs, and obtaining a target preset load threshold according to the target message category; The processing time and resource consumption of the message body are monitored in real time to determine whether the processing time and resource consumption exceed a target preset load threshold.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the message processing method based on stress testing according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the message processing method based on stress testing according to any one of claims 1 to 6.

Images