Short message sending test method, system, equipment and medium

Abstract

The invention is applicable to the technical field of short message transmission, and provides a short message sending test method, which comprises the following steps: aggregating a plurality of test users according to user attributes to form a plurality of user groups, and configuring different short message channels for different user groups; configuring different short message templates for each short message channel, and distributing a plurality of test users corresponding to the short message channels to different short message templates according to a preset distribution proportion; in the testing process, a network simulation method and a user behavior simulation method are adopted, network fluctuation and operation behaviors of corresponding user groups are simulated, testing data are obtained, a multi-target optimization model is constructed based on the performance of a short message channel, common tags of the user groups and the testing data, and a multi-target optimization result is obtained. And obtaining a matching scheme of the short message template, the user group and the short message channel. Through the test method, more accurate and personalized short message service can be provided for the user, the delivery rate and the reading rate of the short message are improved, and the participation degree and the loyalty degree of the user are enhanced.

Description

Technical Field

[0001] The present application belongs to the technical field of SMS transmission, and in particular, relates to a test method, system, device and medium for sending SMS. Background Art

[0002] In the current digital communication field, SMS, as a widely used means of communication, plays an important role in multiple business scenarios such as marketing notifications, business reminders, and emergency notifications. In different business scenarios, there are differences between different user groups. For example, new users, old users, and users in different regions have different preferences and acceptance levels for SMS content. At the same time, the quality of SMS channels is also uneven. The delivery rate, delay, and cost effect of different channels vary significantly. Even the performance of the same channel may fluctuate in different time periods. With the sharp increase in the amount of SMS sent, it is necessary to ensure the quality of SMS delivery and improve the arrival rate and reading rate of SMS.

[0003] In the SMS system of the prior art, the SMS channel is usually manually configured to send the corresponding SMS content. The manual configuration method is highly random and has defects such as low efficiency and high cost effect. In addition, there is a lack of correlation between user groups, SMS channels and SMS content, which will affect the user experience. The delivery rate and reading rate of the corresponding marketing SMS cannot be guaranteed. Summary of the invention

[0004] The embodiments of the present application provide a test method, system, device and medium for sending SMS messages, which can solve the problem of lack of correlation between the above-mentioned user groups, SMS channels and SMS contents, provide users with more accurate and personalized SMS services, and improve the delivery rate and reading rate of SMS messages.

[0005] In a first aspect, an embodiment of the present application provides a method for testing text message sending, including:

[0006] Loading preset test users and SMS channels, aggregating multiple test users according to user attributes to form multiple user groups, and configuring different SMS channels for different user groups;

[0007] Configuring a different SMS template for each SMS channel, and allocating a plurality of test users corresponding to the SMS channel to different SMS templates according to a preset diversion ratio;

[0008] Generate a test task scheduling schedule according to a preset test task scheduling rule, wherein the test task scheduling schedule is used to send a test SMS to the test user within a preset time, wherein the test SMS is a SMS sent in a standard format using the SMS template;

[0009] During the test, a network simulation method and a user behavior simulation method are used to simulate network fluctuations and the operation behavior of the corresponding user group to obtain test data, wherein the test data includes the operation behavior of the test user and the performance data of the SMS channel and the SMS template;

[0010] Based on the performance of the SMS channel, the common labels of the user groups and the test data, a multi-objective optimization model is constructed, and a heuristic optimization algorithm is used to obtain a matching solution of the SMS template, the user group and the SMS channel.

[0011] Further, the loading of preset test users and SMS channels, aggregating a plurality of the test users according to user attributes to form a plurality of user groups, and configuring different SMS channels for the test users in different user groups include:

[0012] According to the user attributes of the test users, the test users are aggregated according to preset aggregation rules to generate aggregation results, wherein the aggregation results include multiple user groups, each of which includes a group identifier and user attributes;

[0013] Setting a channel identifier for each of the SMS channels and initializing a channel weight value for each of the SMS channels;

[0014] Monitor the load of each SMS channel in real time and dynamically adjust the channel weight value of each SMS channel, wherein the load includes the number of concurrent requests and the response time;

[0015] Based on the aggregation result and the channel weight value of each SMS channel, a weighted round-robin algorithm is adopted to divert the test users of each user group to different SMS channels.

[0016] Furthermore, after the test users are aggregated according to the preset aggregation rule to generate the aggregation result, the method further includes:

[0017] Extracting common features of the user groups to form aggregation rules between the user groups;

[0018] Adopting association rule mining technology, based on the historical behavior data of each user group, obtaining the association rules of the test users in each user group, wherein the association rules are the behavior patterns and preferences of each test user in the user group;

[0019] Based on the association rules of each of the user groups, generating common tags corresponding to the user groups;

[0020] storing each of the user groups, the aggregation rules, the association rules and the common tags in a user analysis database;

[0021] During the test, the behavior data of the test users of each user group is monitored in real time;

[0022] Using a decision tree algorithm, judging whether the aggregation rule needs to be updated based on the behavior data and the preset test target;

[0023] If the aggregation rule needs to be updated, the plurality of test users are re-aggregated based on the updated aggregation rule, and the data in the user analysis database is updated at the same time.

[0024] Further, configuring a different SMS template for each SMS channel and allocating a plurality of test users in the user group to different SMS templates according to a preset diversion ratio includes:

[0025] Establishing a configuration mapping table between the SMS channels and the SMS templates, wherein the configuration mapping table includes a list of SMS templates corresponding to each SMS channel;

[0026] During the test, for each SMS channel, a test user is randomly assigned to each SMS template in the SMS template list according to a preset diversion ratio;

[0027] For the preset diversion ratio, during the test process, the performance data of each sending combination is tracked in real time, the performance data is analyzed, and the diversion ratio is dynamically adjusted. The sending combination is the SMS channel and SMS template with a corresponding relationship in the configuration mapping table. The performance data includes SMS delivery rate, sending success rate and response time.

[0028] Furthermore, the test task scheduling rules include dependencies, priorities and execution times between the test tasks;

[0029] The test task scheduling schedule is generated according to the preset test task scheduling rules, and the test task scheduling schedule is used to send a test SMS to the test user within a preset time, and the test SMS is a SMS sent in a standard format using the SMS template, including:

[0030] Based on the test task arrangement rules, a topological sorting algorithm is used to determine the execution order of each test task and generate a test task scheduling plan table, which includes the test ID, task name, execution time and execution frequency of each test task;

[0031] Acquire test data for each test task, wherein the test data includes a sending combination and a target test user;

[0032] By using a regular expression matching and replacement method, the variables in the SMS template are replaced to generate a test SMS, and the test SMS is sent to a target test user.

[0033] Furthermore, during the test, a network simulation method and a user behavior simulation method are used to simulate network fluctuations and the operation behavior of the corresponding user group to obtain test data, and the test data includes the operation behavior of the test user and the performance data of the SMS channel and the SMS template, including:

[0034] Using network simulation technology, configuring network fluctuation parameters for the test environment, and obtaining test data under the test environment, wherein the network fluctuation parameters include network transmission delay and packet loss rate;

[0035] Dynamically adjusting the network fluctuation parameter, based on the change trend of the network fluctuation parameter, dynamically optimizing the network fluctuation parameter by analyzing the test data;

[0036] Based on the common tags of different user groups, a user behavior simulation engine is used to randomly generate operation behavior data for the test users in each of the user groups, and the operation behavior data includes sending SMS, receiving SMS, reading SMS and deleting SMS;

[0037] A machine learning algorithm is used to comprehensively analyze the network fluctuation parameters and the operation behavior data to predict the performance data of each user group in each sending combination and the performance data of each sending combination under different network environments.

[0038] Furthermore, based on the performance of the SMS channel, the common labels of the user groups and the test data, a multi-objective optimization model is constructed, and a heuristic optimization algorithm is used to obtain a matching solution of the SMS template, the user group and the SMS channel, including:

[0039] Calculate the term frequency-inverse document frequency of each SMS template using the TF-IDF algorithm to obtain a feature vector representation of each SMS template;

[0040] Based on the operation behavior of the test user, the click rate of each SMS template is calculated, and the click rate is associated with the feature vector representation to obtain a template effect factor;

[0041] Taking the common labels of each user group as user factors;

[0042] Acquire performance indicator data of each of the SMS channels, evaluate and analyze the performance indicator data, and obtain channel performance factors, wherein the performance indicator data includes a sending success rate, a delivery speed, and a cost effect;

[0043] Based on the template effect factors, user factors and the channel performance factors, a multi-objective optimization model is constructed, and a genetic algorithm is used to solve the multi-objective optimization model to determine the best matching solution for SMS templates, user groups and channels. The best matching solution is the optimal solution obtained by the genetic algorithm.

[0044] In a second aspect, an embodiment of the present application provides a test system for sending a text message, including:

[0045] The first processing module is used to load preset test users and SMS channels, aggregate multiple test users according to user attributes to form multiple user groups, and configure different SMS channels for different user groups;

[0046] The second processing module is used to configure a different SMS template for each SMS channel, and allocate multiple test users corresponding to the SMS channel to different SMS templates according to a preset diversion ratio;

[0047] The third processing module is used to generate a test task scheduling plan according to a preset test task scheduling rule, wherein the test task scheduling plan is used to send a test SMS to the test user within a preset time, wherein the test SMS is a SMS sent in a standard format using the SMS template;

[0048] The fourth processing module is used to use a network simulation method and a user behavior simulation method to simulate network fluctuations and operation behaviors of corresponding user groups during the test process to obtain test data, wherein the test data includes the operation behaviors of the test users and the performance data of the SMS channel and the SMS template;

[0049] The fifth processing module is used to construct a multi-objective optimization model based on the performance of the SMS channel, the common tags of the user group and the test data, and adopt a heuristic optimization algorithm to obtain a matching solution of the SMS template, the user group and the SMS channel.

[0050] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned test method for sending text messages when executing the computer program.

[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned test method for sending text messages is implemented.

[0052] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0053] The present invention discloses a test method for sending short messages, which forms multiple user groups with similar characteristics by accurately aggregating test users, and configuring different short message channels for different user groups, so as to ensure that each user group can obtain the communication service that best suits its characteristics, thereby improving communication efficiency and user satisfaction. In addition, each short message channel is configured with different short message templates, and the test users are assigned to different templates according to the preset diversion ratio, so that each combination of short message template and short message channel can obtain the corresponding test user for testing. In the test process, by adopting the network simulation method and the user behavior simulation method, the network fluctuation and the operation behavior of the user group are truly simulated, which helps to obtain more accurate and reliable test data, and the test data covers the operation behavior of the test user and the performance data of the short message channel and the short message template, which provides a rich data source for subsequent analysis and optimization. In addition, by constructing a target model, a plurality of factors are comprehensively considered, including communication efficiency, user satisfaction, cost-effectiveness, etc., so as to obtain the optimal matching scheme, and then the test method can provide users with more accurate and personalized short message services, improve the delivery rate and reading rate of short messages, and enhance user participation and loyalty. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 It is a flow chart of a test method for sending a text message provided by an embodiment of the present invention;

[0056] Figure 2 It is a structural schematic diagram of a test system for sending short messages provided by an embodiment of the present invention;

[0057] Figure 3 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0060] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0061] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0062] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0063] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0064] See also Figure 1 As shown, the present invention is a test method for sending a text message, comprising the following steps:

[0065] S100, loading preset test users and SMS channels, aggregating a plurality of the test users according to user attributes to form a plurality of user groups, and configuring different SMS channels for different user groups;

[0066] In this embodiment, the user group refers to a set of users preset in the SMS test environment in order to simulate real user behavior and verify the SMS sending effect. Usually, during the test process, the user group formed according to the preset aggregation rules is imported into the test environment for subsequent test operations.

[0067] In this embodiment, user group aggregation is usually divided according to specific attributes, such as age, gender, region, mobile phone model, etc., so that during the test process, the behavioral characteristics of different user groups when receiving text messages, such as open rate, click rate, conversion rate, etc., can be more accurately simulated, so as to more accurately evaluate the effect of text message sending. By analyzing the test results of different user groups, the preferences and reactions of different user groups to text messages can be understood, thereby optimizing the text message sending strategy and improving the arrival rate and conversion rate of text messages.

[0068] In some embodiments, the above step S100 includes:

[0069] Aggregating the test users according to the user attributes of the test users to generate an aggregation result, wherein the aggregation result includes a plurality of user groups, each of the user groups includes a group identifier and user attributes;

[0070] Setting a channel identifier for each of the SMS channels and initializing a channel weight value for each of the SMS channels;

[0071] Monitor the load of each SMS channel in real time, and dynamically adjust the channel weight value of each SMS channel based on the load and the cost effect of the SMS channel, wherein the load includes the number of concurrent requests and the response time;

[0072] Based on the aggregation result and the channel weight value of each SMS channel, a weighted round-robin algorithm is adopted to divert the test users in each user group to different SMS channels.

[0073] In this embodiment, corresponding user attributes are configured for each test user in the test user database, wherein the user attributes include age, gender, region and mobile phone model. When forming a user group, multiple test users are divided according to a preset aggregation rule to generate multiple user groups. A unique group identifier is set for each user group for subsequent calls to the corresponding user group and for association with SMS channels and SMS templates. Similarly, each SMS channel is also set with a communication identifier for user calls to the SMS channel.

[0074] In this embodiment, the user attributes of the test user are cleaned and preprocessed, such as removing noise data such as missing values ​​and outliers, and converting non-numeric data into numeric data, such as converting the gender "male" and "female" into 0 and 1 respectively, and then the Min-Max standardization method is used to standardize the preprocessed data, scaling the data to the interval [0, 1] to eliminate the dimensional influence between the attributes. The K-means clustering algorithm is used to cluster the users. Specifically, the number of clusters K is set, and the Euclidean distance is used to measure the similarity between user attributes. Through iterative optimization, the users are finally divided into multiple clusters, each cluster representing a user group.

[0075] In this embodiment, the channel weight value of each SMS channel is the same in the initial state. Then, the channel weight value of each SMS channel is dynamically adjusted according to the load condition and cost effect of each SMS channel. Specifically, the number of concurrent requests and the response time are compared with the preset thresholds respectively. If an indicator of the load condition of the SMS channel is lower than or higher than the preset threshold, the weight adjustment is triggered. At the same time, the SMS channel also increases or decreases the corresponding channel weight value according to the cost effect.

[0076] In one embodiment, the adjustment range of the channel weight value is set according to the degree of deviation of the corresponding indicator from the threshold value. Specifically, the calculation formula of the adjustment range of the channel weight value is as follows:

[0077] Adjustment range = (current indicator value - preset threshold) × (cost-effectiveness / cost-effectiveness threshold) × adjustment coefficient

[0078] The adjustment coefficient is a constant, which is set according to the test requirements and is used to dynamically adjust the corresponding adjustment range.

[0079] In this embodiment, for multiple test users in each user group, a weighted polling algorithm is used to divert the above-mentioned multiple test users to different SMS channels according to the channel weight values ​​of different SMS channels. For example, in one embodiment, the multiple SMS channels are channel A, channel B and channel C, and the multiple user groups are group 1, group 2 and group 3, wherein the ratio of the channel weight values ​​between the above-mentioned channels A, channel B and channel C is 2:1:1, then for each test user in group 1, there is a 50% probability of being assigned to channel A, and a 25% probability of being assigned to channel B and channel C. Similarly, for each test user in group 2 and group 3, there is a 50% probability of being assigned to channel A, and a 25% probability of being assigned to channel B and channel C. In the process of polling scheduling, the test users of each user group are scheduled separately, and a channel is selected for each test user according to the ratio of the channel weight value to process the corresponding test task. When all test users have been scheduled once, start again from the beginning and perform the next round of scheduling.

[0080] In this embodiment, during the SMS sending process, performance indicator data of each SMS channel, such as sending success rate, delivery speed and cost effect, etc., are collected in real time, and the performance indicator data is associated with the SMS channel configuration and stored to form a historical performance data set of the channel. The performance data is analyzed regularly to identify SMS channels with poor stability and high latency, and their channel weight values ​​are dynamically adjusted to reduce the scheduling frequency of the SMS channel, and the test users of each user group are switched to the SMS channel with better performance for a new round of testing.

[0081] In some embodiments, for an ongoing test task, if the channel weight value of the SMS channel changes, the change in the channel weight value is compared with the context information during the execution of the test task, and the specific impact of the change on the task execution is analyzed, such as changes in test users in each user group, adjustments to channel performance indicator data, etc. Based on this, it is determined whether it will affect the ongoing test task. Specifically, according to the requirements of the test task, an evaluation standard for the scope of impact is set. If the impact of the change on the test task is within the allowable evaluation standard, the task continues to be executed without interruption. If the impact of the change on the test task exceeds the allowable evaluation standard, the test task is marked as interruptible and restarted or adjusted at an appropriate time, such as when the current test phase ends or test resources are idle. In addition, before restarting, ensure that all necessary context information has been restored or updated.

[0082] In some embodiments, after aggregating the test users to generate an aggregated result, the method includes:

[0083] Extracting common features of the user groups to form aggregation rules between the user groups;

[0084] Adopting association rule mining technology, based on the historical behavior data of each user group, obtaining the association rules of the test users in each user group, wherein the association rules are the behavior patterns and preferences of each test user in the user group;

[0085] Based on the association rules of each of the user groups, generating common tags corresponding to the user groups;

[0086] storing each of the user groups, the aggregation rules, the association rules and the common tags in a user analysis database;

[0087] During the test, the behavior data of the test users of each user group is monitored in real time;

[0088] Using a decision tree algorithm, judging whether the aggregation rule needs to be updated based on the behavior data and the preset test target;

[0089] If the aggregation rule needs to be updated, the plurality of test users are re-aggregated based on the updated aggregation rule, and the data in the user analysis database is updated at the same time.

[0090] In this embodiment, for each user group, the distribution of user attributes in each test user is analyzed, and common characteristics among the test users are extracted through statistical analysis technology. For example, the age is mainly concentrated between 20 and 30 years old, and male users account for a high proportion. These common characteristics are extracted to form the aggregation rules of the user group.

[0091] In this embodiment, association rule mining technology is used to mine frequent item sets and association rules from the historical behavior data of the user group, and the behavior patterns and preferences of each test user in the user group are obtained, wherein the historical behavior data is the behavior of the test user reading text messages, such as receiving text messages, reading text messages, deleting text messages after reading, directly deleting text messages, or setting text messages as stars, etc. The above historical behavior data generated by each test user is cleaned to remove repeated, missing or abnormal data, and the behavior types in the above historical behavior data are encoded, such as encoding "receiving text messages" as 1, "reading text messages" as 2, "deleting text messages" as 3, "setting text messages as stars" as 4, etc. An association rule mining algorithm such as the Apriori algorithm or the FP-growth algorithm is used, and parameters such as the minimum support and the minimum confidence are set according to business requirements. The quantity and quality of the mined frequent item sets and association rules are determined by the above parameters. At the same time, the selected association rule algorithm and the set parameters are used to perform association rule mining on the historical behavior data of the user group, and the mined results will include frequent item sets and association rules. For example, the rule "If the user reads the text message, he / she may set it as a star" (support=x%, confidence=y%) may be found, where x% and y% are specific values ​​of support and confidence respectively, and the user's behavior pattern and preference can be obtained through the above association rules.

[0092] In this embodiment, each user group is profiled according to the association rules, and common tags are generated to form an accurate description of the user group. The above user groups, aggregation rules, association rules and common tags are associated and stored in the user analysis database to provide data support for subsequent SMS sending tests.

[0093] In this embodiment, a suitable decision tree algorithm is selected, such as ID3, C4.5, CART, etc., and the parameters such as the depth and splitting conditions of the decision tree are set according to the preset test target, wherein the preset test target is to test the efficiency and effect of SMS sending, such as improving the reading rate, click rate, conversion rate, etc. of SMS. The behavior data of each test user is used as training data to train the decision tree model, so that the decision tree model can predict the behavior data of the user group, obtain the predicted value of the SMS sending effect, compare the predicted value with the preset threshold, and determine whether the aggregation rule needs to be updated. If the predicted value is lower than the threshold, it indicates that the current aggregation rule is not suitable for the current user group, and the aggregation rule needs to be updated to improve the SMS sending effect.

[0094] S200, configuring a different SMS template for each SMS channel, and allocating a plurality of test users in the user group to different SMS templates according to a preset diversion ratio, so that each combination of SMS template and SMS channel can obtain a corresponding test user for testing;

[0095] In some embodiments, the step S200 includes:

[0096] Establishing a configuration mapping table between the SMS channels and the SMS templates, wherein the configuration mapping table includes a list of SMS templates corresponding to each SMS channel;

[0097] During the test, for each SMS channel, a test user is randomly assigned to each SMS template in the SMS template list according to a preset diversion ratio;

[0098] For the preset diversion ratio, during the test process, the performance data of each sending combination is tracked in real time, the performance data is analyzed, and the diversion ratio is dynamically adjusted. The sending combination is the SMS channel and SMS template with a corresponding relationship in the configuration mapping table. The performance data includes SMS delivery rate, sending success rate and response time.

[0099] It can be understood that the test users assigned to different SMS templates in the same SMS channel are obtained by diverting the test users of each user group using a weighted round-robin algorithm based on the channel weight value of each SMS channel in the above step S100.

[0100] In this embodiment, the correspondence data between the SMS channels and the SMS templates are stored in a JSON file. In one embodiment, the specific content is as follows: `{"channel_A":["template_a","template_b"],"channel_B":["template_c","template_d","template_e"],"channel_C":["template_a","template_c"]}, where channel_A and template_a are a sending combination.

[0101] In this embodiment, the performance data generated during the test of each sending combination is counted and analyzed to determine the sending combination with poor performance, and then dynamically adjust its diversion ratio. Specifically, the performance data is compared with the preset performance data. If one of the performance data is lower than the preset performance data, the probability of diverting it to the corresponding sending combination is reduced, thereby performing a new round of test tasks.

[0102] S300, generating a test task scheduling schedule according to a preset test task scheduling rule, wherein the test task scheduling schedule is used to send a test SMS to the test user within a preset time, wherein the test SMS is a SMS sent in a standard format using the SMS template;

[0103] In some implementations, the test task scheduling rules include dependencies, priorities, and execution times between test tasks;

[0104] The above step S300 includes:

[0105] Based on the test task arrangement rules, a topological sorting algorithm is used to determine the execution order of each test task and generate a test task scheduling plan table, which includes the test ID, task name, execution time and execution frequency of each test task;

[0106] Acquire test data for each test task, wherein the test data includes a sending combination and a target test user;

[0107] By using a regular expression matching and replacement method, the variables in the SMS template are replaced to generate a test SMS, and the test SMS is sent to a target test user.

[0108] In the present embodiment, the test task scheduling rules are obtained from the pre-set test task rule base, wherein the test task scheduling rules include information such as the dependency, priority, and execution time between each test task. For example, one of the test task scheduling rules defines that test task A depends on test task B, with a priority of 1 and an execution time of 09:00 to 11:00. According to the acquired test task scheduling rules, the Kahn topological sorting algorithm is adopted, and the time complexity is set to O(V+E), wherein V is the number of tasks, and E is the number of dependencies between tasks. All test tasks with an in-degree of 0 are obtained by the above-mentioned topological sorting algorithm, and are added to the task queue, and then the first task is continuously taken out from the task queue, and the in-degree of the task connected to its outgoing edge is reduced by 1, and the task with the in-degree reduced to 0 is added to the queue until the queue is empty, thereby obtaining an orderly test task sequence. According to the above-mentioned test task scheduling rules, the execution time and frequency of each test task are determined. If task A depends on task B, and the execution time of task B is t1 and the execution frequency is f1, then the execution time of task A must be later than t1+1 / f1 to ensure that task A is executed after task B is completed. Based on the above analysis, a test task scheduling table is generated, in which each item in the test task scheduling table includes information such as the test ID, task name, execution time and execution frequency of the test task.

[0109] In this embodiment, for each test task, corresponding test data is extracted from a preset test database, wherein the test data is the sending combination determined in the above step S100 and step S200 and the test users assigned to each sending combination. Specifically, the test user corresponding to the sending combination is used as the target test user, and the regular expression matching and replacement method is adopted to automatically fill in the variables in the SMS template to generate a test SMS, and the test SMS is sent to the target test user. For the variables in the SMS template, the SMS template usually sets corresponding variables according to its marketing needs, so that the SMS template can meet various needs. For example, in a SMS template, {{name}} is defined to represent the user name variable, and {{date}} represents the date variable. Then, the regular expression "\\{\\{name\\}\\}" can be used to match the user name variable, and then it can be replaced with a specific user name according to the test data. In some embodiments, the SMS template also has content that needs to be dynamically generated according to the test data, so it can also be processed by calling the corresponding Python function.

[0110] S400, during the test, using a network simulation method and a user behavior simulation method to simulate network fluctuations and operation behaviors of a corresponding user group to obtain test data, wherein the test data includes the operation behaviors of the test users and performance data of the SMS channel and the SMS template;

[0111] In some embodiments, the step S400 includes:

[0112] Using network simulation technology, configuring network fluctuation parameters for the test environment, and obtaining test data under the test environment, wherein the network fluctuation parameters include network transmission delay and packet loss rate;

[0113] Dynamically adjusting the network fluctuation parameter, based on the change trend of the network fluctuation parameter, dynamically optimizing the network fluctuation parameter by analyzing the test data;

[0114] Based on the common tags of different user groups, a user behavior simulation engine is used to randomly generate operation behavior data for the test users in each of the user groups, and the operation behavior data includes sending SMS, receiving SMS, reading SMS and deleting SMS;

[0115] A machine learning algorithm is used to comprehensively analyze the network fluctuation parameters and the operation behavior data to predict the performance data of each user group in each sending combination under different network environments.

[0116] In this embodiment, during the testing process, the network fluctuation parameters of the test environment need to be simulated through network simulation technology to make the test environment closer to the actual network environment, thereby making the testing process more realistic. Testers can evaluate the performance and stability of SMS sending in the actual network environment based on the above network fluctuation parameters.

[0117] In this embodiment, for the test user, it is necessary to use the user behavior simulation engine to randomly generate operation behavior data based on the common tags of the user group to which the test user belongs, and dynamically simulate the diverse behavior characteristics of the user group. It can be understood that the simulated behavior data is generated based on the common tags of the user group. Therefore, the generated operation behavior data can be closer to the operation habits of real users, thereby being able to more realistically restore the user's behavior in the actual use process, making the test results more authentic and credible.

[0118] In this embodiment, network simulation technology and user behavior simulation engine are combined to dynamically generate diverse user behavior data while simulating network fluctuations, thereby improving the authenticity and comprehensiveness of the test scenarios. Machine learning algorithms, such as decision trees and support vector machines, are used to comprehensively analyze network fluctuation parameters and operational behavior data to predict the performance data of each user group in each sending combination and the performance data of each sending combination under different network environments. In this process, the accuracy and effectiveness of the test are continuously improved to ensure that the test results can truly reflect the performance of the system in actual applications.

[0119] During the testing process, the performance data of each user group in each sending combination is obtained, and the feedback effect of each user group on different sending combinations is measured through the performance data. The above performance data includes the reading rate, click rate and conversion rate of the test SMS within each user group. In addition, the performance data of each sending combination includes the SMS delivery rate, sending success rate and response time of each sending combination for the test SMS.

[0120] S500: Based on the performance of the SMS channel, the common tags of the user groups and the test data, a multi-objective optimization model is constructed, and a heuristic optimization algorithm is used to obtain a matching solution of the SMS template, the user group and the SMS channel.

[0121] In some embodiments, the step S500 includes:

[0122] Calculate the term frequency-inverse document frequency of each SMS template using the TF-IDF algorithm to obtain a feature vector representation of each SMS template;

[0123] Based on the operation behavior of the test user, the click rate of each SMS template is calculated, and the click rate is associated with the feature vector representation to obtain a template effect factor;

[0124] Taking the common labels of each user group as user factors;

[0125] Acquire performance indicator data of each of the SMS channels, evaluate and analyze the performance indicator data, and obtain channel performance factors, wherein the performance indicator data includes a sending success rate, a delivery speed, and a cost effect;

[0126] Based on the template effect factors, user factors and the channel performance factors, a multi-objective optimization model is constructed, and a genetic algorithm is used to solve the multi-objective optimization model to determine the best matching solution for SMS templates, user groups and channels. The best matching solution is the optimal solution obtained by the genetic algorithm.

[0127] In this embodiment, each SMS template is segmented and the SMS template is split into multiple terms. Each term in each SMS template is counted and the number of times it appears in the template is calculated. Specifically, the formula for calculating the term frequency is: TF(t, d) = t / d, where t is the number of times any term appears in the SMS template, and d is the total number of words in the SMS template. In addition, the total number of SMS templates is calculated, and for each term, the number of SMS templates containing it is counted, and then the inverse document frequency of each term is calculated. The calculation formula for the inverse document frequency is: IDF(t) = log(total number of SMS templates / number of SMS templates containing term t). For each term in each SMS template, its TF-IDF value is calculated. The calculation formula for the TF-IDF value is: TF-IDF(t, d) = TF(t, d) × IDF(t).

[0128] In this embodiment, when constructing a feature vector, for determining feature terms, all terms that have appeared can be selected as feature terms, or they can be screened based on a certain threshold, such as the size of the TF-IDF value. For each SMS template, a feature vector is constructed based on the feature terms it contains and their corresponding TF-IDF values, where the dimension of the feature vector is equal to the number of feature terms, and the value of each dimension corresponds to the TF-IDF value of the term in the SMS template. If a certain SMS template does not contain a certain feature term, its value on the feature vector is 0. Then, for each SMS template, its corresponding feature vector representation is output.

[0129] In this embodiment, for each SMS template, its click-through rate is calculated based on the collected test user operation behavior data, and the calculated click-through rate is associated with the above-mentioned feature vector representation to form a template effect factor. The common label of the user group obtained by analysis is used as a user factor, and it is used as an influencing factor of the user group and SMS channel matching scheme for subsequent determination of the best matching scheme. For each SMS channel, its performance indicator data is obtained, including sending success rate, delivery speed and cost effect. The obtained performance indicator data is evaluated and analyzed to form a channel performance factor. Finally, based on the template effect factor, user factor and channel performance factor, a multi-objective optimization model is constructed to maximize the overall effect, such as click-through rate, user satisfaction, cost-effectiveness, etc.

[0130] In this embodiment, a heuristic optimization algorithm is used. If a genetic algorithm solves the multi-objective optimization problem in the above multi-objective optimization model, it can find the optimal solution in a complex search space. Specifically, a group of possible matching schemes are randomly generated as the initial population. According to the value of the multi-objective optimization model, individuals with higher fitness are selected as parents. The parent generation is crossover operated to generate new offspring. The offspring is mutated to increase the diversity of the population. The above process is repeated until a predetermined number of iterations is reached or the optimal solution that meets the conditions is found, which is the best matching scheme for SMS templates, user groups and SMS channels. It maximizes the overall effect based on comprehensive consideration of template effects, user characteristics and channel performance.

[0131] See also Figure 2 As shown, the present invention also provides a test system for sending short messages, the system comprising:

[0132] The first processing module 201 is used to load preset test users and SMS channels, aggregate multiple test users according to user attributes to form multiple user groups, and configure different SMS channels for different user groups;

[0133] The second processing module 202 is used to configure a different SMS template for each SMS channel, and allocate multiple test users corresponding to the SMS channel to different SMS templates according to a preset diversion ratio;

[0134] The third processing module 203 is used to generate a test task scheduling plan according to a preset test task scheduling rule, wherein the test task scheduling plan is used to send a test SMS to the test user within a preset time, wherein the test SMS is a SMS sent in a standard format using the SMS template;

[0135] The fourth processing module 204 is used to use a network simulation method and a user behavior simulation method to simulate network fluctuations and operation behaviors of corresponding user groups during the test process to obtain test data, wherein the test data includes the operation behaviors of the test users and the performance data of the SMS channel and the SMS template;

[0136] The fifth processing module 205 is used to construct a multi-objective optimization model based on the performance of the SMS channel, the common tags of the user groups and the test data, and adopt a heuristic optimization algorithm to obtain a matching solution of the SMS template, the user group and the SMS channel.

[0137] It is understandable that if Figure 1 The contents of the test method embodiment of SMS sending shown in the figure are applicable to the test system embodiment of SMS sending. The functions specifically implemented by the test system embodiment of SMS sending are similar to those of Figure 1The test method embodiment of the text message sending shown is the same as that of the embodiment of the text message sending shown in FIG. Figure 1 The beneficial effects achieved by the test method embodiment of text message sending shown are also the same.

[0138] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0139] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0140] See also Figure 3 As shown, an embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, a test method for sending text messages as described in any one of the above methods is implemented.

[0141] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0142] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit of the computer device 3 and an external storage device. The memory 302 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0144] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the test method for sending a short message as described in any one of the above methods is implemented.

[0145] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / computer device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, USB flash drive, mobile hard disk, disk or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0146] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A test method for sending a text message, characterized in that: include: Loading preset test users and SMS channels, aggregating multiple test users according to user attributes to form multiple user groups, and configuring different SMS channels for different user groups; Configuring a different SMS template for each SMS channel, and allocating a plurality of test users corresponding to the SMS channel to different SMS templates according to a preset diversion ratio; Generate a test task scheduling schedule according to a preset test task scheduling rule, wherein the test task scheduling schedule is used to send a test SMS to the test user within a preset time, wherein the test SMS is a SMS sent in a standard format using the SMS template; During the test, a network simulation method and a user behavior simulation method are used to simulate network fluctuations and the operation behavior of the corresponding user group to obtain test data, wherein the test data includes the operation behavior of the test user and the performance data of the SMS channel and the SMS template; Based on the performance of the SMS channel, the common labels of the user groups and the test data, a multi-objective optimization model is constructed, and a heuristic optimization algorithm is used to obtain a matching solution of the SMS template, the user group and the SMS channel.

2. The method according to claim 1, characterized in that The loading of preset test users and SMS channels, aggregating a plurality of the test users according to user attributes to form a plurality of user groups, and configuring different SMS channels for test users in different user groups, includes: According to the user attributes of the test users, the test users are aggregated according to preset aggregation rules to generate aggregation results, wherein the aggregation results include multiple user groups, each of which includes a group identifier and user attributes; Setting a channel identifier for each of the SMS channels and initializing a channel weight value for each of the SMS channels; Monitor the load of each SMS channel in real time and dynamically adjust the channel weight value of each SMS channel, wherein the load includes the number of concurrent requests and the response time; Based on the aggregation result and the channel weight value of each SMS channel, a weighted round-robin algorithm is adopted to divert the test users of each user group to different SMS channels.

3. The method according to claim 2, characterized in that After the test users are aggregated according to the preset aggregation rule to generate the aggregation result, the method includes: Extracting common features of the user groups to form aggregation rules between the user groups; Adopting association rule mining technology, based on the historical behavior data of each user group, obtaining the association rules of the test users in each user group, wherein the association rules are the behavior patterns and preferences of each test user in the user group; Based on the association rules of each of the user groups, generating common tags corresponding to the user groups; storing each of the user groups, the aggregation rules, the association rules and the common tags in a user analysis database; During the test, the behavior data of the test users of each user group is monitored in real time; Using a decision tree algorithm, judging whether the aggregation rule needs to be updated based on the behavior data and the preset test target; If the aggregation rule needs to be updated, the plurality of test users are re-aggregated based on the updated aggregation rule, and the data in the user analysis database is updated at the same time.

4. The method according to claim 1, characterized in that The configuring a different SMS template for each SMS channel and allocating a plurality of test users in the user group to different SMS templates according to a preset diversion ratio includes: Establishing a configuration mapping table between the SMS channels and the SMS templates, wherein the configuration mapping table includes a list of SMS templates corresponding to each SMS channel; During the test, for each SMS channel, a test user is randomly assigned to each SMS template in the SMS template list according to a preset diversion ratio; For the preset diversion ratio, during the test process, the performance data of each sending combination is tracked in real time, the performance data is analyzed, and the diversion ratio is dynamically adjusted. The sending combination is the SMS channel and SMS template with a corresponding relationship in the configuration mapping table. The performance data includes SMS delivery rate, sending success rate and response time.

5. The method according to claim 1, characterized in that The test task scheduling rules include dependencies, priorities and execution time between test tasks; The test task scheduling schedule is generated according to the preset test task scheduling rules, and the test task scheduling schedule is used to send a test SMS to the test user within a preset time, and the test SMS is a SMS sent in a standard format using the SMS template, including: Based on the test task arrangement rules, a topological sorting algorithm is used to determine the execution order of each test task and generate a test task scheduling plan table, which includes the test ID, task name, execution time and execution frequency of each test task; Acquire test data for each test task, wherein the test data includes a sending combination and a target test user; By using a regular expression matching and replacement method, the variables in the SMS template are replaced to generate a test SMS, and the test SMS is sent to a target test user.

6. The method according to any one of claims 1 to 5, characterized in that: During the test, a network simulation method and a user behavior simulation method are used to simulate network fluctuations and the operation behavior of the corresponding user group to obtain test data, which includes the operation behavior of the test user and the performance data of the SMS channel and the SMS template, including: Using network simulation technology, configuring network fluctuation parameters for the test environment, and obtaining test data under the test environment, wherein the network fluctuation parameters include network transmission delay and packet loss rate; Dynamically adjusting the network fluctuation parameter, based on the change trend of the network fluctuation parameter, dynamically optimizing the network fluctuation parameter by analyzing the test data; Based on the common tags of different user groups, a user behavior simulation engine is used to randomly generate operation behavior data for the test users in each of the user groups, and the operation behavior data includes sending SMS, receiving SMS, reading SMS and deleting SMS; A machine learning algorithm is used to comprehensively analyze the network fluctuation parameters and the operation behavior data to predict the performance data of each user group in each sending combination and the performance data of each sending combination under different network environments.

7. The method according to claim 1, characterized in that Based on the performance of the SMS channel, the common labels of the user groups and the test data, a multi-objective optimization model is constructed, and a heuristic optimization algorithm is used to obtain a matching solution of the SMS template, the user group and the SMS channel, including: Calculate the term frequency-inverse document frequency of each SMS template using the TF-IDF algorithm to obtain a feature vector representation of each SMS template; Based on the operation behavior of the test user, the click rate of each SMS template is calculated, and the click rate is associated with the feature vector representation to obtain a template effect factor; Taking the common labels of each user group as user factors; Acquire performance indicator data of each of the SMS channels, evaluate and analyze the performance indicator data, and obtain channel performance factors, wherein the performance indicator data includes a sending success rate, a delivery speed, and a cost effect; Based on the template effect factors, user factors and the channel performance factors, a multi-objective optimization model is constructed, and a genetic algorithm is used to solve the multi-objective optimization model to determine the best matching solution for SMS templates, user groups and channels. The best matching solution is the optimal solution obtained by the genetic algorithm.

8. A test system for sending short messages, characterized in that: include: The first processing module is used to load preset test users and SMS channels, aggregate multiple test users according to user attributes to form multiple user groups, and configure different SMS channels for different user groups; The second processing module is used to configure a different SMS template for each SMS channel, and allocate multiple test users corresponding to the SMS channel to different SMS templates according to a preset diversion ratio; The third processing module is used to generate a test task scheduling plan according to a preset test task scheduling rule, wherein the test task scheduling plan is used to send a test SMS to the test user within a preset time, wherein the test SMS is a SMS sent in a standard format using the SMS template; The fourth processing module is used to use a network simulation method and a user behavior simulation method to simulate network fluctuations and operation behaviors of corresponding user groups during the test process to obtain test data, wherein the test data includes the operation behaviors of the test users and the performance data of the SMS channel and the SMS template; The fifth processing module is used to construct a multi-objective optimization model based on the performance of the SMS channel, the common tags of the user group and the test data, and adopt a heuristic optimization algorithm to obtain a matching solution of the SMS template, the user group and the SMS channel.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, comprising a computer program stored in the computer-readable storage medium, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images