Performance analysis method and device for test object, equipment and medium

Abstract

The embodiment of the invention provides a performance analysis method and device for a test object, equipment and a medium. The scheme comprises the steps of obtaining access data generated when a user accesses a service access end; based on the access data, judging whether the user is a user of a specific type or not; if the user is not the user of the specific type, the user is allocated to a second experiment platform, and the second experiment platform is a platform for carrying out experiments on multiple types of users based on services provided by the test object; the second experiment platform analyzes the performance of the test object based on service result data generated after the service is provided for the various types of users for the test object; if the user is the user of the specific type, the user is allocated to a first experiment platform, and the first experiment platform is a platform for carrying out an experiment on the user of the specific type based on the service provided by the test object; and the first experimental platform analyzes the performance of the test object based on service result data generated after the service is provided for the specific type of user for the test object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, and in particular to a performance analysis method and device for a test object, equipment and a medium. BACKGROUND

[0002] In actual online application scenarios, each vertical scene usually runs an online experiment (i.e., a vertical experiment) for all users. With the continuous development of technology, the demand for platform-level experiments gradually increases, especially experiments for specific types of people (such as high net worth users). Platform-level experiments can be verified simultaneously across multiple scenes, for example, verifying the effect of a new generation of models in multiple vertical scenes.

[0003] In the prior art, when selecting experiment users from all users for a vertical experiment to carry out a platform experiment, the selected experiment users usually include multiple types of users. Since the feedback results of different types of users are involved in the experiment, the feedback results of other types of users may interfere with the performance of the test object in the target user group, and cannot truly reflect the performance of the test object in the target user group, thereby resulting in inaccurate performance analysis results of the test object in the target user group.

[0004] Therefore, how to improve the accuracy of performance analysis of the test object has become a technical problem to be solved. SUMMARY

[0005] The embodiments of the present specification provide a performance analysis method, device, equipment and medium for a test object to improve the accuracy of performance analysis of the test object.

[0006] To solve the above technical problems, the embodiments of the present specification are implemented as follows.

[0007] In a first aspect, the embodiments of the present specification provide a performance analysis method for a test object. The method is applied to an experiment platform, the experiment platform includes a first experiment platform, a second experiment platform, and a first user shunting end. The input end of the first user shunting end is connected with a business access end, and the output end of the first user shunting end is connected with the first experiment platform and the second experiment platform. The method includes:

[0008] The first user shunting end acquires access data generated by a user accessing the business access end;

[0009] Based on the access data, it is determined whether the user is a specific type of user, and a first determination result is obtained;

[0010] If the first judgment result indicates that the user is not a user of a specific type, the user is assigned to the second experimental platform, the second experimental platform being a platform for conducting experiments on all users of multiple types; the second experimental platform uses the test object to provide services for users, and analyzes the performance of the test object for all users based on service result data generated after the test object provides services for users;

[0011] If the first judgment result indicates that the user is a user of a specific type, the user is assigned to the first experimental platform, the first experimental platform being a platform for conducting experiments on users of the specific type; the specific type being any one of the multiple types; the first experimental platform uses the test object to provide services for users, and analyzes the performance of the test object for users of the specific type based on service result data generated after the test object provides services for users.

[0012] In a second aspect, an embodiment of the present specification provides a performance analysis device of a test object, the device being applied to an experimental platform, the device comprising a first experimental module, a second experimental module, and a first user shunting module, an input end of the first user shunting module being connected with a service access end, an output end of the first user shunting module being connected with the first experimental module and the second experimental module, the first user shunting module comprising:

[0013] an acquisition unit, configured to acquire access data generated by a user accessing the service access end;

[0014] a judgment unit, configured to judge whether the user is a user of a specific type based on the access data, to obtain a first judgment result;

[0015] a first assignment unit, configured to, if the first judgment result indicates that the user is not a user of a specific type, assign the user to the second experimental module, the second experimental module being a module for conducting experiments on all users of multiple types; the second experimental module uses the test object to provide services for users, and analyzes the performance of the test object for all users based on service result data generated after the test object provides services for users;

[0016] a second assignment unit, configured to, if the first judgment result indicates that the user is a user of a specific type, assign the user to the first experimental module, the first experimental module being a module for conducting experiments on users of the specific type; the specific type being any one of the multiple types; the first experimental module uses the test object to provide services for users, and analyzes the performance of the test object for users of the specific type based on service result data generated after the test object provides services for users.

[0017] In a third aspect, an apparatus for computing is provided, and the apparatus comprises:

[0018] at least one processor; and

[0019] a memory connected with the at least one processor; wherein

[0020] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the performance analysis method of the test object.

[0021] In a fourth aspect, a computer readable medium is provided, and the computer readable medium stores computer readable instructions executable by a processor to implement the performance analysis method of the test object.

[0022] At least one embodiment of the present specification can achieve the following beneficial effects:

[0023] The first user shunting end can allocate the specific type of users accessing the service access end to the first experimental platform, and allocate the users not belonging to the specific type to the second experimental platform. The service result data in the experiment corresponding to the first experimental platform only involves the feedback of the specific type of users, so as to avoid the interference of the service result data of other types of users on the performance of the test object in the specific type of user group, so as to truly reflect the performance of the test object in the specific type of user group, and improve the accuracy of performance analysis of the test object in the specific type of user group.

[0024] On the other hand, the first user shunting end can also allocate a plurality of types of full users to the second experimental platform, so that the second experimental platform can evaluate the performance of the test object for a plurality of types of full users based on the service result data of a plurality of types of users, so as to improve the comprehensiveness of the evaluation of the test object. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0026] Figure 1 is an application scenario diagram of a performance analysis method of a test object provided by the present specification;

[0027] Figure 2 is a flowchart of a performance analysis method of a test object provided by an embodiment of the present specification;

[0028] Figure 3 is a schematic diagram of the framework structure of an experimental platform provided by an embodiment of the present specification;

[0029] Figure 4 is a schematic diagram of the framework structure of another experimental platform provided by an embodiment of the present specification;

[0030] Figure 5 is a flowchart of a user shunting method provided by an embodiment of the present specification;

[0031] Figure 6 is a schematic diagram of a performance analysis device of a test object corresponding to Figure 2 provided by an embodiment of the present specification;

[0032] Figure 7 is a schematic diagram of a performance analysis device of a test object provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of one or more embodiments of the present specification clearer, the technical solutions of one or more embodiments of the present specification will be described clearly and completely below in conjunction with specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present specification, not all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of one or more embodiments of the present specification.

[0034] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards in relevant regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0035] In order to facilitate the understanding of the present specification, at least part of the terms of the present specification are explained as follows:

[0036] User traffic portal: an access point used to receive and shunt user requests in the experiment, a control node responsible for experimental traffic allocation.

[0037] Vertical experiment: Generally refers to experiments conducted for a specific industry, field, or business direction. It focuses on specific business scenarios and conducts performance testing and verification in a specific industry or field. Such experiments are usually more professional and aim to solve problems in a specific field, optimize the performance of a certain type of product or service in a specific field.

[0038] Platform experiment refers to experiments conducted at a more extensive infrastructure or platform level. Its goal is usually to test, evaluate, and optimize platform-level functions or performance, covering a wider range of business scenarios. Platform experiments can involve different vertical scenarios to improve the overall performance of the platform and ensure the stability and effectiveness of the system or service in different business scenarios.

[0039] Currently, when conducting platform experiments for specific types of users, the experimental users selected from the full users of vertical experiments usually include multiple types of users. Since the experimental results involve feedback from other types of users in addition to the feedback from specific types of users, the feedback from other types of users may interfere with the performance of the test object in the specific type of user group, making it difficult to accurately reflect the performance of the test object in the specific type of user group, resulting in inaccurate performance analysis results of the test object in the specific type of user group.

[0040] The technical solutions provided by the embodiments of the present specification will be described in detail below with reference to the accompanying drawings. Figure 1 is an application scenario diagram of a performance analysis method of a test object provided by an embodiment of the present specification.

[0041] As Figure 1 shown, the application scenario diagram includes a terminal 101 and an experimental platform 102.

[0042] In the embodiments of the present specification, the experimental platform 102 can be a platform running in a terminal device or a platform running in a server, which is not limited. The terminal device can include but is not limited to at least one of a smartphone, a tablet computer, a notebook computer, a smart interactive device, a wearable device, a vehicle-mounted intelligent terminal, etc. The wearable device can include but is not limited to a smart bracelet, a smart watch, smart glasses, etc. The server can include but is not limited to at least one of any device, equipment, platform, device cluster, or cloud computing service center with computing and processing capabilities. The terminal 101 can be any of the terminal devices.

[0043] A communication connection is established between the terminal 101 and the experimental platform 102, and the communication connection can include, but is not limited to, a local area network connection, a wide area network connection, an Internet connection, a short distance communication connection, or other types of data network connections. The short distance communication connection includes, but is not limited to, a near field communication (NFC), a local area network, a Bluetooth, an infrared, and the like.

[0044] The terminal 101 can be deployed with a service access end for a user to access a preset service, and the experimental platform 102 can be deployed with a user shunting end for shunting a user accessing the service access end. Based on the shunting result of the user shunting end for the user, the user can be shunted to the first experimental platform or the second experimental platform. The first experimental platform and the second experimental platform can use the same test object to provide services for the users allocated to the respective platforms. The first experimental platform uses the test object to provide services for a specific type of user allocated to the first experimental platform, and can also analyze the performance of the test object based on the generated service result data. The second experimental platform uses the test object to provide services for a plurality of types of users allocated to the second experimental platform, and can also analyze the performance of the test object based on the generated service result data.

[0045] Next, a performance analysis method of a test object provided by the embodiments of the present specification will be described in conjunction with the accompanying drawings.

[0046] Figure 2 is a flowchart of a performance analysis method of a test object provided by the embodiments of the present specification. From the perspective of the program, the execution subject of the flowchart can be a program loaded in an experimental platform for analyzing the performance of a test object. From the perspective of hardware, the execution subject of the flowchart can be an experimental platform for analyzing the performance of a test object. The experimental platform can include a first experimental platform, a second experimental platform, a first user shunting end, and a second user shunting end. The first user shunting end has an input end, a first output end, and a second output end. The input end of the first user shunting end is connected to the service access end. The first output end of the first user shunting end is connected to the first experimental platform. The second output end of the first user shunting end is connected to the input end of the second user shunting end. The output end of the second user shunting end is connected to the second experimental platform.

[0047] As shown in Figure 2 , the flowchart can include the following steps:

[0048] Step 202: The first user shunting end acquires access data generated by a user accessing the service access end.

[0049] In the embodiments of the present specification, the experimental platform can be a service platform that provides experimental functions without interfering with the normal use of service by users. Users can handle related businesses through the experimental platform, and participate in experiments in the platform in the process of handling the businesses. These experiments can be imperceptible to users.

[0050] In the embodiments of the present specification, the experimental platform can also be a platform specially used for experimental data analysis. When users handle related businesses through the service platform, the result data generated by the service platform will be sent to the experimental platform, and the experimental platform will subsequently analyze the service result data.

[0051] In the embodiments of the present specification, the second experimental platform can be a platform for experiments on all types of users, and the first experimental platform can be a platform for experiments on specific types of users. The output end of the first user shunting end can include a first output port and a second output port, wherein the first output port is connected to the first experimental platform, and the second output port is connected to the second experimental platform.

[0052] In the embodiments of the present specification, the first user shunting end can be a port deployed in a server for shunting users accessing a business access end. The first user shunting end can obtain access data from the business access end in real time. For example, when a user accesses the business access end, corresponding access data will be generated. The first user shunting end can obtain the data in time after the generation of the access data, so as to perform subsequent processing and analysis.

[0053] In actual application, the business access end can include a client or a server, and the user can handle a preset business through the client. The server based on the client object provides access data of the user handling the preset business to the experimental platform, wherein the preset business can include at least one of fund business, insurance business, credit business, stock business, and treasury bond business, etc. The business access end can be an access end that can access multiple business scenarios, or the business access end can be an access end that can access one business scenario, which is not limited.

[0054] For example, a user handles a fund consulting business through a fund APP in a mobile terminal for handling fund business. The user can input fund consulting business request data in the fund APP in the mobile terminal, and the fund APP sends the fund consulting business request data to the server. In response to the fund consulting business request data, the server sends corresponding business data to the mobile terminal, so as to display the business data to the user in the fund APP. In this example, the business access end can refer to the fund APP in the mobile terminal, or the server. The access data can refer to the fund consulting business request data.

[0055] Step 204: determining whether the user is a specific type of user based on the access data, to obtain a first determination result.

[0056] In the embodiments of the present specification, the access data can include user identification information of the user, wherein the user identification information can be user ID information, user name information, etc. The user ID information can be the ID information of the user in the application after the user logs in the application, or the user ID information can also be a certificate ID for reflecting user identity information, or the user ID information can also be terminal identification information of a terminal device used by the user, etc.

[0057] In the embodiments of the present specification, the specific type of user refers to a user who meets a preset user level, wherein the user level is used to represent the awareness degree of the user to a certain product or service. The user level can include multiple dimensions, such as a cognitive level (the user has a basic understanding of the product), an interest level (the user has interest in the product and further explores), a purchase level (the user has a purchase intention and actually makes a purchase), and a loyalty level (the user has a repeated purchase behavior for the product), etc. The specific user can be a user at the purchase level, or can also be a user at the loyalty level, which can be set according to actual test requirements.

[0058] In the embodiments of the present specification, the specific type of user can also refer to a user who meets a preset user attribute. For example, if the resource quantity of the user meets a preset requirement, for example, the total asset amount is greater than or equal to a threshold, it indicates that the user is a specific type of user, for example, the user is a high net worth user, and the specific type of user can refer to a high net worth user. The high net worth user can be a user whose asset amount is greater than or equal to a threshold.

[0059] Step 206: if the first determination result indicates that the user is not a specific type of user, the user is allocated to the second experimental platform.

[0060] The second experimental platform is a platform for conducting experiments on a plurality of types of full-amount users; the second experimental platform provides services to the test object, and analyzes the performance of the test object for full-amount users based on service result data generated after the test object provides services to the user.

[0061] In the embodiments of the present specification, the first user shunting end can allocate the user to the second experimental platform through the output end of the first user shunting end. Specifically, the first user shunting end can allocate the user to the second experimental platform through the second output end in the output end of the first user shunting end.

[0062] In the embodiments of the present specification, the multiple types of full users can include various types of full users accessing the service access end. The second experimental platform can be a platform for testing the performance of different types of users after using the test object. In the second experimental platform, the test object can provide services to multiple types of users assigned to the platform, and the second experimental platform can analyze service result data after the multiple types of users use the test object to evaluate the performance of the test object for the multiple types of users. If the service result data reflects that the multiple types of users successfully perform the preset operation through the provided services, it can be considered that the performance of the test object for the multiple types of users is better; otherwise, if the service result data reflects that the multiple types of users fail to perform the preset operation, it can be considered that the performance of the test object for the multiple types of users is poor.

[0063] Step 208: If the first judgment result indicates that the user is a specific type of user, the user is assigned to the first experimental platform.

[0064] The first experimental platform is a platform for conducting experiments on the specific type of user; the specific type is any one of the multiple types; the first experimental platform uses the test object to provide services to users, and analyzes the performance of the test object for the specific type of user based on service result data generated after the test object provides services to users.

[0065] In the embodiments of the present specification, the first user distribution end can assign users to the first experimental platform through the output end of the first user distribution end. Specifically, the first user distribution end can assign users to the first experimental platform through the first output end in the output end of the first user distribution end.

[0066] In the embodiments of the present specification, the specific type can be any one of the multiple types described above, and the first experimental platform can be a platform for testing the performance of the specific type of user after using the test object. In the first experimental platform, the test object can provide services to the specific type of user assigned to the platform, and the first experimental platform can analyze service result data after the specific type of user uses the test object to evaluate the performance of the test object for the specific type of user. If the service result data reflects that the specific type of user successfully performs the preset operation through the provided services, it can be considered that the performance of the test object for the specific type of user is better; otherwise, if the service result data reflects that the specific type of user fails to perform the preset operation, it can be considered that the performance of the test object for the specific type of user is poor.

[0067] For example, assuming that the test object is a model providing fund purchase service to users, if the purchase rate of users of various types is significantly increased after the model provides fund purchase service to users of various types, it indicates that the performance of the test object for users of various types is excellent; if the purchase rate of users of various types is low, it indicates that the performance of the test object for users of various types is not ideal. If the purchase rate of users of a specific type is significantly increased after the model provides fund purchase service to users of the specific type, it indicates that the performance of the test object for users of the specific type is excellent; if the purchase rate of users of the specific type is low, it indicates that the performance of the test object for users of the specific type is not ideal.

[0068] For example, assuming that the test object is a model providing stock business consulting service to users, if the access time of users of various types to the stock consulting service is significantly prolonged after the model provides consulting service to users of various types, it indicates that the performance of the test object for users of various types is excellent; if the access time of users of various types to the stock consulting service is short, it indicates that the performance of the test object for users of various types is not ideal. If the access time of users of a specific type to the stock consulting service is significantly prolonged after the model provides consulting service to users of the specific type, it indicates that the performance of the test object for users of the specific type is excellent; if the access time of users of the specific type to the stock consulting service is short, it indicates that the performance of the test object for users of the specific type is not ideal.

[0069] It should be understood that the order of some steps in the method described in one or more embodiments of the present specification can be exchanged according to actual needs, or some steps can be omitted or deleted.

[0070] Figure 2 The method in the first aspect of the present specification can be used to assign users of a specific type accessing the service access end to a first experimental platform, and assign users not belonging to the specific type to a second experimental platform. In this way, the service result data in the experiment corresponding to the first experimental platform only involves feedback of users of the specific type, so that the interference of service result data of other types of users on the performance of the test object in the specific type of user group can be avoided, so as to truly reflect the performance of the test object in the specific type of user group, and improve the accuracy of performance analysis of the test object in the specific type of user group.

[0071] In addition, the first user shunting end can also allocate a plurality of types of full users to the second experimental platform, so that the second experimental platform can comprehensively evaluate the performance of the test object for a plurality of types of full users based on the service result data fed back by the plurality of types of users, so as to improve the comprehensiveness of the evaluation of the test object.

[0072] Based on the method, Figure 2 the embodiments of the present specification also provide some specific embodiments of the method, which are described below.

[0073] Figure 3 is a schematic diagram of the framework structure of an experimental platform architecture provided by the embodiments of the present specification.

[0074] The existing experimental platform architecture can include a first experimental platform, a second experimental platform, and a user shunting end, as shown in Figure 3 , wherein the user shunting end includes one input end and two output ends, the input end of the user shunting end is connected with the service access end, one of the output ends of the user shunting end is connected with the first experimental platform, and the other output end of the user shunting end is connected with the second experimental platform, and the user shunting end allocates users accessing the service access end to the first experimental platform or the second experimental platform. Since the user shunting end does not have the ability to screen specific types of users, the user shunting end can only allocate part of the full users to the first experimental platform, and the users allocated to the first experimental platform include a plurality of types of users, so that the experiment performed in the first experimental platform involves service result data fed back by different types of users, and other types of user feedback service result data may interfere with the performance of the test object in the specific type of user group, and cannot truly reflect the performance of the test object in the specific type of user group, thereby resulting in inaccurate performance analysis results of the test object in the specific type of user group. This inaccurate shunting method not only wastes valuable user traffic resources, but also when the experiment of the first experimental platform needs to expand the experimental traffic, due to the inability to control the specific affected user group, it is easy to cause traffic contention with the experiment of the second experimental platform that is running, affecting the stability and reliability of the experiment of the second experimental platform.

[0075] As an implementation manner, the structure of the experimental platform provided in the embodiments of the present specification can be the same as the existing experimental platform architecture, as shown in Figure 3The user shunting end in the embodiment of the present specification has a function program different from that of the user shunting end in the existing implementation platform mechanism, and has the ability to screen specific types of users. The specific types of users can be screened from a variety of types of full-amount users to participate in the experiment of the first experimental platform, so that the service result data of the feedback of the specific types of users is involved in the experiment corresponding to the first experimental platform, thereby avoiding the interference of the service result data of the feedback of other types of users on the performance of the test object in the specific types of user groups, so as to truly reflect the performance of the test object in the specific types of user groups, and improve the accuracy of performance analysis of the test object in the specific types of user groups.

[0076] Figure 4 is another schematic diagram of the framework structure of the experimental platform provided by the embodiment of the present specification.

[0077] As another implementation, as shown in Figure 4 The experimental platform provided by the embodiment of the present specification can include two user shunting ends, a first user shunting end and a second user shunting end. The first user shunting end can be connected with the second experimental platform through the second user shunting end. The connection of the first user shunting end with the second experimental platform can be direct connection, for example, the output end of the first user shunting end is directly connected with the second experimental platform, or can also mean that the second user shunting end is used to connect with the second experimental platform, that is, the first user shunting end can be connected with the second experimental platform through the second user shunting end. Specifically, the output end of the first user shunting end can be connected with the input end of the second user shunting end, and the output end of the second user shunting end is connected with the second experimental platform, so as to indirectly connect the first user shunting end with the second experimental platform.

[0078] The first user shunting end can be included in the first experimental platform, or the first user shunting end can be arranged outside the first experimental platform and can interact with the first experimental platform. The second user shunting end can be included in the second experimental platform, or the second user shunting end can be arranged outside the second experimental platform and can interact with the second experimental platform. The attribution of the first user shunting end and the second user shunting end is not limited here, as long as the preset function can be realized.

[0079] In an optional embodiment of the present specification, the second experimental platform comprises a second user shunt end, and the output end of the first user shunt end is connected to the input end of the second user shunt end. Specifically, the allocation of the user to the second experimental platform can include: the first user shunt end provides the access data to the input end of the second user shunt end through the output end of the first user shunt end; and the second user shunt end allocates the user to the second experimental platform through the output end of the second user shunt end.

[0080] In the embodiment of the present specification, the second user shunt end can be a user shunt end originally in the experimental platform, and the first user shunt end can be a user shunt end newly added on the basis of the second user shunt end.

[0081] In the embodiment of the present specification, on the basis of the above-mentioned existing experimental platform architecture, the first user shunt end is newly added to obtain an updated experimental platform architecture. The updated experimental platform architecture can include a first experimental platform, a second experimental platform, a first user shunt end, and a second user shunt end. The first user shunt end has an input end, a first output end, and a second output end. The input end of the first user shunt end is connected to the service access end. The first output end of the first user shunt end is connected to the first experimental platform. The second output end of the first user shunt end is connected to the input end of the second user shunt end. Through the second user shunt end, the second experimental platform can obtain an experimental user participating in a vertical experiment. The first user shunt end has the ability to screen users of a specific type. The first user shunt end can allocate the screened users of the specific type to the first experimental platform and allocate users of other types to the second experimental platform. In this way, only the service result data of the specific type of user feedback is involved in the experiment corresponding to the first experimental platform, so that the interference of the service result data of the other type of user feedback on the performance of the test object in the specific type of user group can be avoided. The performance of the test object in the specific type of user group can be truly reflected, and the accuracy of the performance analysis of the test object in the specific type of user group can be improved.

[0082] In the embodiment of the present specification, by adding the first user shunt end with the function of screening users of a specific type in front of the second user shunt end, the accuracy of user shunting can be improved without directly modifying the running experimental environment. The second user shunt end is not directly modified, the stability of the original experiment is not disturbed, and the running of the original experiment is not affected.

[0083] In the embodiment of the present specification, the access data includes user identification data of an access user. Whether the access user is a user of a specific type can be determined based on the user identification data.

[0084] Optionally, the access data can include user identification data, and the determining whether the user is a specific type of user based on the access data to obtain a first determination result can specifically include: obtaining target user identification data in the access data; querying whether the target user identification data is included in a user identification database to obtain a query result; the user identification database includes user identification data of each user belonging to the specific type.

[0085] Correspondingly, if the first determination result indicates that the user is not a specific type of user, the user is assigned to the second experimental platform, which can specifically include: if the query result indicates that the target user identification data is not included in the user identification database, the user is assigned to the second experimental platform.

[0086] Correspondingly, if the first determination result indicates that the user is a specific type of user, the user is assigned to the first experimental platform, which can specifically include: if the query result indicates that the target user identification data is included in the user identification database, the user is assigned to the first experimental platform.

[0087] In the embodiments of the present specification, the user identification data can include at least one of user ID information and user name information. For the obtained access data, the user identification data of the corresponding access user can be parsed from the access data, and it is queried whether the parsed user identification data exists in a pre-established user identification database, wherein each user identification data in the user identification database corresponds to a user of a specific type. Specifically, the way of querying whether the parsed user identification data exists in the user identification database can include: matching the parsed user identification data (user identification data a) with each user identification data in the user identification database respectively, if the user identification data a matches one of the user identification data in the user identification database successfully, it indicates that the user identification data a is included in the user identification database; if the user identification data a cannot match each user identification data in the user identification database successfully, it indicates that the user identification data a is not included in the user identification database.

[0088] In the embodiments of the present specification, if the user identification data a is not included in the user identification database, it indicates that the user corresponding to the user identification data a does not belong to the specific type of user, and the user is assigned to the second experimental platform; if the user identification data a is included in the user identification database, it indicates that the user corresponding to the user identification data a belongs to the specific type of user, and the user is assigned to the first experimental platform.

[0089] Optionally, the access data can include user identification data, and the determining whether the user is a specific type of user based on the access data can include determining whether the user is a specific type of user based on the user identification data.

[0090] In the embodiments of the present specification, the user identification data can be user classification identification, such as high net worth user identification and low net worth user identification. The access data can carry the user classification identification. Whether the user is a specific type of user can be determined by determining whether the user classification identification is an identification corresponding to a specific type of user. For example, the high net worth user identification is represented by "H", and the low net worth user identification is represented by "L". If "H" is carried in the access data, it indicates that the user is a high net worth user. If "L" is carried in the access data, it indicates that the user is a low net worth user. The high net worth user can be a user whose asset amount is greater than or equal to a preset threshold, and the low net worth user can be a user whose asset amount is less than the preset threshold.

[0091] In the embodiments of the present specification, a specific embodiment is also proposed for the method of constructing the user identification database.

[0092] Optionally, the business access end is an access end for providing business services in a client loaded in a user terminal. Before the first user shunting end obtains the access data generated by the user accessing the business access end, the method can further include: determining resource information of each user using the client according to client information of the client; selecting target resource information from each resource information, wherein the target resource information corresponds to a resource quantity greater than or equal to a second threshold; and storing user identification data of the user using the target resource information into the user identification database.

[0093] In the embodiments of the present specification, the user identification database can be created by the experimental platform described above, or can be created by other service platforms other than the experimental platform. The present specification does not limit the creator of the user identification database.

[0094] In the embodiments of the present specification, the resource information can include asset information, such as fund information. The resource quantity can include asset quantity, such as asset amount. The second threshold can be set according to actual conditions. For example, if there is an inflation risk, the second threshold can be set relatively large, such as 1 million yuan. If there is a deflation risk, the second threshold can be set relatively small, such as 300,000 yuan.

[0095] In the embodiments of the present specification, the resource information can further include historical purchase information of the user in the business access end corresponding to the business scene to purchase goods or services, and the resource quantity can refer to the purchase quantity. The identification data of the user with the purchase quantity greater than or equal to a second threshold value is stored to the user identification database. For example, the second threshold value can be 3 or other numerical values, which are not limited.

[0096] In the embodiments of the present specification, in order to avoid the influence on the experiment in the second experiment platform when a large number of access users are allocated to the first experiment platform, the number of users allocated to the first experiment platform needs to be reasonably controlled.

[0097] Optionally, before the user is allocated to the first experiment platform, the method can further include: determining whether the number of users allocated to the first experiment platform is less than or equal to a first threshold value, to obtain a second determination result.

[0098] Correspondingly, the user is allocated to the first experiment platform, and the method can specifically include: if the second determination result indicates that the number of users allocated to the first experiment platform is less than or equal to the first threshold value, the user is allocated to the first experiment platform.

[0099] In the embodiments of the present specification, the user allocated to the first experiment platform can refer to the user allocated to the first experiment platform to participate in the test of the corresponding test object within one experiment period. One experiment period can refer to within 1 second, within 1 minute, or within another time point or time period. The first threshold value can be set according to the actual situation. If the number of users currently accessing the business access end is relatively large, the first threshold value can be set to be relatively large. If the number of users currently accessing the business access end is relatively small, the first threshold value can be set to be relatively small.

[0100] In actual application, determining whether the number of users allocated to the first experiment platform is less than or equal to the first threshold value can also be understood as determining whether the user diversion ratio of the user allocated to the first experiment platform is less than or equal to a third threshold value. The user diversion ratio corresponding to the first experiment platform can refer to the ratio between the number of users allocated to the first experiment platform within one experiment period and the number of all users accessing the business access end within the experiment period. Alternatively, the user diversion ratio corresponding to the first experiment platform can refer to the ratio between the number of users allocated to the first experiment platform within one experiment period and the number of specific types of users among all users accessing the business access end within the experiment period.

[0101] In the embodiments of the present specification, if the number of users allocated to the first experimental platform is less than or equal to the first threshold value, the experimental users continue to be allocated to the first experimental platform, i.e., the corresponding users are allocated to the first experimental platform; if the number of users allocated to the first experimental platform is greater than the first threshold value, the allocation of experimental users to the first experimental platform is ended, i.e., the corresponding users are allocated to the second experimental platform.

[0102] For example, assuming that a specific type of user is a high net worth user, the first user shunting end determines that user a is a high net worth user, and when user a is allocated to the first experimental platform, it is further determined whether the number of high net worth users allocated to the first experimental platform is greater than or equal to the first threshold value. For example, the first threshold value is 10,000, if the number of high net worth users allocated to the first experimental platform is 10,000, user a will not be allocated to the first experimental platform; if the number of high net worth users allocated to the first experimental platform is less than 10,000, user a will be allocated to the first experimental platform.

[0103] In the embodiments of the present specification, when allocating users participating in the experiment to the first experimental platform, the number of users participating in the experiment allocated to the first experimental platform or the user shunting ratio is reasonably controlled, so that the influence on the experiment in the second experimental platform can be reduced to achieve low invasiveness to the second experimental platform.

[0104] In the embodiments of the present specification, for the entire experimental stage of the experiment corresponding to the first experimental platform, the maximum number of users allocated to the first experimental platform, i.e., the first threshold value, can gradually increase as the experiment progresses. For example, in the initial stage of the experiment, the first threshold value can be the smallest, such as 1%, in the middle stage of the experiment, the first threshold value can be moderate, such as 20%, and in the final stage of the experiment, the first threshold value can be the largest, such as 40%. In addition, when setting the first threshold value, the test result data of the test object in the experiment process also needs to be considered, and the first threshold value is updated based on the test result data.

[0105] Optionally, before the judging whether the number of users allocated to the first experimental platform is less than or equal to the first threshold value, the method further comprises: obtaining historical analysis result information, the historical analysis result information being result information of performance analysis on the test object by using historical service result data; the historical service result being service result data of a historical user who enjoys the service provided by the test object in the first experimental platform; if the historical analysis result information meets a preset requirement, increasing a preset value on the basis of a second threshold value to obtain the first threshold value; the second threshold value being a threshold value used when the historical user is allocated to the first experimental platform; if the number of users allocated to the first experimental platform is less than or equal to the second threshold value before the historical user is allocated to the first experimental platform, the historical user is allocated to the first experimental platform.

[0106] In the embodiments of the present specification, the experiment corresponding to the first experimental platform can include a first experimental stage and a second experimental stage, wherein the experimental time corresponding to the first experimental stage is different from the experimental time corresponding to the second experimental stage, the user participating in the experiment in the first experimental stage is different from the user participating in the experiment in the second experimental stage, and the test object tested in the first experimental stage is the same as the test object tested in the second experimental stage.

[0107] In the embodiments of the present specification, the user in claim 1 can refer to the user participating in the experiment in the second experimental stage, the historical user can refer to the user participating in the experiment in the first experimental stage, the experimental time of the first experimental stage is earlier than the experimental time of the second experimental stage. The historical service result data can refer to the result data exhibited by the historical experimental user based on the service provided by the test object.

[0108] In the embodiments of the present specification, the historical analysis result information meeting the preset requirement can refer to the performance of the test object being excellent for the historical user, for example, the purchase rate of the user purchasing a preset commodity or service being greater than or equal to a preset threshold value, or the user paying attention to a preset business for a time greater than or equal to a preset threshold value. In actual application, if the performance of the test object for the historical user is tested to meet the preset requirement in the first experimental stage, that is, more users can be allowed to participate in the experiment, the first threshold value of the second experimental stage can be set to be greater than the second threshold value of the first experimental stage, that is, the first threshold value is obtained by increasing the preset value on the basis of the second threshold value. If the performance of the test object is tested to not meet the preset requirement in the first experimental stage, that is, more users are avoided to participate in the experiment, the first threshold value of the second experimental stage can be set to be less than the second threshold value of the first experimental stage, that is, the first threshold value is obtained by reducing the preset value on the basis of the second threshold value.

[0109] In the embodiments of the present specification, according to the feedback data of the users in the early experimental stage, the number of users participating in the later experiment can be reasonably adjusted. By analyzing the experimental data in the early stage, if it is found that the performance of the test object is poor, the number of users participating in the test object can be appropriately reduced, so that more users can be avoided to experience the low-performance test object, thereby reducing the customer loss rate of the business party.

[0110] In the embodiments of the present specification, after the user is assigned to the first experimental platform, if the first experimental platform does not carry out a scene experiment consistent with the business accessed by the user, the user can also be rolled back to the second experimental platform.

[0111] Optionally, the access data can include scene identification information, the first experimental platform can be a platform for testing a preset business scene, and the method can further include: obtaining the scene identification information in the access data; determining whether the business scene corresponding to the scene identification information belongs to the preset business scene to obtain a third determination result; and if the third determination result indicates that the business scene does not belong to the preset business scene, assigning the user to the second experimental platform.

[0112] In the embodiments of the present specification, after determining that the user is a specific type of user, the scene identification information in the access data can be obtained before the user is assigned to the first experimental platform, and it is determined whether the business scene corresponding to the scene identification information belongs to the above-mentioned preset business scene. Alternatively, the scene identification information in the access data can also be obtained after the user is assigned to the first experimental platform, and it is determined whether the business scene corresponding to the scene identification information belongs to the above-mentioned preset business scene. This is not limited.

[0113] In the embodiments of the present specification, the access data can also include scene identification information, the scene identification information is used to represent the identification information of the business scene corresponding to the business accessed by the user, and the scene identification information corresponding to different business scenes is different, for example, the fund business scene corresponds to fund business scene identification information, the insurance business scene corresponds to insurance business scene identification information, the stock business scene corresponds to stock business scene identification information, the national debt business scene corresponds to national debt business scene identification information, and the credit business scene corresponds to credit business scene identification information. Wherein, the identification information of the business scene can include scene ID, scene name, business label and the like.

[0114] In the embodiments of the present specification, the first experimental platform can be an experimental platform for testing the test object under a preset business scene, wherein the preset business scene can include one business scene, or can also include multiple business scenes.

[0115] In the embodiments of the present specification, after the user is assigned to the first experimental platform, it is further determined whether the service scenario corresponding to the scenario identification information carried in the access data of the user belongs to a preset service scenario corresponding to an experiment performed by the first experimental platform. If the service scenario belongs to the preset service scenario, the user is assigned to the experiment corresponding to the preset service scenario; if the service scenario does not belong to the preset service scenario, the user is re-assigned to the second experimental platform, so that the user completes the service based on the service provided by the second experimental platform, and the user experience is improved.

[0116] In the embodiments of the present specification, if the first experimental platform does not carry out a scenario experiment consistent with the service accessed by the user, the user assigned to the first experimental platform can be rolled back to the second experimental platform, avoiding waste of user resources, improving utilization of user data, and further reducing the impact on the experiment in the second experimental platform.

[0117] Optionally, the preset service scenario includes a plurality of service scenarios, and the first experimental platform includes a plurality of sub-experimental platforms, each of the sub-experimental platforms corresponding to one of the plurality of service scenarios. After determining whether the service scenario corresponding to the scenario identification information belongs to the preset service scenario, the method can further include: if the third determination result indicates that the service scenario belongs to the preset service scenario, determining a target sub-experimental platform corresponding to the service scenario; and assigning the user to the target sub-experimental platform.

[0118] In the embodiments of the present specification, the first experimental platform can be an experimental platform that simultaneously tests a test object under a plurality of service scenarios. For example, the test object is a preset model, which can be applied to a fund service scenario to assist a user in handling fund business; applied to an insurance service scenario to assist a user in handling insurance business; applied to a stock service scenario to assist a user in handling stock business; and applied to a government bond service scenario to assist a user in handling government bond business. The first experimental platform can be an experimental platform that simultaneously tests the preset model under the fund service scenario, the insurance service scenario, the stock service scenario, and the government bond service scenario.

[0119] In actual application, the first experiment platform can be provided with a sub-experiment platform corresponding to each business scenario, for example, the first experiment platform is provided with a sub-experiment platform for testing the preset model in the fund business scenario for the fund business scenario; the first experiment platform is provided with a sub-experiment platform for testing the preset model in the insurance business scenario for the insurance business scenario; the first experiment platform is provided with a sub-experiment platform for testing the preset model in the stock business scenario for the stock business scenario; the first experiment platform is provided with a sub-experiment platform for testing the preset model in the national debt business scenario for the national debt business scenario. The business scenario and the sub-experiment platform are one-to-one correspondence, that is, one business scenario corresponds to one sub-experiment platform, and one sub-experiment platform corresponds to one business scenario.

[0120] In the embodiments of the present specification, if the business scenario corresponding to the scenario identification information in the access data belongs to the preset business scenario corresponding to the first experiment platform, the target sub-experiment platform corresponding to the business scenario is determined, and the user is allocated to the target sub-experiment platform.

[0121] In the embodiments of the present specification, the experiment corresponding to the first experiment platform can include a control group experiment and a test group experiment, and each sub-experiment in the first experiment platform can also include a control group sub-experiment and a test group sub-experiment. For the same user, the same type of experiment group needs to be allocated in the first experiment platform and each sub-experiment platform.

[0122] Optionally, the first experiment platform can include a control group and a test group, and the target sub-experiment platform can include a sub-control group and a sub-test group. The user can be allocated to the target sub-experiment platform, specifically including: if the user is allocated to the control group in the first experiment platform, the user is allocated to the sub-control group in the target sub-experiment platform; if the user is allocated to the test group in the first experiment platform, the user is allocated to the sub-test group in the target sub-experiment platform.

[0123] In the embodiments of the present specification, the control group in the first experiment platform can be a control group experiment platform for performing a control group experiment, and the test group can be a test group experiment platform for performing a test group experiment. The sub-control group in the sub-experiment platform can be a control group sub-experiment platform for performing a control group sub-experiment, and the sub-test group can be a test group sub-experiment platform for performing a test group sub-experiment.

[0124] In the embodiments of the present specification, in the first experiment platform, the control group experiment can be an experiment for providing services to a benchmark object, and the test group experiment can be an experiment for providing services to a test object. In the sub-experiment platform, the control group sub-experiment can be an experiment for providing services to a benchmark object, and the test group sub-experiment can be an experiment for providing services to a test object.

[0125] In the embodiments of the present specification, for the same user, if the user is assigned to the control group in the first experimental platform, the user will also be assigned to the sub-control group in the target sub-experimental platform; if the user is assigned to the test group in the first experimental platform, the user will also be assigned to the sub-test group in the target sub-experimental platform.

[0126] In actual application, for different sub-experimental platforms, such as a first sub-experimental platform and a second sub-experimental platform. The same user accessed the scene service corresponding to the first sub-experimental platform at a first time, and the user was assigned to the sub-test group of the first sub-experimental platform, if the user accessed the scene service corresponding to the second sub-experimental platform at a second time, and the user was assigned to the second sub-experimental platform, the user will also be assigned to the sub-test group of the second sub-experimental platform.

[0127] In the embodiments of the present specification, for the same user, the user will be assigned to the same type of experimental group in different sub-experimental platforms of the first experimental platform, that is, either all be assigned to the control group or all be assigned to the test group, so as to ensure that the same user has consistent experimental experience in different business scene sub-experiments, and improve the user experience.

[0128] In the embodiments of the present specification, a specific embodiment is also proposed for the grouping method of assigning the user to the control group or the test group.

[0129] Optionally, the access data can include user ID information, the first experimental platform can include a control group and a test group, and the assigning the user to the first experimental platform can specifically include: obtaining the user ID information in the access data; performing hash processing on the user ID information by using a hash algorithm to obtain a hash value; if the hash value is a first type of data, assigning the user to the control group; and if the hash value is a second type of data, assigning the user to the test group.

[0130] In the embodiments of the present specification, the control group can be an experimental group that provides services for a reference object, and the test group can be an experimental group that provides services for a test object.

[0131] In the embodiments of the present specification, the user ID in the access data is processed by using a hash algorithm to obtain a hash value corresponding to the user ID.

[0132] In actual application, the hash value is divided into the first type data and the second type data based on the parity of the hash value. For example, if the determined hash value is an odd number, the hash value is determined as the first type data; if the determined hash value is an even number, the hash value is determined as the second type data. Alternatively, the determined hash value is divided by 2, and the hash value is divided into the first type data and the second type data based on the remainder after the division. For example, if the remainder is 0, the hash value is determined as the first type data; if the remainder is 1, the hash value is determined as the second type data.

[0133] In the embodiments of the present specification, the correspondence between the control group and the first data type, and the correspondence between the test group and the second data type are established in advance. If the hash value is the first data type, the user corresponding to the user ID is allocated to the control group; if the hash value is the second data type, the user corresponding to the user ID is allocated to the test group. Alternatively, the correspondence between the control group and the second data type, and the correspondence between the test group and the first data type are established in advance. If the hash value is the first data type, the user corresponding to the user ID is allocated to the test group; if the hash value is the second data type, the user corresponding to the user ID is allocated to the control group.

[0134] In the embodiments of the present specification, for the same user, when the user is allocated to the control group or the test group in the first experimental platform, and the user is allocated to the sub-control group or the sub-test group in each sub-experimental platform, the hash value corresponding to the user ID of the user can be calculated by using the same hash algorithm based on the user ID of the user, and the grouping can be performed based on the same allocation principle, so that the user can have consistent experimental experience in different sub-experimental platforms, and the user experience is improved.

[0135] In the embodiments of the present specification, for any sub-experimental platform in the first experimental platform, the principle of allocating the number of user or the user shunting ratio of the experimental user to the sub-experimental platform can be that the number of user or the user shunting ratio of the experimental user gradually increases from an initial value as the experiment continues.

[0136] Optionally, the first experimental platform can include a plurality of sub-experimental platforms, and any sub-experimental platform is configured with a sub-user shunting end for setting a user shunting ratio of participating in the any sub-experimental platform. The method can further include:

[0137] If a target sub-experimental platform is added for the first experimental platform, the target sub-user shunting end of the target sub-experimental platform is used to set that the experimental user is allocated to the target sub-experimental platform in a manner of gradually increasing from an initial user shunting ratio.

[0138] In the embodiments of the present specification, the first experiment platform can include a plurality of sub-experiment platforms, each of which corresponds to a service scenario. Each sub-experiment platform is configured with a sub-user shunting end for setting a user shunting ratio of a user participating in an experiment of the sub-experiment platform. The sub-user shunting end is used to allocate the user participating in the first experiment platform to the corresponding sub-experiment platform. The user shunting ratio of the sub-experiment platform can refer to the ratio between the number of users allocated to the corresponding sub-experiment platform and the number of users of a specific type among the access users in the scenario service corresponding to the sub-experiment platform in an experiment period, or the ratio between the number of users allocated to the corresponding sub-experiment platform and the number of access users in the scenario service corresponding to the sub-experiment platform in an experiment period, or the ratio between the number of users allocated to the corresponding sub-experiment platform and the number of users of a specific type among the access users in an experiment period, or the ratio between the number of users allocated to the corresponding sub-experiment platform and the number of access users in an experiment period.

[0139] In the embodiments of the present specification, when a target sub-experiment platform under a preset scenario is added to the first experiment platform, the user shunting ratio of the target sub-experiment platform can be set to gradually increase from an initial user shunting ratio through the target sub-user shunting end of the target sub-experiment platform. The initial user shunting ratio can be set according to actual needs, for example, the initial user shunting ratio can be 1%, 2%, 5%, etc., which is not limited.

[0140] For example, the current first experiment platform includes sub-experiment platform a and sub-experiment platform b, the user diversion ratio corresponding to the first experiment platform is 20%, and the user diversion ratios corresponding to sub-experiment platform a and sub-experiment platform b are both 20%. The user diversion ratio corresponding to the first experiment platform is the ratio between the number of users allocated to the first experiment platform in the experiment interval from the start of the experiment to the current experiment stage and the number of users of a specific type among the users accessing the business scenarios corresponding to sub-experiment platform a and sub-experiment platform b in the experiment interval. The user diversion ratio corresponding to sub-experiment platform a is the ratio between the number of users allocated to sub-experiment platform a in the time interval and the number of users of a specific type among the users accessing the business scenarios corresponding to sub-experiment platform a in the experiment interval. The user diversion ratio corresponding to sub-experiment platform b is the ratio between the number of users allocated to sub-experiment platform b in the time interval and the number of users of a specific type among the users accessing the business scenarios corresponding to sub-experiment platform b in the experiment interval. If sub-experiment platform c under a preset scenario is added in the first experiment platform to carry out an experiment under the preset scenario, the user diversion ratio corresponding to sub-experiment platform c can gradually increase from the initial user diversion ratio, for example, gradually increase from 1%, without needing to remain consistent with the user diversion ratio corresponding to the current first experiment platform, which is 20%, at the start of the experiment. It should be noted that if the user diversion ratio set for the new business scenario in the first experiment platform is also 20%, and the user diversion ratio of the sub-experiment platform corresponding to the new business scenario from the first experiment platform is 1%, the user corresponding to the difference user diversion ratio of 19% can be rolled back to the second experiment platform.

[0141] In the embodiments of the present specification, for each sub-experiment platform included in the first experiment platform, the sub-experiment platform can autonomously control the user diversion ratio of allocating experiment users to the sub-experiment platform based on the configured sub-user diversion end. Especially for the experiment corresponding to the newly added sub-experiment platform under a new business scenario, the experiment users can be allocated from a smaller initial user diversion ratio at the start of the experiment, so that the adverse consequences brought by recklessly allocating a large number of experiment users to the experiment under the new business scenario when the performance of the test object under the new business scenario is poor can be avoided.

[0142] In the embodiments of the present specification, the user diversion ratios corresponding to the sub-experiment platforms included in the first experiment platform can gradually increase by a certain step, and the expansion steps of the user diversion ratios corresponding to the sub-experiment platforms under different business scenarios can be different.

[0143] Optionally, the first experiment platform comprises a first sub-experiment platform and a second sub-experiment platform, and the method further comprises: if the user shunting proportion of users participating in experiments is expanded in a gradually increasing manner for both a first experiment under the first sub-experiment platform and a second experiment under the second sub-experiment platform, obtaining a first access amount of accessing a first business scenario corresponding to the first sub-experiment platform, and obtaining a second access amount of accessing a second business scenario corresponding to the second sub-experiment platform; and if the first access amount is greater than the second access amount, setting an expansion step of expanding the user shunting proportion of users participating in experiments for the first experiment to be less than an expansion step of expanding the user shunting proportion of users participating in experiments for the second experiment.

[0144] In the embodiments of the present specification, the explanation of the sub-experiment platform and the user shunting proportion corresponding to the sub-experiment platform can refer to the above content, which will not be repeated here.

[0145] In the embodiments of the present specification, the first experiment platform can comprise a first sub-experiment platform and a second sub-experiment platform, and for both a first experiment under the first sub-experiment platform and a second experiment under the second sub-experiment platform, the user shunting proportion of users participating in experiments can be expanded in a gradually increasing manner, wherein the gradually increasing manner can mean that the user shunting proportion of users starts from an initial user shunting proportion and gradually increases by a certain step.

[0146] In the embodiments of the present specification, the first access amount of the first business scenario can mean the number of users of a specific type among access users accessing the first business scenario within a preset time interval, i.e., a time period, or at a time, i.e., a time point, or the number of users of access users accessing the first business scenario. The explanation of the second access amount of the second business scenario can refer to the explanation of the first access amount of the first business scenario, which will not be repeated here. The first access amount and the second access amount are access amounts determined for the same time period or the same time.

[0147] In the embodiments of the present specification, if the first access amount is greater than the second access amount, the expansion step of expanding the user shunting proportion of users participating in experiments for the first experiment is less than the expansion step of expanding the user shunting proportion of users participating in experiments for the second experiment; and if the first access amount is less than the second access amount, the expansion step of expanding the user shunting proportion of users participating in experiments for the first experiment is greater than the expansion step of expanding the user shunting proportion of users participating in experiments for the second experiment. The expansion step can mean the expansion amount of expanding the user shunting proportion once, for example, the current user shunting proportion is 10%, the expansion step can be 2%, and after one expansion, the user shunting proportion can be expanded from 10% to 12%.

[0148] In the embodiments of the present specification, for a service scenario with a large access volume, a smaller user shunting expansion step can be set to gradually increase the number of experimental users, reasonably control the experimental scale, and avoid poor performance of test objects leading to poor user experience and affecting the reputation of the service scenario. For a service scenario with a small access volume, a larger expansion step can be set to ensure that the number of users participating in the experiment meets the experimental requirements, thereby improving the effectiveness and reliability of the experimental results.

[0149] Optionally, the first experimental platform is configured to perform a platform experiment on the test object, and the second experimental platform is configured to perform a vertical experiment on the test object. The platform experiment is configured to test the performance of the test object in multiple scenarios, and the vertical experiment is configured to test the performance of the test object in a single scenario. The test object includes an algorithm or a model for providing a service.

[0150] In the embodiments of the present specification, the test object can include an algorithm and / or a model for providing a service, such as a large language model that can be applied to a financial service scenario to assist in handling financial services. In the fund service scenario, the insurance service scenario, the stock service scenario, and the government bond service scenario, vertical experiments can be performed on the large language model for all users in each service scenario. In order to verify the performance of the large language model for some specific types of users, such as high net worth users, platform experiments can be performed on the large language model for specific types of users in each service scenario in the fund service scenario, the insurance service scenario, the stock service scenario, and the government bond service scenario.

[0151] In this embodiment, the first user routing endpoint can be the global traffic entry point for platform experiments. All users accessing the business access endpoint can be first sent to the global traffic entry point for platform experiments. Through the global traffic entry point for platform experiments, users of specific types who meet the requirements can be assigned to platform experiments, and the remaining users are assigned to vertical category experiments. Multiple sub-platform experiments under different business scenarios can be conducted simultaneously within the platform experiment. Each sub-platform experiment can have a scenario platform experiment traffic entry point, i.e., the sub-user routing endpoint in the above embodiment. Through the scenario platform experiment traffic entry point, users of specific types within the business scenario corresponding to that scenario platform experiment traffic entry point can be assigned to the corresponding sub-platform experiment. Through the global traffic entry point for platform experiments, the overall performance of the test object in various business scenarios can be directly obtained, and the individual performance data for each business scenario can be further analyzed. Based on these individual performance data, it can be determined in which business scenarios the test object is suitable for use and in which business scenarios it is unsuitable. Furthermore, through the traffic entry point for scenario platform experiments, the performance of the test object in a specific business scenario can be evaluated individually. In other words, the global traffic entry point of the platform experiment can aggregate the overall performance of the test object as well as its individual performance in various business scenarios, so that business service providers can easily view the various performance aspects of the test object.

[0152] It should be noted that users assigned to vertical experiments can also be assigned to various vertical experiments of an object through the traffic entry points of each vertical experiment. The specific allocation method can be carried out in accordance with existing technologies, which will not be elaborated here.

[0153] Figure 5 This is a flowchart illustrating a user traffic splitting method provided in the embodiments of this specification.

[0154] like Figure 5 As shown, all users of the business access terminal will be sent to the platform experiment global traffic entry point. The platform experiment global traffic entry point can assign specific types of users who meet the requirements from the full business user group to the platform experiment, and the remaining users will be assigned to the vertical category experiment. The business access terminal can be an access terminal that can access multiple business scenarios, or it can be an access terminal that can access one business scenario.

[0155] The platform experiments include Platform Experiment 1 and Platform Experiment 2. Platform Experiment 1 is an experiment that tests Test Object 1 (e.g., Model 1), and Platform Experiment 2 is an experiment that tests Test Object 2 (e.g., Model 2). Platform Experiment 1 and Platform Experiment 2 share the users assigned to the platform experiments. For example, if 10 users are assigned to the platform experiments, then 5 users can be assigned to Platform Experiment 1, and the remaining 5 users can be assigned to Platform Experiment 2.

[0156] Taking Platform Experiment 1 as an example, the first experimental platform can be the platform used to conduct experiments for Platform Experiment 1. Users assigned to Platform Experiment 1 will be randomly assigned to either the control group or the test group within Platform Experiment 1. Platform Experiment 1 can include several sub-platform experiments under different business scenarios, such as Sub-platform Experiment 1 under the fund business scenario, Sub-platform Experiment 2 under the insurance business scenario, and Sub-platform Experiment 3 under the credit business scenario. A scenario 1 platform experiment traffic entry point is configured for Sub-platform Experiment 1, a scenario 2 platform experiment traffic entry point is configured for Sub-platform Experiment 2, and a scenario 3 platform experiment traffic entry point is configured for Sub-platform Experiment 3. Users from Platform Experiment 1 can be assigned to the corresponding sub-platform experiments through these scenario platform experiment traffic entry points.

[0157] If user A is assigned to the test group in platform experiment 1 at the first moment, and the business scenario accessed by user A at the first moment is the fund business scenario, then user A will also be assigned to the test group in sub-platform experiment 1 when assigned to sub-platform experiment 1. If user A accesses the insurance business scenario at the second moment, the user will still be assigned to the test group in platform experiment 1 when reassigned to platform experiment 1, and will also be assigned to the test group in sub-platform experiment 2 when assigned to sub-platform experiment 2. This ensures consistency of experience for the same user across experiments in different business scenarios.

[0158] It should be noted that the above explanation uses Platform Experiment 1 as an example. The explanation of Platform Experiment 2 can be found in the explanation of Platform Experiment 1, and will not be repeated here. However, it should be noted that if Platform Experiment 2 also includes several sub-platform experiments under different business scenarios, such as sub-platform experiment a under the fund business scenario, sub-platform experiment b under the insurance business scenario, and sub-platform experiment c under the credit business scenario, then Platform Experiment 2 and Platform Experiment 1 share the same scenario platform experiment traffic entry point under the same business scenario. For example, sub-platform experiment a under the fund business scenario of Platform Experiment 2 shares the scenario 1 platform experiment traffic entry point with sub-platform experiment 1 under the fund business scenario of Platform Experiment 1; sub-platform experiment b under the insurance business scenario of Platform Experiment 2 shares the scenario 2 platform experiment traffic entry point with sub-platform experiment 2 under the insurance business scenario of Platform Experiment 1; and sub-platform experiment c under the credit business scenario of Platform Experiment 2 shares the scenario 3 platform experiment traffic entry point with sub-platform experiment 3 under the credit business scenario of Platform Experiment 1.

[0159] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 6 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a performance analysis device for a test object. (Example) Figure 6As shown, the apparatus can include: a first user shunting module 602, a first experiment module 604, and a second experiment module 606, an input end of the first user shunting module 602 is connected with a service access end, an output end of the first user shunting module 602 is connected with the first experiment module 604 and the second experiment module 606, and the first user shunting module 602 can include:

[0160] An acquisition unit 6022 is configured to acquire access data generated by a user accessing the service access end;

[0161] A judgment unit 6024 is configured to judge whether the user is a specific type of user based on the access data, to obtain a first judgment result.

[0162] A first distribution unit 6026 is configured to distribute the user to the second experiment module if the first judgment result indicates that the user is not a specific type of user, the second experiment module being a module for performing experiments on a plurality of types of full-amount users; the second experiment module provides services for the test object as a user, and analyzes the performance of the test object for full-amount users based on service result data generated after the test object provides services for the user.

[0163] A second distribution unit 6028 is configured to distribute the user to the first experiment module if the first judgment result indicates that the user is a specific type of user, the first experiment module being a module for performing experiments on the specific type of user; the specific type being any one of the plurality of types; the first experiment module provides services for the test object as a user, and analyzes the performance of the test object for the specific type of user based on service result data generated after the test object provides services for the user.

[0164] Based on the apparatus, Figure 6 the embodiments of the present specification also provide some specific implementation solutions, which are described below.

[0165] Optionally, the second experiment module includes a second user shunting end, and an output end of the first user shunting module is connected with an input end of the second user shunting module.

[0166] Optionally, the first distribution unit 6026 can specifically include:

[0167] A providing sub-unit is configured to provide the access data to the input end of the second user shunting module through the output end of the first user shunting module.

[0168] A first distribution sub-unit is configured to distribute the user to the second experiment module through the output end of the second user shunting module.

[0169] Optionally, the access data comprises user identification data.

[0170] Optionally, the judging unit 6024 can specifically include:

[0171] The obtaining sub-unit is configured to obtain user identification data contained in the access data.

[0172] The querying sub-unit is configured to query whether the user identification data is contained in a user identification database to obtain a query result; the user identification database comprises user identification data of each user belonging to the specific type.

[0173] Optionally, the first distribution unit 6026 can specifically include:

[0174] The second distribution sub-unit is configured to, if the query result indicates that the user identification data is not contained in the user identification database, distribute the user to the second experimental module.

[0175] The second distribution unit 6028 can specifically include:

[0176] The third distribution sub-unit is configured to, if the query result indicates that the user identification data is contained in the user identification database, distribute the user to the first experimental module.

[0177] Optionally, the first user shunting module 602 can further include:

[0178] The second judging unit is configured to judge whether the number of users distributed to the first experimental module is less than or equal to a first threshold value to obtain a second judgment result.

[0179] The second distribution unit 6028 can specifically include:

[0180] The fourth distribution sub-unit is configured to, if the second judgment result indicates that the number of users distributed to the first experimental module is less than or equal to the first threshold value, distribute the user to the first experimental module.

[0181] Optionally, the first user shunting module 602 can further include:

[0182] The second obtaining unit is configured to obtain historical analysis result information, the historical analysis result information being result information of performance analysis of the test object by using historical service result data; the historical service result being service result data of a historical user who enjoys the test object to provide a service in the first experimental module.

[0183] The increasing unit is configured to increase a preset value on the basis of a second threshold value if the historical analysis result information meets preset requirements, to obtain the first threshold value; the second threshold value is a threshold value used when the historical user is allocated to the first experimental module; if the number of users allocated to the first experimental module before the historical user is allocated to the first experimental module is less than or equal to the second threshold value, the historical user is allocated to the first experimental module.

[0184] Optionally, the access data can include scene identification information, and the first experimental module is a platform for testing a preset business scene.

[0185] Optionally, the first user shunting module 602 can further include:

[0186] The third obtaining unit is configured to obtain scene identification information in the access data.

[0187] The third determining unit is configured to determine whether a business scene corresponding to the scene identification information belongs to the preset business scene, to obtain a third determination result.

[0188] The third allocation unit is configured to allocate the user to the second experimental module if the third determination result indicates that the business scene does not belong to the preset business scene.

[0189] Optionally, the preset business scene can include a plurality of business scenes, and the first experimental module can include a plurality of sub-experimental platforms, each of the sub-experimental platforms corresponding to one of the plurality of business scenes; the first user shunting module can further include:

[0190] The determining unit is configured to determine a target sub-experimental platform corresponding to the target business scene if the third determination result indicates that the business scene belongs to the preset business scene.

[0191] The fourth allocation unit is configured to allocate the user to the target sub-experimental platform.

[0192] Optionally, the first experimental module includes a control group and a test group, and the target sub-experimental platform includes a sub-control group and a sub-test group.

[0193] The fourth allocation unit can specifically include:

[0194] The fifth allocation sub-unit is configured to allocate the user to the sub-control group in the target sub-experimental platform if the user is allocated to the control group in the first experimental module.

[0195] A sixth allocation subunit is configured to allocate the user to the sub-test group in the target sub-experimental platform if the user is allocated to the test group in the first experimental module.

[0196] Optionally, the access data can include user ID information, the first experimental module includes a control group and a test group, and the second allocation unit 6028 specifically can include:

[0197] A third acquisition unit is configured to acquire user ID information in the access data.

[0198] A processing unit is configured to perform hash processing on the user ID information by using a hash algorithm to obtain a hash value.

[0199] A fifth allocation unit is configured to allocate the user to the control group if the hash value is first type data.

[0200] A sixth allocation unit is configured to allocate the user to the test group if the hash value is second type data.

[0201] Optionally, the first experimental module can include a plurality of sub-experimental platforms, any sub-experimental platform is configured with a sub-user shunt end for setting a user shunt ratio of participating in the any sub-experimental platform, and if a target sub-experimental platform is added to the first experimental platform, a target sub-user shunt end of the target sub-experimental platform is used to set a gradually increasing initial user shunt ratio for allocating experimental users to the target sub-experimental platform.

[0202] Optionally, the first experimental module can include a first sub-experimental platform and a second sub-experimental platform, and the first user shunt module 602 can further include:

[0203] A fourth acquisition unit is configured to acquire a first access amount of accessing a first business scenario corresponding to the first sub-experimental platform and a second access amount of accessing a second business scenario corresponding to the second sub-experimental platform if a user shunt ratio of users participating in experiments is gradually increased for both the first experiment under the first sub-experimental platform and the second experiment under the second sub-experimental platform.

[0204] A first setting unit is configured to set an expansion step of expanding the user shunt ratio of users participating in the first experiment to be smaller than an expansion step of expanding the user shunt ratio of users participating in the second experiment if the first access amount is greater than the second access amount.

[0205] Optionally, the first experiment module is configured to perform a platform experiment on the test object, and the second experiment module is configured to perform a vertical experiment on the test object. The platform experiment is configured to test performance of the test object in multiple scenarios, and the vertical experiment is configured to test performance of the test object in a single scenario. The test object includes an algorithm or a model configured to provide a service.

[0206] Based on the same idea, the present specification also provides a device corresponding to the above method.

[0207] Figure 7 is a structural schematic diagram of a performance analysis device of a test object provided by the present specification. As shown in Figure 7 , the device 700 can include:

[0208] at least one processor 710; and

[0209] a memory 730 in communication connection with the at least one processor.

[0210] Corresponding to the method shown in Figure 2 , the memory 730 stores instructions 720 executable by the at least one processor 710. The instructions are executed by the at least one processor 710 to enable the at least one processor 710 to perform the above-mentioned performance analysis method of the test object.

[0211] Based on the same idea, the present specification also provides a computer readable medium corresponding to the above method. The computer readable medium stores computer readable instructions executable by a processor to implement the above-mentioned performance analysis method of the test object.

[0212] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device shown in Figure 7 , since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0213] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0214] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051 F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to implementing the controller purely in computer readable program code, it is possible to implement the controller to perform the same functions in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps. Such a controller can therefore be considered to be a hardware component, and the means included within it for performing the various functions can also be considered to be structures within the hardware component. Alternatively, the means for performing the various functions can even be considered to be both a software module implementing the method and a structure within a hardware component.

[0215] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0216] For the sake of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of the units can be implemented in one or more software and / or hardware in the implementation of the present application.

[0217] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage etc.) containing computer usable program code.

[0218] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0219] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0220] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0221] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0222] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer-readable media.

[0223] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0224] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed, or other elements inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0225] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0226] The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0227] The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A performance analysis method for a test object, the method being applied to an experimental platform, the experimental platform including a first experimental platform, a second experimental platform, and a first user traffic distribution terminal, wherein the input terminal of the first user traffic distribution terminal is connected to a service access terminal, and the output terminal of the first user traffic distribution terminal is connected to the first experimental platform and the second experimental platform, the method comprising: The first user routing terminal obtains access data generated by the user accessing the service access terminal; Based on the access data, it is determined whether the user is a specific type of user, and a first determination result is obtained; If the first determination result indicates that the user is not a specific type of user, then the user will be assigned to the second experimental platform, which is a platform for conducting experiments on all users of multiple types; The second experimental platform uses test objects to provide services to users, and analyzes the performance of the test objects for all users based on the service result data generated after the test objects provide services to users. If the first determination result indicates that the user is a specific type of user, then the user is assigned to the first experimental platform, which is a platform for conducting experiments on the specific type of user; the specific type is any one of the multiple types; the first experimental platform uses the test object to provide services to the user, and analyzes the performance of the test object for the specific type of user based on the service result data generated after the test object provides services to the user.

2. The method according to claim 1, wherein the second experimental platform includes a second user distribution terminal, the output terminal of the first user distribution terminal is connected to the input terminal of the second user distribution terminal, and the step of assigning the user to the second experimental platform specifically includes: The first user routing terminal provides the access data to the input terminal of the second user routing terminal through the output terminal of the first user routing terminal; The second user distribution terminal distributes the user to the second experimental platform through its output terminal.

3. The method according to claim 1, wherein the access data includes user identification data, and the step of determining whether the user is a specific type of user based on the access data to obtain a first determination result specifically includes: Obtain the user identification data contained in the access data; Query whether the user identifier data is contained in the user identifier database, and obtain the query results; The user identifier database includes user identifier data for each user belonging to the specific type; If the first determination result indicates that the user is not a specific type of user, then the user is assigned to the second experimental platform, specifically including: If the query result indicates that the user identifier data is not contained in the user identifier database, then the user will be assigned to the second experimental platform; If the first determination result indicates that the user is a specific type of user, then the user is assigned to the first experimental platform, specifically including: If the query result indicates that the user identifier data is contained in the user identifier database, then the user will be assigned to the first experimental platform.

4. The method according to claim 1, further comprising, before assigning the user to the first experimental platform: Determine whether the number of users assigned to the first experimental platform is less than or equal to a first threshold to obtain a second determination result; The process of assigning the user to the first experimental platform specifically includes: If the second determination result indicates that the number of users assigned to the first experimental platform is less than or equal to the first threshold, then the user will be assigned to the first experimental platform.

5. The method according to claim 4, further comprising, before determining whether the number of users assigned to the first experimental platform is less than or equal to the first threshold: Obtain historical analysis results information, which is the result information of performance analysis of the test object using historical service result data; The historical service results are the service result data of historical users who enjoyed the services provided by the test object in the first experimental platform; If the historical analysis results meet the preset requirements, a preset value is added to the second threshold to obtain the first threshold; the second threshold is the threshold used when allocating historical users to the first experimental platform; if the number of users already allocated to the first experimental platform is less than or equal to the second threshold before allocating the historical users to the first experimental platform, then the historical users are allocated to the first experimental platform.

6. The method according to claim 1, wherein the access data includes scenario identification information, the first experimental platform is a platform for testing preset business scenarios, and the method further includes: Obtain the scene identification information from the access data; Determine whether the business scenario corresponding to the scenario identification information belongs to the preset business scenario to obtain a third determination result; If the third judgment result indicates that the business scenario does not belong to the preset business scenario, then the user will be assigned to the second experimental platform.

7. The method according to claim 6, wherein the preset business scenario includes multiple business scenarios, the first experimental platform includes multiple sub-experimental platforms, each of the sub-experimental platforms corresponds to one of the multiple business scenarios, and after determining whether the business scenario corresponding to the scenario identifier information belongs to the preset business scenario, the method further includes: If the third judgment result indicates that the business scenario belongs to the preset business scenario, then the target sub-experimental platform corresponding to the business scenario is determined; The user is assigned to the target sub-experimental platform.

8. The method according to claim 7, wherein the first experimental platform includes a control group and a test group, and the target sub-experimental platform includes a sub-control group and a sub-test group; The process of assigning the user to the target sub-experimental platform specifically includes: If the user is assigned to the control group in the first experimental platform, then the user is assigned to the sub-control group in the target sub-experimental platform; If the user is assigned to the test group in the first experimental platform, then the user is assigned to the sub-test group in the target sub-experimental platform.

9. The method according to claim 1, wherein the access data includes user ID information, the first experimental platform includes a control group and a test group, and the step of assigning the user to the first experimental platform specifically includes: Obtain the user ID information from the access data; The user ID information is hashed using a hash algorithm to obtain a hash value. If the hash value is of the first type of data, then the user is assigned to the control group; If the hash value is of the second type of data, then the user is assigned to the test group.

10. The method according to claim 1, wherein the first experimental platform comprises a plurality of sub-experimental platforms, each sub-experimental platform is configured with a sub-user routing terminal for setting the user routing ratio participating in the sub-experimental platform, and the method further comprises: If a new target sub-experimental platform is added to the first experimental platform, experimental users are allocated to the target sub-experimental platform by using the target sub-user diversion terminal settings of the target sub-experimental platform in a manner that gradually increases the initial user diversion ratio.

11. The method according to claim 1, wherein the first experimental platform comprises a first sub-experimental platform and a second sub-experimental platform. For the first experiment under the first sub-experiment platform and the second experiment under the second sub-experiment platform, the user traffic splitting ratio of the users participating in the experiment is increased in a gradually increasing manner; if the first access volume of the first business scenario corresponding to the first sub-experiment platform is greater than the second access volume of the second business scenario corresponding to the second sub-experiment platform, then the expansion step size for increasing the user traffic splitting ratio of the first experiment is set to be smaller than the expansion step size for increasing the user traffic splitting ratio of the second experiment.

12. The method according to claim 1, wherein the first experimental platform is used to conduct platform experiments on the test object, the second experimental platform is used to conduct vertical experiments on the test object, the platform experiments are used to test the performance of the test object in multiple scenarios, the vertical experiments are used to test the performance of the test object in a single scenario, and the test object includes an algorithm or model for providing business services.

13. A performance analysis device for a test object, the device being applied to an experimental platform, the device comprising a first experimental module, a second experimental module, and a first user traffic splitting module, wherein the input terminal of the first user traffic splitting module is connected to a service access terminal, and the output terminal of the first user traffic splitting module is connected to the first experimental module and the second experimental module, the first user traffic splitting module comprising: The acquisition unit is used to acquire access data generated by the user accessing the service access terminal; The judgment unit is used to determine whether the user is a specific type of user based on the access data, and to obtain a first judgment result; The first allocation unit is configured to allocate the user to the second experimental module if the first judgment result indicates that the user is not a specific type of user. The second experimental module is a module that conducts experiments on all users of multiple types. The second experimental module uses test objects to provide services to users and analyzes the performance of the test objects for all users based on the service result data generated after the test objects provide services to users. The second allocation unit is configured to allocate the user to the first experimental module if the first determination result indicates that the user is a specific type of user. The first experimental module is a module that conducts experiments on the specific type of user. The specific type is any one of the multiple types. The first experimental module uses the test object to provide services to the user and analyzes the performance of the test object for the specific type of user based on the service result data generated after the test object provides services to the user.

14. A computing device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform performance analysis of the test object according to any one of claims 1 to 12.

15. A computer-readable medium having stored thereon computer-readable instructions that can be executed by a processor to perform performance analysis of the test object according to any one of claims 1 to 12.

Images