Product testing method and system based on AI large model and machine learning

Abstract

The invention relates to the technical field of data processing, in particular to a product testing method and system based on an AI large model and machine learning, and the method comprises the steps: collecting product testing data, sequentially carrying out the cleaning, merging and conversion of the product testing data, and outputting optimized data, extracting features of the optimized data by using a machine learning algorithm, and training an original model by using the features to output an AI large model; updating the AI large model by using newly collected product test data, and predicting a test result of the product by using the AI large model to output a prediction result; determining the stability of the product test based on the packet loss rate of the product test data; if the stability of the product test does not meet the requirement, adjusting the transmission rate of the product test data; and if the training effectiveness does not meet the requirement, adjusting the learning rate of the AI large model. According to the invention, the stability of product testing is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a product testing method and system based on AI big model and machine learning. Background Art

[0002] In the existing technology, AI big models refer to "large parameter" models trained using large-scale data and powerful computing power. These models are usually highly versatile and generalizable, and can be applied to natural language processing, image recognition, speech recognition and other fields. Machine learning is a multidisciplinary cross-disciplinary major that covers knowledge of probability theory, statistics, approximate theory and complex algorithms. It uses computers as tools and is committed to simulating human learning methods in real time, and divides existing content into knowledge structures to effectively improve learning efficiency. The combination of AI big models and machine learning algorithms can effectively improve the test stability during product testing.

[0003] Chinese Patent Publication No.: CN118484388A discloses a product testing method and system, the method comprising: connecting the product to be tested to a preset product testing system, and triggering a test switch according to the connection status between the product to be tested and the product testing system; obtaining the operating system type of the product to be tested, calling the corresponding main test program based on the operating system type, and determining the test parameters of the main test program; testing the product to be tested according to the test parameters of the main test program; when the main test program is finished running, uploading the test data to the server to complete the product test. It can be seen that the product testing method and system have the problem of inaccurate prediction of the trained model due to network instability when multiple devices collaborate to collect data, resulting in partial data loss. Summary of the invention

[0004] To this end, the present invention provides a product testing method and system based on AI big model and machine learning, so as to overcome the problem in the prior art that due to network instability when multiple devices collaborate to collect data, some data is lost, resulting in inaccurate prediction of the trained model and thus causing reduced stability of product testing.

[0005] To achieve the above-mentioned objectives, the present invention provides a product testing method based on AI big model and machine learning, comprising: collecting product test data, cleaning, merging and converting the product test data in sequence to output optimized data, extracting features of the optimized data using a machine learning algorithm, and training the original model using the features to output the AI ​​big model; updating the AI ​​big model using the newly collected product test data, and predicting the test results of the product using the AI ​​big model to output the prediction results; respectively obtaining the amount of lost data and the total amount of product test data in several collection cycles; determining the stability of the product test based on the packet loss rate of the product test data; if the stability of the product test does not meet the requirements, adjusting the transmission rate of the product test data, or determining the training effectiveness of the AI ​​big model based on the extension ratio of the training time of the AI ​​big model; if the training effectiveness does not meet the requirements, adjusting the learning rate of the AI ​​big model, or adjusting the batch size of the product test data based on the average update delay time of the AI ​​big model.

[0006] Further, determining the stability of the product test includes:

[0007] Compare the packet loss rate of the product test data with the preset first packet loss rate;

[0008] If the packet loss rate of the product test data is greater than the preset first packet loss rate, it is determined that the stability of the product test does not meet the requirements.

[0009] Furthermore, determining the training effectiveness of the AI ​​big model includes:

[0010] Comparing the packet loss rate of the product test data with the preset first packet loss rate and the preset second packet loss rate respectively;

[0011] If the packet loss rate of the product test data is greater than the preset first packet loss rate and less than or equal to the preset second packet loss rate, it is preliminarily determined that the training effectiveness of the AI ​​big model does not meet the requirements, and whether the training effectiveness of the AI ​​big model meets the requirements is determined based on the extension ratio of the training time of the AI ​​big model.

[0012] Further, adjusting the transmission rate of the product test data includes:

[0013] Comparing the packet loss rate of the product test data with the preset second packet loss rate;

[0014] If the packet loss rate of the product test data is greater than a preset second packet loss rate, the transmission rate of the product test data is reduced.

[0015] Furthermore, the reduction range of the transmission rate of the product test data is determined by the difference between the packet loss rate of the product test data and a preset second packet loss rate.

[0016] Furthermore, the learning rate of the AI ​​large model is adjusted, including:

[0017] Compare the training time extension ratio of the AI ​​large model with the preset first extension ratio and the preset second extension ratio respectively;

[0018] If the extension ratio of the training time of the AI ​​large model is greater than the preset first extension ratio, it is determined that the training effectiveness of the AI ​​large model does not meet the requirements;

[0019] If the extension ratio of the training time of the AI ​​large model is greater than the preset first extension ratio and less than or equal to the preset second extension ratio, reducing the learning rate of the AI ​​large model;

[0020] If the training time extension ratio of the AI ​​big model is greater than the preset second extension ratio, it is preliminarily determined that the real-time update of the AI ​​big model does not meet the requirements, and whether the real-time update of the AI ​​big model meets the requirements is determined based on the average update delay time of the AI ​​big model.

[0021] Furthermore, the reduction range of the learning rate of the AI ​​large model is determined by the difference between the training time extension ratio of the AI ​​large model and a preset first extension ratio.

[0022] Further, adjusting the batch size of the product test data includes:

[0023] Compare the average update delay time of the AI ​​large model with the preset delay time;

[0024] If the average update delay duration of the AI ​​big model is greater than the preset delay duration, it is determined that the update real-time performance of the AI ​​big model does not meet the requirements, and the batch size of the product test data is reduced.

[0025] Furthermore, the reduction extent of the batch size of the product test data is determined by the difference between the average update delay duration of the AI ​​large model and the preset delay duration.

[0026] The present invention also provides a product testing system based on AI big model and machine learning, characterized in that it includes:

[0027] Data acquisition module, used to collect product test data;

[0028] A data processing module connected to the data acquisition module, comprising a preprocessing unit for preprocessing the product test data to output optimized data and a feature extraction unit connected to the preprocessing unit for extracting features from the optimized data using a machine learning algorithm;

[0029] A model training module, which is connected to the data processing module, including a model training unit connected to the feature extraction unit for training the original model according to the feature to output an AI big model, a result prediction unit connected to the model training unit for predicting the test result of the product according to the AI ​​big model to output the prediction result, and a model updating unit connected to the model training unit for updating the AI ​​big model according to newly collected product test data;

[0030] A storage module, which is respectively connected to the data acquisition module, the data processing module and the model training module, and is used to store the product test data, the optimization data, the features, the machine learning algorithm, the AI ​​big model and the prediction results respectively;

[0031] A control module, which is respectively connected to the data acquisition module, the data processing module, the model training module and the storage module, and is used to determine the transmission rate of the product test data according to the packet loss rate based on the product test data, or to determine the learning rate of the AI ​​big model according to the extension ratio of the training time of the AI ​​big model, and to determine the batch size of the product test data according to the extension ratio of the training time of the AI ​​big model and the average update delay duration of the AI ​​big model.

[0032] Compared with the prior art, the beneficial effect of the present invention lies in that the method of the present invention adjusts the transmission rate of product test data according to the packet loss rate of product test data. Since the network is unstable when multiple devices collaborate to collect data, some data is lost, which leads to inaccurate prediction of the trained model. By reducing the transmission rate of product test data, data can be transmitted in the network in a more stable manner, reducing the network burden, reducing the probability of data loss, and enabling more data to reach the destination intact for model training. The learning rate of the AI ​​large model is adjusted according to the extension ratio of the training time of the AI ​​large model. Since the collected data contains multiple types of data, some data noise is not cleaned up during cleaning, resulting in noise interference, which is not conducive to model training. During training, it will try to fit these noises, resulting in an increase in the number of training rounds, which will lead to overtraining of the model. By reducing the learning rate of the machine learning model, the model can have more time and opportunities to distinguish between valid information and noise in the data, avoiding overtraining due to fitting noise too quickly. The batch size of the product test data is adjusted according to the average update delay time of the AI ​​large model. Due to the large amount of data collected, it takes a long time to store, read and process large-scale data, making it impossible for the model to quickly obtain the latest processed data for update. Reducing the batch size of product test data can complete the processing of each batch more quickly, so that the model can obtain some processed data for update more quickly, thereby improving the stability of product testing.

[0033] Furthermore, the method of the present invention adjusts the transmission rate of product test data by setting a preset first packet loss rate and a preset second packet loss rate. Since the network is unstable when multiple devices collaborate to collect data, some data is lost, which leads to inaccurate predictions of the trained model. By reducing the transmission rate of product test data, data can be transmitted in the network in a smoother manner, reducing the network burden, reducing the probability of data loss, and enabling more data to reach the destination intact for model training, further improving the stability of product testing.

[0034] Furthermore, the method of the present invention adjusts the learning rate of the AI ​​large model by setting a preset first extension ratio and a preset second extension ratio. Since the collected data contains multiple types of data, some data noise is not cleaned up during cleaning, resulting in noise interference. During model training, the model will try to fit these noises, resulting in an increase in the number of training rounds, thereby causing over-training of the model. By reducing the learning rate of the machine learning model, the model can have more time and opportunities to distinguish between valid information and noise in the data, avoiding over-training due to fitting noise too quickly, and further improving the stability of product testing.

[0035] Furthermore, the method of the present invention adjusts the batch size of product test data by setting a preset delay time. Since the amount of collected data is too large, it will take a long time to store, read and process large-scale data, making it impossible for the model to quickly obtain the latest processed data for updating. Reducing the batch size of product test data can complete the processing of each batch more quickly, so that the model can obtain some processed data for updating more quickly, further improving the stability of product testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is an overall flow chart of a product testing method based on an AI big model and machine learning according to an embodiment of the present invention;

[0037] Figure 2 This is an overall structural block diagram of a product testing system based on an AI big model and machine learning according to an embodiment of the present invention;

[0038] Figure 3 This is a logical flow chart of a product testing method based on an AI big model and machine learning according to an embodiment of the present invention;

[0039] Figure 4 This is a specific flow chart of the process of adjusting the transmission rate of product test data in the product testing method based on the AI ​​big model and machine learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0042] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0043] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] See also Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, they are respectively the overall flow chart of the product testing method and system based on AI big model and machine learning according to the embodiment of the present invention, the overall structure block diagram, the logic flow chart and the specific flow chart of the process of adjusting the transmission rate of product test data. Figure 1 As shown, the present invention provides a product testing method based on AI big model and machine learning, comprising:

[0045] Step S1, collecting product test data, cleaning, merging and converting the product test data in sequence to output optimized data, using a machine learning algorithm to extract features of the optimized data, and using the features to train the original model to output an AI large model;

[0046] Step S2, using the newly collected product test data to update the AI ​​big model, and using the AI ​​big model to predict the product test results to output the prediction results;

[0047] Step S3, respectively obtaining the amount of lost data of product test data and the total amount of product test data in several acquisition cycles;

[0048] Step S4, determining the stability of the product test based on the packet loss rate of the product test data;

[0049] Step S5, if the stability of the product test does not meet the requirements, the transmission rate of the product test data is adjusted, or the training effectiveness of the AI ​​big model is determined based on the extension ratio of the training time of the AI ​​big model;

[0050] Step S6: If the training effectiveness does not meet the requirements, the learning rate of the AI ​​large model is adjusted, or the batch size of the product test data is adjusted based on the average update delay time of the AI ​​large model.

[0051] Specifically, the products include office software, image processing software, and electronic products.

[0052] Specifically, product testing data includes the number of user clicks, software startup time, and software memory usage data.

[0053] Specifically, the optimized data includes software response time after removing missing values, product names in a unified format, and merged user names.

[0054] Specifically, machine learning algorithms include decision trees, random forests, and support vector machines.

[0055] Specifically, the features include the average response time of the software, the byte size of the product name, and the byte size of the user name.

[0056] Specifically, large AI models include Transformer models, neural network models, and GPT models.

[0057] Specifically, the newly collected product test data is the product test data collected after the training of the AI ​​big model is completed.

[0058] Specifically, the prediction results include resource utilization during product operation, fluctuations in product sales, and the probability of errors in product functions.

[0059] Specifically, the learning rate of the AI ​​large model is the step size used by the AI ​​large model each time it updates the model parameters during the training process.

[0060] Specifically, the batch size of product test data is the number of data samples processed simultaneously each time during the product testing process.

[0061] In implementation, the method of the present invention adjusts the transmission rate of product test data according to the packet loss rate of product test data. Since the network is unstable when multiple devices collaborate to collect data, some data is lost, resulting in inaccurate prediction of the trained model. By reducing the transmission rate of product test data, data can be transmitted in the network in a more stable manner, reducing the network burden, reducing the probability of data loss, and allowing more data to reach the destination intact for model training. The learning rate of the AI ​​large model is adjusted according to the extension ratio of the training time of the AI ​​large model. Since the collected data contains multiple types of data, some data noise is not cleaned up during cleaning, resulting in noise interference, and the model will try to fit during training. These noises lead to an increase in the number of training rounds, which in turn causes overtraining of the model. By reducing the learning rate of the machine learning model, the model can have more time and opportunities to distinguish between valid information and noise in the data, avoiding overtraining due to fitting noise too quickly. The batch size of the product test data is adjusted according to the average update delay time of the AI ​​large model. Due to the large amount of data collected, it takes a long time to store, read and process large-scale data, making it impossible for the model to quickly obtain the latest processed data for updating. Reducing the batch size of product test data can complete the processing of each batch more quickly, so that the model can obtain some processed data for updating more quickly, thereby improving the stability of product testing.

[0062] Specifically, the stability of the product test is determined, including:

[0063] Obtaining the amount of lost data and the total amount of product test data in several collection cycles respectively, and calculating the packet loss rate of the product test data;

[0064] Comparing the packet loss rate of the product test data with a preset first packet loss rate;

[0065] If the packet loss rate of the product test data is greater than the preset first packet loss rate, it is determined that the stability of the product test does not meet the requirements.

[0066] Specifically, determining the training effectiveness of the AI ​​big model includes:

[0067] Comparing the packet loss rate of the product test data with the preset first packet loss rate and the preset second packet loss rate respectively;

[0068] If the packet loss rate of the product test data is greater than the preset first packet loss rate and less than or equal to the preset second packet loss rate, it is preliminarily determined that the training effectiveness of the AI ​​big model does not meet the requirements, and whether the training effectiveness of the AI ​​big model meets the requirements is determined based on the extension ratio of the training time of the AI ​​big model.

[0069] It can be understood that the three intervals corresponding to the preset first packet loss rate and the preset second packet loss rate correspond to three situations respectively:

[0070] The first interval is when the packet loss rate of the product test data is less than or equal to the preset first packet loss rate, corresponding to the situation where the stability of the product test meets the requirements;

[0071] The second interval is when the packet loss rate of the product test data is greater than the preset first packet loss rate and less than or equal to the preset second packet loss rate. The corresponding collected data contains multiple types of data, which results in some data noise not being cleaned up during cleaning, resulting in noise interference. During model training, the model will try to fit these noises, resulting in an increase in the number of training rounds, which leads to overtraining of the model.

[0072] The third interval is when the packet loss rate of product test data is greater than the preset second packet loss rate. This corresponds to the situation where the network is unstable when multiple devices collaborate to collect data, resulting in partial data loss, which in turn leads to inaccurate predictions by the trained model.

[0073] In practice, the preset first packet loss rate is generally selected in the range of [4%, 6%], and the preset second packet loss rate is generally selected in the range of [7%, 9%].

[0074] Preferably, the preferred embodiment of the preset first packet loss rate is 5%, and the preferred embodiment of the preset second packet loss rate is 8%.

[0075] Specifically, the packet loss rate of product test data is the ratio of the amount of lost data of product test data in several collection cycles to the total amount of product test data.

[0076] In implementation, the method of the present invention determines the stability of product testing by setting a preset first packet loss rate and a preset second packet loss rate, thereby reducing the impact of decreased accuracy of product testing due to inaccurate determination of the stability of product testing, and further improving the stability of product testing.

[0077] Specifically, adjusting the transmission rate of the product test data includes:

[0078] Comparing the packet loss rate of the product test data with the preset second packet loss rate;

[0079] If the packet loss rate of the product test data is greater than a preset second packet loss rate, the transmission rate of the product test data is reduced.

[0080] Specifically, the reduction range of the transmission rate of the product test data is determined by the difference between the packet loss rate of the product test data and a preset second packet loss rate.

[0081] Specifically, when the difference between the packet loss rate of the product test data and the preset second packet loss rate is within 3%, the transmission rate of the product test data is reduced to 0.95 times the original rate; when the difference between the packet loss rate of the product test data and the preset second packet loss rate exceeds 3%, the transmission rate of the product test data is reduced by 50KB / s for every 2% exceeding it. For example, the difference between the packet loss rate of the product test data and the preset second packet loss rate is 7%, the current transmission rate of the product test data is 600KB / s, and the reduced transmission rate of the product test data is 600×0.95-50×2=470KB / s.

[0082] In implementation, the method of the present invention adjusts the transmission rate of product test data by setting a preset first packet loss rate and a preset second packet loss rate. Since the network is unstable when multiple devices collaborate to collect data, some data is lost, which leads to inaccurate prediction of the trained model. By reducing the transmission rate of product test data, data can be transmitted in the network in a smoother manner, reducing the network burden, reducing the probability of data loss, and enabling more data to reach the destination intact for model training, further improving the stability of product testing.

[0083] Specifically, the learning rate of the AI ​​large model is adjusted, including:

[0084] Obtain the actual training time and standard training time of the AI ​​large model respectively, and calculate the extension ratio of the training time of the AI ​​large model;

[0085] Compare the training time extension ratio of the AI ​​large model with the preset first extension ratio and the preset second extension ratio respectively;

[0086] If the extension ratio of the training time of the AI ​​large model is greater than the preset first extension ratio, it is determined that the training effectiveness of the AI ​​large model does not meet the requirements;

[0087] If the extension ratio of the training time of the AI ​​large model is greater than the preset first extension ratio and less than or equal to the preset second extension ratio, reducing the learning rate of the AI ​​large model;

[0088] If the training time extension ratio of the AI ​​big model is greater than the preset second extension ratio, it is preliminarily determined that the real-time update of the AI ​​big model does not meet the requirements, and whether the real-time update of the AI ​​big model meets the requirements is determined based on the average update delay time of the AI ​​big model.

[0089] It can be understood that the three intervals corresponding to the preset first extension ratio and the preset second extension ratio correspond to three situations respectively:

[0090] The first interval is when the training time extension ratio of the AI ​​large model is greater than the preset first extension ratio, corresponding to the situation where the training effectiveness of the AI ​​large model meets the requirements;

[0091] The second interval is when the training time extension ratio of the AI ​​large model is greater than the preset first extension ratio and less than or equal to the preset second extension ratio. The corresponding collected data contains multiple types of data, which results in some data noise not being cleaned during cleaning, resulting in noise interference. During model training, the model will try to fit these noises, resulting in an increase in the number of training rounds, thus causing the model to be overtrained.

[0092] The third interval is when the training time extension ratio of the large AI model is greater than the preset second extension ratio. This corresponds to the situation where the amount of collected data is too large, resulting in a long time spent on storing, reading and processing large-scale data, making it impossible for the model to quickly obtain the latest processed data for updating.

[0093] In practice, the preset first extension ratio is generally selected in the range of [8%, 12%], and the preset second extension ratio is generally selected in the range of [13%, 17%].

[0094] Preferably, the preset first extension ratio is 10%, and the preset second extension ratio is 15%.

[0095] Specifically, the extension ratio of the training time of the AI ​​large model is the ratio of the difference between the actual training time of the AI ​​large model and the standard training time of the AI ​​large model to the standard training time of the AI ​​large model.

[0096] Specifically, the standard training time of the AI ​​big model is the time required for AI big model training when the product testing method described in this solution is not applied.

[0097] In implementation, the method of the present invention determines the training effectiveness of the AI ​​large model by setting a preset first extension ratio and a preset second extension ratio, thereby reducing the impact of the decreased stability of product testing due to inaccurate determination of the training effectiveness of the AI ​​large model, and further improving the stability of product testing.

[0098] Specifically, the reduction extent of the learning rate of the AI ​​large model is determined by the difference between the training time extension ratio of the AI ​​large model and the preset first extension ratio.

[0099] Specifically, when the difference between the extension ratio of the training time of the AI ​​large model and the preset first extension ratio is within 5%, the learning rate of the AI ​​large model is reduced to 0.92 times the original; when the difference between the extension ratio of the training time of the AI ​​large model and the preset first extension ratio exceeds 5%, the learning rate of the AI ​​large model is reduced by 0.003 for every 2% exceeding it. For example, the difference between the extension ratio of the training time of the AI ​​large model and the preset first extension ratio is 9%. The current learning rate of the AI ​​large model is 0.05, and the reduced learning rate of the AI ​​large model is 0.05×0.92-0.003×2=0.04.

[0100] In implementation, the method of the present invention adjusts the learning rate of the AI ​​large model by setting a preset first extension ratio and a preset second extension ratio. Since the collected data contains multiple types of data, some data noise is not cleaned up during cleaning, resulting in noise interference. During model training, the model will try to fit these noises, resulting in an increase in the number of training rounds, thereby causing over-training of the model. By reducing the learning rate of the machine learning model, the model can have more time and opportunities to distinguish between valid information and noise in the data, avoiding over-training due to fitting noise too quickly, and further improving the stability of product testing.

[0101] Specifically, adjusting the batch size of the product test data includes:

[0102] Obtain the update delay time of the AI ​​big model in several update cycles, and calculate the average update delay time of the AI ​​big model;

[0103] Compare the average update delay time of the AI ​​large model with the preset delay time;

[0104] If the average update delay duration of the AI ​​big model is greater than the preset delay duration, it is determined that the update real-time performance of the AI ​​big model does not meet the requirements, and the batch size of the product test data is reduced.

[0105] It can be understood that the two intervals corresponding to the preset delay time correspond to two situations:

[0106] The first interval is when the average update delay of the AI ​​big model is less than or equal to the preset delay, corresponding to the situation where the update real-time performance of the AI ​​big model meets the requirements;

[0107] The second interval is when the average update delay of the large AI model is longer than the preset delay, which corresponds to a large amount of collected data. As a result, it takes a long time to store, read and process large-scale data, making it impossible for the model to quickly obtain the latest processed data for updating.

[0108] In practice, the preset delay time is generally selected in the range of [3s, 7s].

[0109] Preferably, the preferred embodiment of the preset delay time length is 5s.

[0110] Specifically, the average update delay time of the AI ​​big model is the ratio of the total update delay time of the AI ​​big model in several update cycles to the number of update cycles.

[0111] In implementation, the method of the present invention determines the real-time update of the AI ​​big model by setting a preset delay period, thereby reducing the impact of the decreased stability of product testing due to inaccurate determination of the real-time update of the AI ​​big model, and further improving the stability of product testing.

[0112] Specifically, the reduction extent of the batch size of the product test data is determined by the difference between the average update delay time of the AI ​​large model and the preset delay time.

[0113] Specifically, when the difference between the average update delay of the AI ​​large model and the preset delay is within 2s, the batch size of the product test data is reduced to 0.9 times the original size; when the difference between the average update delay of the AI ​​large model and the preset delay exceeds 2s, the batch size of the product test data is reduced by 3 for every 2s exceeding the preset delay. For example, the difference between the average update delay of the AI ​​large model and the preset delay is 6s. The current batch size of the product test data is 250, and the reduced batch size of the product test data is 250×0.9-3×2=219.

[0114] In implementation, the method of the present invention adjusts the batch size of product test data by setting a preset delay time. Since the amount of collected data is too large, it will take a long time to store, read and process large-scale data, making it impossible for the model to quickly obtain the latest processed data for updating. Reducing the batch size of product test data can complete the processing of each batch more quickly, so that the model can obtain part of the processed data for updating more quickly, further improving the stability of product testing.

[0115] like Figure 2 As shown, an embodiment of the present invention provides a product testing system based on an AI big model and machine learning, including:

[0116] Data acquisition module, used to collect product test data;

[0117] A data processing module connected to the data acquisition module, comprising a preprocessing unit for preprocessing the product test data to output optimized data and a feature extraction unit connected to the preprocessing unit for extracting features from the optimized data using a machine learning algorithm;

[0118] A model training module, which is connected to the data processing module, including a model training unit connected to the feature extraction unit for training the original model according to the feature to output an AI big model, a result prediction unit connected to the model training unit for predicting the test result of the product according to the AI ​​big model to output the prediction result, and a model updating unit connected to the model training unit for updating the AI ​​big model according to newly collected product test data;

[0119] A storage module, which is respectively connected to the data acquisition module, the data processing module and the model training module, and is used to store the product test data, the optimization data, the features, the machine learning algorithm, the AI ​​big model and the prediction results respectively;

[0120] A control module, which is respectively connected to the data acquisition module, the data processing module, the model training module and the storage module, and is used to determine the transmission rate of the product test data according to the packet loss rate based on the product test data, or to determine the learning rate of the AI ​​big model according to the extension ratio of the training time of the AI ​​big model, and to determine the batch size of the product test data according to the extension ratio of the training time of the AI ​​big model and the average update delay duration of the AI ​​big model.

[0121] Specifically, preprocessing includes cleaning, merging, and transforming product test data in sequence.

[0122] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A product testing method based on AI big model and machine learning, characterized in that: include: Collect product test data, clean, merge and convert the product test data in sequence to output optimized data, use a machine learning algorithm to extract features of the optimized data, and use the features to train the original model to output an AI large model; Using the newly collected product test data to update the AI ​​big model, using the AI ​​big model to predict the product test results to output the prediction results; respectively obtain the amount of lost data and the total amount of product test data in several collection cycles; Determine the stability of product testing based on the packet loss rate of product test data; If the stability of the product test does not meet the requirements, the transmission rate of the product test data is adjusted, or the training effectiveness of the AI ​​big model is determined based on the extension ratio of the training time of the AI ​​big model; If the training effectiveness does not meet the requirements, the learning rate of the AI ​​large model is adjusted, or the batch size of the product test data is adjusted based on the average update delay time of the AI ​​large model.

2. The product testing method based on AI big model and machine learning according to claim 1 is characterized in that: Determine the robustness of the product test, including: Compare the packet loss rate of the product test data with the preset first packet loss rate; If the packet loss rate of the product test data is greater than the preset first packet loss rate, it is determined that the stability of the product test does not meet the requirements.

3. The product testing method based on AI big model and machine learning according to claim 2 is characterized in that: Determine the training effectiveness of the AI ​​big model, including: Comparing the packet loss rate of the product test data with the preset first packet loss rate and the preset second packet loss rate respectively; If the packet loss rate of the product test data is greater than the preset first packet loss rate and less than or equal to the preset second packet loss rate, it is preliminarily determined that the training effectiveness of the AI ​​big model does not meet the requirements, and whether the training effectiveness of the AI ​​big model meets the requirements is determined based on the extension ratio of the training time of the AI ​​big model.

4. The product testing method based on AI big model and machine learning according to claim 3 is characterized in that: The transmission rate of the product test data is adjusted, including: Comparing the packet loss rate of the product test data with the preset second packet loss rate; If the packet loss rate of the product test data is greater than a preset second packet loss rate, the transmission rate of the product test data is reduced.

5. The product testing method based on AI big model and machine learning according to claim 4 is characterized in that: The reduction range of the transmission rate of the product test data is determined by the difference between the packet loss rate of the product test data and a preset second packet loss rate.

6. The product testing method based on AI big model and machine learning according to claim 3 is characterized in that: The learning rate of the AI ​​large model is adjusted, including: Compare the training time extension ratio of the AI ​​large model with the preset first extension ratio and the preset second extension ratio respectively; If the extension ratio of the training time of the AI ​​large model is greater than the preset first extension ratio, it is determined that the training effectiveness of the AI ​​large model does not meet the requirements; If the extension ratio of the training time of the AI ​​large model is greater than the preset first extension ratio and less than or equal to the preset second extension ratio, reducing the learning rate of the AI ​​large model; If the training time extension ratio of the AI ​​big model is greater than the preset second extension ratio, it is preliminarily determined that the real-time update of the AI ​​big model does not meet the requirements, and whether the real-time update of the AI ​​big model meets the requirements is determined based on the average update delay time of the AI ​​big model.

7. The product testing method based on AI big model and machine learning according to claim 6 is characterized in that: The reduction range of the learning rate of the AI ​​large model is determined by the difference between the extension ratio of the training time of the AI ​​large model and the preset first extension ratio.

8. The product testing method based on AI big model and machine learning according to claim 7 is characterized in that: The batch size of the product test data is adjusted, including: Compare the average update delay time of the AI ​​large model with the preset delay time; If the average update delay duration of the AI ​​big model is greater than the preset delay duration, it is determined that the update real-time performance of the AI ​​big model does not meet the requirements, and the batch size of the product test data is reduced.

9. The product testing method based on AI big model and machine learning according to claim 8 is characterized in that: The reduction extent of the batch size of the product test data is determined by the difference between the average update delay time of the AI ​​large model and the preset delay time.

10. A product testing system using the product testing method based on AI big model and machine learning according to any one of claims 1 to 9, characterized in that: include: Data acquisition module, used to collect product test data; A data processing module connected to the data acquisition module, comprising a preprocessing unit for preprocessing the product test data to output optimized data and a feature extraction unit connected to the preprocessing unit for extracting features from the optimized data using a machine learning algorithm; A model training module, which is connected to the data processing module, including a model training unit connected to the feature extraction unit for training the original model according to the feature to output an AI big model, a result prediction unit connected to the model training unit for predicting the test result of the product according to the AI ​​big model to output the prediction result, and a model updating unit connected to the model training unit for updating the AI ​​big model according to newly collected product test data; A storage module, which is respectively connected to the data acquisition module, the data processing module and the model training module, and is used to store the product test data, the optimization data, the features, the machine learning algorithm, the AI ​​big model and the prediction results respectively; A control module, which is respectively connected to the data acquisition module, the data processing module, the model training module and the storage module, and is used to determine the transmission rate of the product test data according to the packet loss rate based on the product test data, or to determine the learning rate of the AI ​​big model according to the extension ratio of the training time of the AI ​​big model, and to determine the batch size of the product test data according to the extension ratio of the training time of the AI ​​big model and the average update delay duration of the AI ​​big model.

Images