Model precision problem checking method and device
By gradually reducing the module size of the deep learning model and selecting some modules for troubleshooting, the problem of inefficiency of existing tools in complex models is solved, efficient accuracy problem positioning and optimization is achieved, and the analysis process is simplified.
Patent Information
- Application Number
- CN202411275511.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-17
AI Technical Summary
The existing deep learning model accuracy problem-solving tools are inefficient when facing complex models, and require a lot of time and energy to check module by module. Traditional methods are difficult to identify accuracy problems during multiple trainings.
By gradually reducing the size of the model, selecting some modules from multiple modules as troubleshooting modules, gradually reducing the scope of the problem until a single module, optimizing the parameters of the problem module to locate the accuracy problem, and avoiding extensive and lengthy checks.
It improves the efficiency of accuracy problem detection, simplifies the analysis process, makes problem detection more controllable and easy to manage, reduces unnecessary module inspections, and improves overall efficiency.
Smart Images
Figure CN119150139A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for troubleshooting model accuracy problems. Background Art
[0002] During the development of deep learning models, it is critical to ensure the accuracy and reliability of the models. Troubleshooting accuracy issues during the training and inference phases of the model is a key step in evaluating model performance. However, accuracy issues in deep learning models are different from general software issues and are often more complex. Traditional software issues often involve errors in program code, which can be relatively easy to locate by checking the code and output results. However, in deep learning models, the root cause of accuracy issues may involve multiple factors, including data preprocessing, model architecture, training algorithms, and implementation details during inference.
[0003] Current accuracy troubleshooting tools are generally limited and can only troubleshoot accuracy problems for a single operator or a single module. When facing a complex model, it usually takes a lot of time to troubleshoot accuracy problems. If multiple modules in the model have accuracy problems, current accuracy troubleshooting tools can only compare them one by one and troubleshoot the accuracy of each module. This method involves a lot of repetitive work and requires a lot of energy and time. Summary of the invention
[0004] The present invention provides a method and device for troubleshooting model accuracy problems, so as to solve the defects existing in the related technology.
[0005] The present invention provides a method for troubleshooting model accuracy problems, comprising the following steps: Determine current output data of the current model and reference data corresponding to the current output data; If the current output data is not aligned with the reference data, some modules are selected from the multiple modules of the current model as current screening modules; After taking the model formed by the current screening modules as the current model, returning to the step of determining the current output data of the current model and the benchmark data corresponding to the current output data until the current model includes only one module; If the current output data is not aligned with the reference data, and the current model includes only one module, one module of the current model is used as a problem module.
[0006] According to a model accuracy problem troubleshooting method provided by the present invention, the selecting of some modules from the multiple modules of the current model as current troubleshooting modules includes: determining an intermediate module from among the plurality of modules; All modules between the first module and the middle module, the first module and the middle module are used as the current screening modules, or all modules between the middle module and the last module, the middle module and the last module are used as the current screening modules.
[0007] According to a model accuracy problem troubleshooting method provided by the present invention, if all modules between the first module and the intermediate module, the first module and the intermediate module are used as the current troubleshooting module, the current output data of the current model determined and the benchmark data corresponding to the current output data are returned, and then the method further includes: If the current output data is aligned with the benchmark data, all modules between the intermediate module and the last module, the intermediate module and the last module are used as the current screening modules, and the current output data of the current model determined and the benchmark data corresponding to the current output data are returned.
[0008] According to a model accuracy problem troubleshooting method provided by the present invention, if all modules between the intermediate module and the last module, the intermediate module and the last module are used as the current troubleshooting module, the current output data of the current model determined and the benchmark data corresponding to the current output data are returned, and then the method further includes: If the current output data is aligned with the benchmark data, the model composed of all modules between the first module and the intermediate module, the first module and the intermediate modules is used as the current screening module, and the current output data for determining the current model and the benchmark data corresponding to the current output data are returned.
[0009] According to a model accuracy problem troubleshooting method provided by the present invention, the method further includes: The parameters of the target module in the complete model are fixed, and the parameters of the problem module in the complete model are optimized, wherein the target module refers to all modules in the complete model except the problem module.
[0010] According to a model accuracy problem troubleshooting method provided by the present invention, the parameters of the problem module in the complete model are optimized based on the difference between the output data of the problem module and the benchmark data corresponding to the output data.
[0011] According to a model accuracy troubleshooting method provided by the present invention, the current output data is the output data corresponding to the current model when overfitting training is performed.
[0012] The present invention also provides a model accuracy problem troubleshooting device, comprising the following modules: A determination unit, used to determine current output data of a current model and reference data corresponding to the current output data; A selection unit, configured to select some modules from a plurality of modules of the current model as current screening modules if the current output data is not aligned with the reference data; A screening unit, configured to use the model formed by the current screening module as the current model, and then return to execute the process of determining the current output data of the current model and the reference data corresponding to the current output data, until the current model includes only one module; A positioning unit is used to use a module of the current model as a problem module if the current output data is not aligned with the reference data and the current model includes only one module.
[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for troubleshooting model accuracy problems as described above is implemented.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for troubleshooting model accuracy problems.
[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for troubleshooting model accuracy problems as described above is implemented.
[0016] The model accuracy problem troubleshooting method and device provided by the present invention focuses on the accuracy problem by gradually reducing the modules of the current model, rather than troubleshooting each module one by one. The present invention gradually narrows the scope of the problem, avoids extensive and lengthy inspections of all modules, and improves the efficiency of accuracy problem troubleshooting. In addition, when the model complexity is high, directly troubleshooting each module may be very complicated. The present invention can simplify the analysis process by gradually reducing the scale of the current model, making the problem troubleshooting more controllable and easy to manage. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 It is a flow chart of the model accuracy problem troubleshooting method provided by the present invention.
[0019] Figure 2 It is a structural schematic diagram of the current model provided by the present invention.
[0020] Figure 3 It is a flow chart of another method for troubleshooting model accuracy problems provided by the present invention.
[0021] Figure 4 It is a structural schematic diagram of the model accuracy problem troubleshooting device provided by the present invention.
[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0024] Current precision problem troubleshooting tools are generally limited and can only troubleshoot precision problems for a single operator or a single module. When facing a complex model, it usually takes a lot of time to troubleshoot precision problems. If multiple modules in the model have precision problems, current precision problem troubleshooting tools can only compare them one by one and troubleshoot the precision of each module. This method involves a lot of repetitive work and requires a lot of energy and time.
[0025] In addition, when the model is trained based on traditional methods, the behavior and performance of the model may be different in each training, and accuracy problems sometimes only appear after multiple training cycles. In other words, when the model is trained using traditional methods, a few training times may not fully identify accuracy problems in the model, so the model needs to be trained multiple times (i.e., multiple batches of training) to discover or reproduce accuracy problems, which is inefficient.
[0026] Figure 1 is a flow chart of the method for troubleshooting model accuracy problems provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .
[0027] Step 110: Determine current output data of the current model and reference data corresponding to the current output data.
[0028] Here, the current output data is the output data corresponding to the test of the current model, and the current model is a trained model. The current model can be a complete model or a sub-model of the complete model. The sub-model only contains some modules of the complete model. For example, the complete model includes module A, module B and module C, and the sub-model includes module A and module B.
[0029] The benchmark data corresponding to the current output data can be understood as the correct output data of the current model, wherein the benchmark data corresponding to the current output data can be golden data, and the benchmark data corresponding to the current output data can come from labels in the training set, standard knowledge provided by experts, output data of previous models that have been verified to be reliable, etc.
[0030] In addition, the current output data may be reference data corresponding to the output data of the previous module, and the previous module may be the previous module adjacent to the first module of the current model in the complete model.
[0031] Step 120: If the current output data is not aligned with the reference data, some modules are selected from the multiple modules of the current model as current screening modules; Step 130 , after taking the model composed of the current screening modules as the current model, return to execute and determine the current output data of the current model and the benchmark data corresponding to the current output data, until the current model includes only one module.
[0032] Specifically, the current output data and the reference data are not aligned, which means that there is a difference between the current output data and the reference data. The alignment between the current output data and the reference data can be measured by the mean square error, absolute error, etc. between the current output data and the reference data.
[0033] For example, if the mean square error between the current output data and the reference data is greater than a first threshold, it indicates that the two are not aligned, otherwise the two are aligned. For another example, if the absolute error between the current output data and the reference data is greater than a second threshold, it indicates that the two are not aligned, otherwise the two are aligned.
[0034] Furthermore, if the current output data is not aligned with the benchmark data, it indicates that the difference between the current output data and the correct result is large, which means that there is an accuracy problem with the current model. In this case, the accuracy problem of the current model needs to be checked.
[0035] Taking into account that the accuracy of each module in the model is checked one by one in the related art, this method involves a lot of repetitive work and requires a lot of energy and time. In the embodiment of the present invention, when the current output data is not aligned with the benchmark data, some modules are selected from multiple modules of the current model as current checking modules, and the model composed of the current checking modules is used as the current model, and the process returns to repeatedly execute steps 110 and 120 until the current model includes only one module.
[0036] Among them, since the current troubleshooting module is a partial module among multiple modules, the scale of the model composed of the current troubleshooting module is reduced, and steps 110 and 120 are repeated for the current model after the scale is reduced to continue the accuracy troubleshooting. That is, the embodiment of the present invention focuses on the accuracy problem by gradually reducing the modules of the current model, rather than troubleshooting each module one by one. The embodiment of the present invention avoids extensive and lengthy inspections of all modules by gradually narrowing the scope of the problem, thereby improving the efficiency of troubleshooting of accuracy problems.
[0037] In addition, when the model complexity is high, directly troubleshooting each module may be very complicated. The embodiment of the present invention can simplify the analysis process by gradually reducing the scale of the current model, making problem troubleshooting more controllable and easy to manage.
[0038] Step 140: If the current output data is not aligned with the reference data, and the current model includes only one module, then one module of the current model is used as a problem module.
[0039] Specifically, if the current output data is not aligned with the benchmark data, it indicates that the current model has an accuracy problem. Under this condition, if the current model only includes one module, it can be determined that the module must have an accuracy problem, and the module is regarded as a problem module, that is, the problem module can be understood as a module with an accuracy problem.
[0040] The above complete model and current model can be a text generation model, a text-to-graph model, an image classification model, an object detection model, a semantic segmentation model, a video generation model, a speech recognition model, etc. Taking the text generation model as an example, the input data of the complete model is the input text, and the output data is the output text corresponding to the input text.
[0041] Based on the above embodiment, some modules are selected from multiple modules of the current model as current screening modules, including: Identify an intermediate module from among the multiple modules; All modules between the first module and the middle module, the first module and the middle module are used as current screening modules, or all modules between the middle module and the last module, the middle module and the last module are used as current screening modules.
[0042] Specifically, the middle module can be understood as a module in the middle of multiple modules. For example, the current model includes module 1, module 2, module 3, module 4 and module 5 in order, and module 3 is in the middle, so module 3 is regarded as the middle module.
[0043] After the intermediate module is determined, there are two corresponding troubleshooting methods: (a) Select all modules from the first module to the middle module, as well as the first module and the middle module themselves, as the current troubleshooting modules. This troubleshooting method troubleshoots all modules from the first module to the middle module as a group, including the first module and the middle module, and is used to troubleshoot the first half of the module sequence.
[0044] (b) Select all modules from the middle module to the last module, as well as the middle module and the last module themselves, as the current screening modules. This screening method screens all modules from the middle module to the last module as a group, including the middle module and the last module, and is used to screen the second half of the module sequence.
[0045] It can be seen that after determining the middle module, the modules of the current model can be divided into two interval modules (the first half module and the second half module) for troubleshooting, so that the scope of the accuracy problem can be located more quickly, the time for invalid troubleshooting can be reduced, and the overall troubleshooting efficiency can be improved.
[0046] Taking the text generation model as an example, the current model in the embodiment of the present invention can be a complete text generation model, or a model composed of some modules in the complete generation model. Among them, the complete text generation model may include an encoder, a decoder and a generator. The encoder is used to encode the input text to obtain a vector representation of the input text. The decoder is used to gradually generate words or characters of the target text based on the vector representation of the input text. The generator is used to generate an output text corresponding to the input text based on the word or character. For the complete text generation model, the above-mentioned intermediate module is a decoder, the first module is an encoder, and the last module is a generator.
[0047] Figure 2 It is a structural schematic diagram of the current model provided by the present invention, such as Figure 2As shown in the figure, the modules of the current model include module1, module2, module3 and module4, input is the current input data of the current model, and output is the current output data of the current model. If the output cannot be aligned with the corresponding golden data, the golden data corresponding to the output data of module2 is inserted at position 3 (pos3), that is, module3 uses the golden data corresponding to the output data of module2 as input, and the model composed of module3 and module4 is used as the current model.
[0048] If the current output data of the current model composed of module3 and module4 is aligned with the corresponding golden data, it indicates that the accuracy problem exists in module1 and module2. At this time, the golden data corresponding to the output data of module1 can be inserted at position 2 (pos2) to determine whether there is an accuracy problem in module2. If not, it indicates that there is an accuracy problem in module1.
[0049] If the current output data of the current model composed of module3 and module4 is not aligned with the corresponding golden data, the golden data corresponding to the output data of module3 can be inserted at position 4 (pos4) to determine whether module4 has an accuracy problem. If not, it indicates that module3 has an accuracy problem.
[0050] Based on any of the above embodiments, if all modules between the first module and the middle module, the first module and the middle module are used as the current screening modules, then the current output data of the current model and the benchmark data corresponding to the current output data are returned to determine, and then the following is also included: If the current output data is aligned with the benchmark data, all modules between the middle module and the last module, the middle modules and the last module are used as current screening modules, and the current output data of the current model and the benchmark data corresponding to the current output data are returned for execution to be determined.
[0051] Specifically, if all modules between the first module and the middle module, the first module and the middle module are taken as the current screening modules, after returning to execute to determine the current output data of the current model and the benchmark data corresponding to the current output data, it can be determined whether there are accuracy problems in the first half of the modules.
[0052] If the current output data is aligned with the benchmark data, it means that there is no accuracy problem in the first half of the modules. All modules between the middle module and the last module, the middle modules and the last module are used as the current screening modules, and the execution is returned to determine the current output data of the current model and the benchmark data corresponding to the current output data to determine whether there is an accuracy problem in the second half of the modules.
[0053] It can be seen that the embodiment of the present invention reduces unnecessary module inspections by segmented troubleshooting and confirming alignment, and can locate problem areas more efficiently in a shorter time, while reducing unnecessary troubleshooting work and improving the efficiency of precision problem troubleshooting.
[0054] Based on any of the above embodiments, if all modules between the middle module and the last module, the middle module and the last module are used as the current screening modules, then the current output data of the current model and the benchmark data corresponding to the current output data are returned to determine, and then the following is also included: If the current output data is aligned with the benchmark data, the model consisting of all modules between the first module and the middle module, the first module and the middle module is used as the current screening module, and the current output data of the current model and the benchmark data corresponding to the current output data are returned for execution.
[0055] Specifically, if all modules between the middle module and the last module, the middle modules and the last module are taken as the current screening modules, after returning to execute to determine the current output data of the current model and the benchmark data corresponding to the current output data, it can be determined whether there are accuracy problems in the second half of the modules.
[0056] If the current output data is aligned with the benchmark data, it indicates that there is no accuracy problem in the second half of the module. At this time, the model composed of all modules between the first module and the middle module, the first module and the middle module is used as the current troubleshooting module, and the current output data of the current model and the benchmark data corresponding to the current output data are returned to determine whether there is an accuracy problem in the first half of the module.
[0057] It can be seen that the embodiment of the present invention reduces unnecessary module inspections by segmented troubleshooting and confirming alignment, and can locate problem areas more efficiently in a shorter time, while reducing unnecessary troubleshooting work and improving the efficiency of precision problem troubleshooting.
[0058] Based on any of the above embodiments, the method further includes: The parameters of the target module in the complete model are fixed, and the parameters of the problem module in the complete model are optimized. The target module refers to all modules in the complete model except the problem module.
[0059] Specifically, in the complete model, all modules except the problem modules are target modules, and these target modules do not have accuracy problems, so there is no need to optimize the parameters of the target modules, but only the parameters of the problem modules in the complete model need to be optimized.
[0060] In this regard, the embodiment of the present invention fixes the parameters of the target module in the complete model and optimizes the parameters of the problem module in the complete model, so that the parameters of the problem module can be adjusted centrally, the demand for computing resources can be reduced, and the complexity of comprehensive parameter adjustment of all modules of the complete model can be avoided. In addition, fixing the parameters of the target module can ensure that the parameters of the target module are not affected during the process of optimizing the parameters of the problem module in the complete model, thereby ensuring the stability of the target module.
[0061] Taking the text generation model as an example, if the complete text generation model includes an encoder, a decoder and a generator, and based on the above embodiment method, when the problem module is determined to be an encoder, the decoder and the generator can be used as target modules, and the parameters of the decoder and the generator are fixed, and the parameters of the encoder are optimized. After optimizing the encoder parameters, the performance of the complete text generation model can be improved, and the accuracy of the generated text can be guaranteed.
[0062] Based on any of the above embodiments, the parameters of the problem module in the complete model are optimized based on the difference between the output data of the problem module and the reference data corresponding to the output data.
[0063] Specifically, when optimizing the problem module parameters, the optimization may be performed with the goal of minimizing the difference between the output data of the problem module and the reference data corresponding to the output data.
[0064] Optionally, the difference between the output data of the problem module and the reference data corresponding to the output data may be measured based on the mean square error between the two, and the parameters of the problem module may be optimized with the goal of minimizing the mean square error.
[0065] Based on any of the above embodiments, the current output data is the output data corresponding to the current model when overfitting training is performed.
[0066] Considering that overfitting training focuses on accurately fitting the data in the training set and ignores the learning of new data, the current model can be quickly adjusted in a smaller batch of data to reduce training loss during overfitting training. As a result, the training time corresponding to overfitting training is usually less than the training time required by traditional training methods, and the accuracy problem can be reproduced after fewer training cycles, avoiding the problem of low efficiency caused by traditional training methods that require multiple training of the model to reproduce the accuracy problem.
[0067] Figure 3 It is a flow chart of another method for troubleshooting model accuracy problems provided by the present invention. Figure 3 As shown, the method includes: Perform an overfitting test on the complete model to obtain the current output data of the complete model. Determine whether the current output data is aligned with the golden data corresponding to the current output data. If not, use the interval formed by the input position and output position of the complete model as the current overfitting interval, and use the binary search method to determine the insertion position of the golden data within the current overfitting interval.
[0068] The current model is constructed with the second half of the modules at the insertion position (i.e., the insertion position is used as the starting point of the current model), and the current model is overfitted to determine the current output data of the current model. If the current output data is aligned with the corresponding golden data, the insertion position is used as the end point of the current overfitting interval, and the binary search is used to determine the insertion position of the golden data in the current overfitting interval, until the current overfitting interval cannot be divided (i.e., there is only one module in the current overfitting interval), and the module in the current overfitting interval is returned as the problem module. If the current output data is not aligned with the corresponding golden data, the insertion position is used as the starting point of the current overfitting interval, and the binary search is used to determine the insertion position of the golden data in the current overfitting interval, until the current overfitting interval cannot be divided.
[0069] The model accuracy problem troubleshooting device provided by the present invention is described below. The model accuracy problem troubleshooting device described below and the model accuracy problem troubleshooting method described above can be referenced to each other.
[0070] Based on any of the above embodiments, Figure 4 is a schematic diagram of the structure of the device for troubleshooting model accuracy problems provided by the present invention. Figure 4 As shown, the device comprises: A determination unit 410, configured to determine current output data of a current model and reference data corresponding to the current output data; A selection unit 420, configured to select some modules from a plurality of modules of the current model as current screening modules if the current output data is not aligned with the reference data; The screening unit 430 is used to use the model composed of the current screening module as the current model, and then return to determine the current output data of the current model and the reference data corresponding to the current output data, until the current model includes only one module; The positioning unit 440 is used to take a module of the current model as a problem module if the current output data is not aligned with the reference data and the current model includes only one module.
[0071] Based on any of the above embodiments, some modules are selected from multiple modules of the current model as current screening modules, including: Identify an intermediate module from among the multiple modules; All modules between the first module and the middle module, the first module and the middle module are used as current screening modules, or all modules between the middle module and the last module, the middle module and the last module are used as current screening modules.
[0072] Based on any of the above embodiments, if all modules between the first module and the middle module, the first module and the middle module are used as the current screening modules, then the current output data of the current model and the benchmark data corresponding to the current output data are returned to determine, and then the following is also included: If the current output data is aligned with the benchmark data, all modules between the middle module and the last module, the middle modules and the last module are used as current screening modules, and the current output data of the current model and the benchmark data corresponding to the current output data are returned for execution to be determined.
[0073] Based on any of the above embodiments, if all modules between the middle module and the last module, the middle module and the last module are used as the current screening modules, then the current output data of the current model and the benchmark data corresponding to the current output data are returned to determine, and then the following is also included: If the current output data is aligned with the benchmark data, the model consisting of all modules between the first module and the middle module, the first module and the middle module is used as the current screening module, and the current output data of the current model and the benchmark data corresponding to the current output data are returned for execution.
[0074] Based on any of the above embodiments, it also includes: The parameters of the target module in the complete model are fixed, and the parameters of the problem module in the complete model are optimized. The target module refers to all modules in the complete model except the problem module.
[0075] Based on any of the above embodiments, the parameters of the problem module in the complete model are optimized based on the difference between the output data of the problem module and the reference data corresponding to the output data.
[0076] Based on any of the above embodiments, the current output data is the output data corresponding to the current model when overfitting training is performed.
[0077] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the model accuracy problem troubleshooting method, which includes: determining the current output data of the current model and the benchmark data corresponding to the current output data; if the current output data is not aligned with the benchmark data, selecting some modules from multiple modules of the current model as the current troubleshooting modules; after taking the model composed of the current troubleshooting modules as the current model, returning to execute the determination of the current output data of the current model and the benchmark data corresponding to the current output data, until the current model includes only one module; if the current output data is not aligned with the benchmark data, and the current model includes only one module, taking one module of the current model as the problem module.
[0078] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0079] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the model accuracy problem troubleshooting method provided by the above methods, the method includes: determining the current output data of the current model and the benchmark data corresponding to the current output data; if the current output data is not aligned with the benchmark data, then selecting some modules from multiple modules of the current model as current troubleshooting modules; after taking the model composed of the current troubleshooting modules as the current model, returning to execute the determination of the current output data of the current model and the benchmark data corresponding to the current output data until the current model includes only one module; if the current output data is not aligned with the benchmark data, and the current model includes only one module, then taking one module of the current model as a problem module. .
[0080] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by the processor to execute the model accuracy problem troubleshooting method provided by the above methods, the method comprising: determining the current output data of the current model and the benchmark data corresponding to the current output data; if the current output data is not aligned with the benchmark data, selecting some modules from multiple modules of the current model as current troubleshooting modules; after taking the model composed of the current troubleshooting modules as the current model, returning to the determination of the current output data of the current model and the benchmark data corresponding to the current output data, until the current model includes only one module; if the current output data is not aligned with the benchmark data, and the current model includes only one module, taking one module of the current model as a problem module. .
[0081] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.
[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for troubleshooting model accuracy issues. It is characterized in that include: Determine current output data of the current model and reference data corresponding to the current output data; If the current output data is not aligned with the reference data, some modules are selected from the multiple modules of the current model as current screening modules; After taking the model formed by the current screening modules as the current model, returning to the step of determining the current output data of the current model and the benchmark data corresponding to the current output data until the current model includes only one module; If the current output data is not aligned with the reference data, and the current model includes only one module, one module of the current model is used as a problem module.
2. The method for troubleshooting model accuracy problems according to claim 1, It is characterized in that The selecting some modules from the multiple modules of the current model as current screening modules includes: determining an intermediate module from among the plurality of modules; All modules between the first module and the middle module, the first module and the middle module are used as the current screening modules, or all modules between the middle module and the last module, the middle module and the last module are used as the current screening modules.
3. The method for troubleshooting model accuracy problems according to claim 2, It is characterized in that If all modules between the first module and the middle module, the first module and the middle module are used as the current screening module, the current output data of the current model determined and the benchmark data corresponding to the current output data are returned, and then the following is further included: If the current output data is aligned with the benchmark data, all modules between the intermediate module and the last module, the intermediate module and the last module are used as the current screening modules, and the current output data of the current model determined and the benchmark data corresponding to the current output data are returned.
4. The method for troubleshooting model accuracy problems according to claim 2, It is characterized in that If all modules between the intermediate module and the last module, the intermediate module and the last module are used as the current screening module, the current output data of the current model determined and the benchmark data corresponding to the current output data are returned, and then the following is further included: If the current output data is aligned with the benchmark data, the model composed of all modules between the first module and the intermediate module, the first module and the intermediate modules is used as the current screening module, and the current output data for determining the current model and the benchmark data corresponding to the current output data are returned.
5. The method for troubleshooting model accuracy problems according to any one of claims 1 to 4, It is characterized in that The method further comprises: The parameters of the target module in the complete model are fixed, and the parameters of the problem module in the complete model are optimized, wherein the target module refers to all modules in the complete model except the problem module.
6. The method for troubleshooting model accuracy problems according to claim 5, It is characterized in that The parameters of the problem module in the complete model are optimized based on the difference between the output data of the problem module and the reference data corresponding to the output data.
7. The method for troubleshooting model accuracy problems according to any one of claims 1 to 4, It is characterized in that The current output data is the output data corresponding to the current model when overfitting training is performed.
8. A device for troubleshooting model accuracy problems, It is characterized in that include: A determination unit, used to determine current output data of a current model and reference data corresponding to the current output data; A selection unit, configured to select some modules from a plurality of modules of the current model as current screening modules if the current output data is not aligned with the reference data; A screening unit, configured to use the model formed by the current screening module as the current model, and then return to execute the process of determining the current output data of the current model and the reference data corresponding to the current output data, until the current model includes only one module; A positioning unit is used to use a module of the current model as a problem module if the current output data is not aligned with the reference data and the current model includes only one module.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method for troubleshooting model accuracy problems as described in any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the model accuracy problem troubleshooting method as described in any one of claims 1 to 7 is implemented.
11. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the model accuracy problem troubleshooting method as described in any one of claims 1 to 7 is implemented.