Incremental data determination method and device, storage medium, and electronic device
By analyzing the training and verification losses of incremental data during the model training process, combined with the similarity of labeled results, the data set pollution problem caused by noise data in incremental data is solved, ensuring the data set quality and model performance.
CN113486986BActive Publication Date: 2024-12-13ZHEJIANG DAHUA TECH CO LTD
Patent Information
- Application Number
- CN202110883213.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-02
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2041-08-02
Smart Images
Figure CN113486986B_ABST
Abstract
The present invention discloses a method and device for determining incremental data, a storage medium, and an electronic device, wherein the method comprises: in the process of training a first preset model through incremental data, obtaining a first loss result corresponding to the incremental data, the incremental data comprising: training data and verification data, and the first loss result comprising: training loss and verification loss; when the training loss and the verification loss meet a first preset condition, obtaining an analysis result corresponding to the incremental data through the first preset model; determining the similarity between the analysis result and the labeling result corresponding to the incremental data; when the similarity is less than a first preset threshold, determining that the incremental data is unqualified data. The above scheme is adopted to solve the problem in the related art that when the model needs to enrich the data set, if there is noise data in the incremental data, it will cause data set pollution and other problems.
Need to check novelty before this filing date? Find Prior Art
Citation Information
Patent Citations
Image data compression method and device, electronic equipment and storage medium
CN111768457A
Sample set processing method and device, equipment, and storage medium
CN112036492A
Sample screening method, sample screening device and terminal equipment
CN112966754A
Design method, optimization method and annotation method of automatic text classification annotation system based on big data
CN113064993A
Testing of wire systems and end devices installed in industrial processes
US20050182581A1