Dialect sample data extraction method, device and equipment and storage medium

Abstract

The invention discloses a dialect sample data extraction method, and the method comprises the steps: obtaining the first dialect of a plurality of dialect regions and the city data corresponding to each dialect region in the plurality of dialect regions, wherein one dialect region corresponds to one city; classifying the dialect regions with the same first dialect into the same dialect group, andobtaining a plurality of dialect groups; sorting each dialect group according to the city data corresponding to each dialect region, and determining a target dialect region of each dialect group fromeach sorted dialect group; acquiring medical and aesthetic dialogue data of a city corresponding to the target dialect area of each dialect group; and taking the obtained medical and aesthetic dialogue data corresponding to each dialect group as dialect sample data. Therefore, theoretically, the data needs to cover all official areas in terms of machine learning data selection, and the generalization ability of the model can be enhanced.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a dialect sample data extraction method, device, equipment and storage medium. Background technique [0002] In the field of natural language processing and task-based dialogue robots, dialects are often a headache for robots, because there are roughly 7 official dialects in China, and there are hundreds of dialects. The mainstream natural language processing is Chinese, that is, in non-official areas, there may be errors in task recognition, such as intent recognition. For example, people in most places will say "how much money" when asking about the price, while in some dialect areas they will say "how much money". However, the lack of such dialect data in the training sample data makes the dialect comparison robot unable to recognize the dialect, so that the dialogue robot cannot accurately answer the user. Contents of the invention [0003] The present inv...

AI Technical Summary

Problems solved by technology

However, there is a lack of such dialect data in the training sample data, which makes the dialect comparison robot unable to recognize the dialect, so that the dialogue robot cannot accurately answer the user's question.

Images