Speech semantic segmentation method and device, electronic equipment and readable medium

Abstract

The application relates to a script semantic segmentation method and device, electronic equipment and readable medium, wherein the method comprises: performing semantic segmentation on first script data and second script data respectively by using a first semantic segmentation model, and generating a first data set and a second data set according to the segmentation results, wherein the first data set and the second data set each comprise sample data of different difficulty levels; integrating the first data set and the second data set according to the difficulty levels to obtain a third data set; generating training sets of different training stages according to the third data set; performing stage-by-stage optimization training on a second semantic segmentation model by using the training sets to obtain a target semantic segmentation model; and performing semantic segmentation on target script data by using the target semantic segmentation model to obtain a semantic segmentation result. The problem that a large number of training samples are required to improve the accuracy of script semantic segmentation is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and readable medium for semantic segmentation of dialogue. Background Technology

[0002] With the rapid development of internet technology and the widespread adoption of smart devices, the reach of videos and short videos is increasingly broad. In the fields of video content understanding and short video re-creation, accurate segmentation of semantically complete dialogue segments is crucial.

[0003] Currently, methods for dialogue semantic segmentation can be mainly divided into two categories: one category utilizes small models trained on specialized datasets, but due to limitations in training data and model capacity, these small models lack accuracy in dialogue semantic segmentation; the other category employs large models trained on general datasets. These large models, having undergone extensive training on large amounts of data, possess stronger understanding and generalization capabilities. However, due to their higher complexity, their outputs often exhibit a degree of randomness, and they require a large number of training samples, thus placing greater demands on computational resources. Therefore, the need for a large number of training samples to improve the accuracy of dialogue semantic segmentation has become a key problem that needs to be solved in the field of semantic segmentation.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and readable medium for semantic segmentation of dialogue, in order to solve the aforementioned technical problem that "a large number of training samples are needed to improve the accuracy of semantic segmentation of dialogue".

[0006] According to one aspect of the embodiments of this application, this application provides a semantic segmentation method for dialogue, comprising: performing semantic segmentation on first dialogue data and second dialogue data using a first semantic segmentation model, and generating a first dataset and a second dataset based on the segmentation results, wherein both the first dataset and the second dataset include sample data of different difficulty levels; integrating the first dataset and the second dataset according to the difficulty level to obtain a third dataset; generating training sets for different training stages based on the third dataset; performing phased optimization training on the second semantic segmentation model using the training sets to obtain a target semantic segmentation model; and performing semantic segmentation on the target dialogue data using the target semantic segmentation model to obtain a semantic segmentation result.

[0007] Optionally, the first semantic segmentation model is used to perform semantic segmentation on the first dialogue data and the second dialogue data respectively, and the first dataset and the second dataset are generated based on the segmentation results, including: using the first semantic segmentation model to perform semantic segmentation on the first dialogue data to obtain the first segmentation result, and dividing the first segmentation result into a first dataset of different difficulty levels; using the first semantic segmentation model to perform semantic segmentation on the second dialogue data to obtain the second segmentation result, and dividing the second segmentation result into a second dataset of different difficulty levels.

[0008] Optionally, the first segmentation result is divided into a first dataset with different difficulty levels, including: validating the first segmentation result to obtain first samples that pass the validation and second samples that fail the validation, and correcting the second samples that fail the validation; extracting the confidence scores of each sample in the first and second samples, and comparing each confidence score with the target confidence score; determining the samples with confidence scores greater than or equal to the target confidence score as first easy samples, and determining the samples with confidence scores less than the target confidence score and belonging to the first samples as first medium samples, and determining the samples with confidence scores less than the target confidence score and belonging to the second samples as first difficult samples; and determining the first easy samples, first medium samples, and first difficult samples as the first dataset.

[0009] Optionally, the second segmentation result is divided into second datasets with different difficulty levels, including: extracting the confidence score of each sample in the second dataset and comparing each confidence score with the target confidence score; determining the samples with a confidence score greater than or equal to the target confidence score as second simple samples; verifying the samples with a confidence score less than the target confidence score to obtain second medium samples that pass the verification and second difficult samples that fail the verification; and determining the second simple samples, second medium samples, and second difficult samples as the second dataset.

[0010] Optionally, the first dataset and the second dataset are integrated according to the difficulty level to obtain a third dataset, including: calculating the ratio of the number of samples with medium difficulty to hard difficulty in the first dataset to obtain a first ratio, and calculating the ratio of the number of samples with medium difficulty to hard difficulty in the second dataset to obtain a second ratio; calculating the ratio difference between the first ratio and the second ratio; if the ratio difference is less than or equal to the target difference, then the first dataset and the second dataset are integrated into the third dataset.

[0011] Optionally, training sets for different training stages are generated based on the third dataset, including: transforming the third dataset according to the training instruction structure corresponding to the second semantic segmentation model to obtain a training instruction set; dividing the training instruction set into a first instruction set corresponding to the early stage of training, a second instruction set corresponding to the middle stage of training, and a third instruction set corresponding to the late stage of training according to the difficulty level, wherein the first instruction set has the highest proportion of simple difficulty instruction samples, the second instruction set has the highest proportion of medium difficulty instruction samples, and the third instruction set has the highest proportion of hard difficulty instruction samples.

[0012] Optionally, the second semantic segmentation model is fine-tuned and trained in stages using the training set to obtain the target semantic segmentation model, including: using the first instruction set to train the second semantic segmentation model using LORA to obtain the third semantic segmentation model; using the second instruction set to train the third semantic segmentation model using LORA to obtain the fourth semantic segmentation model; and using the third instruction set to train the fourth semantic segmentation model using LORA to obtain the target semantic segmentation model.

[0013] According to another aspect of the embodiments of this application, this application provides a semantic segmentation apparatus for dialogue, comprising: a first segmentation module, configured to perform semantic segmentation on first dialogue data and second dialogue data respectively using a first semantic segmentation model, and generate a first dataset and a second dataset based on the segmentation results, wherein the first dataset and the second dataset both include sample data of different difficulty levels; an integration module, configured to integrate the first dataset and the second dataset according to the difficulty level to obtain a third dataset; a generation module, configured to generate training sets for different training stages based on the third dataset; a training module, configured to perform phased optimization training on the second semantic segmentation model using the training sets to obtain a target semantic segmentation model; and a second segmentation module, configured to perform semantic segmentation on target dialogue data using the target semantic segmentation model to obtain semantic segmentation results.

[0014] According to another aspect of the embodiments of this application, this application provides an electronic device, including a memory, a processor, a communication interface and a communication bus. The memory stores a computer program that can run on the processor. The memory and the processor communicate with each other through the communication bus and the communication interface. When the processor executes the computer program, it implements the steps of the above method.

[0015] According to another aspect of the embodiments of this application, this application also provides a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the above-described method.

[0016] Compared with related technologies, the technical solutions provided in this application have the following advantages:

[0017] This application provides a semantic segmentation method for dialogue, comprising: using a first semantic segmentation model to perform semantic segmentation on first and second dialogue data respectively, and generating a first dataset and a second dataset based on the segmentation results, wherein both the first and second datasets include sample data of different difficulty levels; integrating the first and second datasets according to the difficulty level to obtain a third dataset; generating training sets for different training stages based on the third dataset; using the training sets to perform staged optimization training on the second semantic segmentation model to obtain a target semantic segmentation model; and using the target semantic segmentation model to perform semantic segmentation on the target dialogue data to obtain semantic segmentation results. By first dividing the semantically segmented sample data according to different difficulty levels, then using sample data of different difficulty levels to perform staged optimization training on the semantic segmentation model to obtain an optimized semantic segmentation model, and finally using the optimized semantic segmentation model to perform semantic segmentation of dialogue, this method solves the problem of needing a large number of training samples to improve the accuracy of dialogue semantic segmentation. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of an optional semantic segmentation method for dialogue provided according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating the generation of an optional target semantic segmentation model according to an embodiment of this application;

[0022] Figure 3 This is a block diagram of an optional semantic segmentation device for dialogue provided according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an optional electronic device structure provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0026] With the rapid development of internet technology and the widespread adoption of smart devices, the reach of videos and short videos is increasingly broad. In the fields of video content understanding and short video re-creation, accurate segmentation of semantically complete dialogue segments is crucial.

[0027] Currently, methods for dialogue semantic segmentation can be mainly divided into two categories: one category utilizes small models trained on specialized datasets, but due to limitations in training data and model capacity, these small models lack accuracy in dialogue semantic segmentation; the other category employs large models trained on general datasets. These large models, having undergone extensive training on large amounts of data, possess stronger understanding and generalization capabilities. However, due to their higher complexity, their outputs often exhibit a degree of randomness, and they require a large number of training samples, thus placing greater demands on computational resources. Therefore, the need for a large number of training samples to improve the accuracy of dialogue semantic segmentation has become a key problem that needs to be solved in the field of semantic segmentation.

[0028] To address the problems mentioned in the background art, according to one aspect of the embodiments of this application, an embodiment of a semantic segmentation method for dialogue is provided, such as... Figure 1 As shown, it includes:

[0029] Step 101: Use the first semantic segmentation model to perform semantic segmentation on the first dialogue data and the second dialogue data respectively, and generate the first dataset and the second dataset based on the segmentation results. The first dataset and the second dataset both include sample data of different difficulty levels.

[0030] Step 103: Integrate the first and second datasets according to their difficulty levels to obtain the third dataset;

[0031] Step 105: Generate training sets for different training stages based on the third dataset;

[0032] Step 107: Use the training set to perform phased optimization training on the second semantic segmentation model to obtain the target semantic segmentation model;

[0033] Step 109: Use the target semantic segmentation model to perform semantic segmentation on the target dialogue data to obtain the semantic segmentation results.

[0034] This application solves the problem of needing a large number of training samples to improve the accuracy of semantic segmentation by first dividing the semantically segmented sample data into different difficulty levels, then using sample data of different difficulty levels to perform phased optimization training on the semantic segmentation model to obtain an optimized semantic segmentation model, and finally using the optimized semantic segmentation model to perform semantic segmentation of dialogue.

[0035] The first semantic segmentation model is a small model trained on a dedicated dataset, while the second semantic segmentation model is a lightweight large language model trained on a general dataset.

[0036] Compared to the lightweight large model before training, the target semantic segmentation model significantly improves the format standardization and accuracy of the segmentation results. Compared to the small model segmentation method, the target semantic segmentation model significantly reduces the amount of data required for model optimization and the training cycle, while improving the segmentation effect.

[0037] As an optional embodiment, the first semantic segmentation model is used to perform semantic segmentation on the first dialogue data and the second dialogue data respectively, and the first dataset and the second dataset are generated based on the segmentation results. This includes: performing semantic segmentation on the first dialogue data using the first semantic segmentation model to obtain a first segmentation result, and dividing the first segmentation result into a first dataset with different difficulty levels; performing semantic segmentation on the second dialogue data using the first semantic segmentation model to obtain a second segmentation result, and dividing the second segmentation result into a second dataset with different difficulty levels.

[0038] The first and second dialogue data are different. The second dialogue data is obtained by expanding the scope of video dialogue data collection based on the first dialogue data.

[0039] Using an existing semantic segmentation mini-model (i.e., the first semantic segmentation model) trained on a general or special dataset, semantic segmentation is performed on a small amount of video dialogue data to generate corresponding dialogue segmentation results (i.e., segmentation results).

[0040] The methods for obtaining the first and second datasets are largely the same. The differences lie in the different dialogue data for the first and second datasets, and also in the fact that when obtaining the first dataset, the samples are validated before being categorized into difficulty levels, while when obtaining the second dataset, simpler samples are selected first, then validated, and samples for other difficulty levels are obtained. The following sections will explain in detail how to obtain the first and second datasets.

[0041] As an optional embodiment, the first segmentation result is divided into a first dataset with different difficulty levels, including: verifying the first segmentation result to obtain a first sample that passes the verification and a second sample that fails the verification, and correcting the second sample that fails the verification; extracting the confidence score of each sample in the first sample and the second sample, and comparing each confidence score with a target confidence score; determining the samples with a confidence score greater than or equal to the target confidence score as first easy samples, and determining the samples with a confidence score less than the target confidence score and belonging to the first sample as first medium samples, and determining the samples with a confidence score less than the target confidence score and belonging to the second sample as first hard samples; and determining the first easy samples, first medium samples, and first hard samples as the first dataset.

[0042] The segmentation results generated by the first semantic segmentation model are verified using a verification mechanism (such as manual inspection, rule matching, or other pre-trained models) to identify which segmentation results are accurate (i.e., "the first sample that passed the verification") and which are inaccurate (i.e., "the second sample that failed the verification").

[0043] For example, the first segmentation result can be manually verified to correct any erroneous samples. Samples that fail verification are then corrected. Corrections may include re-segmentation, adjusting segmentation boundaries, and changing annotations. This improves the accuracy of the samples, making them closer to the true semantic segmentation result.

[0044] The segmentation results will include a confidence score. The confidence score of each sample will be compared with the target confidence score to perform difficulty classification and filtering of the dialogue data.

[0045] Confidence level is the degree of confidence a model has in its segmentation results, usually expressed as a probability value or score.

[0046] The target confidence level can be set or adjusted according to actual circumstances, such as ensuring that the accuracy of samples with a confidence level greater than the target confidence level reaches a set value. This application does not limit the specific value of the target confidence level.

[0047] The first simple sample can be understood as a sample with a relatively simple difficulty level that the semantic segmentation model can accurately segment, and the confidence level of these samples is greater than or equal to the target confidence level.

[0048] The first intermediate sample can be understood as a sample with a medium level of difficulty that the semantic segmentation model can accurately segment. The confidence of the first intermediate sample is lower than the target confidence, but it belongs to the first sample that passes the verification.

[0049] The first difficult sample can be understood as a sample that the semantic segmentation model cannot accurately segment, and the confidence level of the first difficult sample is lower than the target confidence level. It belongs to the second sample that failed the verification (although it has been corrected).

[0050] As an optional embodiment, the second segmentation result is divided into a second dataset with different difficulty levels, including: extracting the confidence score of each sample in the second dataset and comparing each confidence score with the target confidence score; determining the samples with a confidence score greater than or equal to the target confidence score as second simple samples; verifying the samples with a confidence score less than the target confidence score to obtain second medium samples that pass the verification and second difficult samples that fail the verification; and determining the second simple samples, second medium samples, and second difficult samples as the second dataset.

[0051] First, identify the samples with confidence levels greater than or equal to the target confidence level as the second simplest samples, and then validate the samples with confidence levels less than the target confidence level. This can reduce the workload of validation to some extent.

[0052] Samples with confidence levels greater than or equal to the target confidence level are directly identified as the second simple samples. Then, samples with confidence levels less than the target confidence level are validated. Based on the validation results, the second medium samples and the second difficult samples are determined. Among them, those that pass the validation are the second medium samples, and those that fail the validation are the second difficult samples.

[0053] The second simple sample, the second medium sample, and the second difficult sample are combined to form a second dataset containing different difficulty levels.

[0054] By constructing datasets containing different levels of difficulty, it is helpful to improve the generalization ability and robustness of semantic segmentation models in terms of training and optimization.

[0055] As an optional embodiment, the first dataset and the second dataset are integrated according to the difficulty level to obtain a third dataset, including: calculating the ratio of the number of samples with medium difficulty to hard difficulty in the first dataset to obtain a first ratio, and calculating the ratio of the number of samples with medium difficulty to hard difficulty in the second dataset to obtain a second ratio; calculating the ratio difference between the first ratio and the second ratio; if the ratio difference is less than or equal to the target difference, then the first dataset and the second dataset are integrated into the third dataset.

[0056] Calculate the difference between the first ratio and the second ratio to obtain the ratio difference. The ratio difference can reflect the degree of difference between the distribution of medium and hard difficulty samples in the two datasets. Therefore, it is necessary to ensure that the difference between the distribution of medium and hard difficulty samples in the two datasets is small, that is, the ratio difference is less than or equal to the target difference.

[0057] If the ratio difference is greater than the target difference, it means that the first ratio and the second ratio cannot be maintained at the same level. In this case, it is necessary to adjust the difficulty level classification of each sample in the second dataset. For example, set stricter filtering conditions (e.g., increase the target confidence level), classify the samples in the second dataset according to the new target confidence level, and calculate the ratio of the number of samples with medium difficulty to difficult difficulty in the second dataset (i.e., the new second ratio). Then, calculate the new ratio difference between the first ratio and the new second ratio until the new ratio difference is less than or equal to the target difference.

[0058] The target difference can be set according to the actual situation, and this application does not limit the specific value of the target difference.

[0059] The first and second datasets were integrated according to their difficulty levels to obtain a third dataset that includes a wider range of difficulty levels, providing a data foundation for subsequent model training.

[0060] As an optional embodiment, training sets for different training stages are generated based on the third dataset, including: transforming the third dataset according to the training instruction structure corresponding to the second semantic segmentation model to obtain a training instruction set; dividing the training instruction set into a first instruction set corresponding to the early stage of training, a second instruction set corresponding to the middle stage of training, and a third instruction set corresponding to the late stage of training according to the difficulty level, wherein the first instruction set has the highest proportion of simple difficulty instruction samples, the second instruction set has the highest proportion of medium difficulty instruction samples, and the third instruction set has the highest proportion of hard difficulty instruction samples.

[0061] The third dataset is transformed according to the training instruction structure corresponding to the second semantic segmentation model. For example, the samples in the dataset are converted into the format required for model training, including input data, labels, and possible additional information (such as weights, masks, etc.). The transformed dataset is called the training instruction set, which contains all the instructions required for model training.

[0062] The training instruction set is divided into three subsets based on difficulty level: the first set, the second set, and the third set. Each subset corresponds to a different stage of training: the first set corresponds to the early stage of training, the second set to the middle stage, and the third set to the late stage.

[0063] When dividing the data, it is necessary to ensure that the proportion of instruction difficulty levels in each subset meets the set requirements. Specifically, the proportion of easy-difficulty instructions is highest in the first subset, the proportion of medium-difficulty instructions is highest in the second subset, and the proportion of difficult-difficulty instructions is highest in the third subset. Reducing the proportion of easy samples in the later stages of training can improve the model's recognition performance on difficult examples.

[0064] The third dataset is used to generate training sets suitable for different training stages, thereby training the second semantic segmentation model more effectively.

[0065] This application adopts a batch-based, multi-round training method to adjust the proportion of difficult examples in the dialogue segmentation instruction set, reduce the difficulty of model convergence, and ensure the effect of supervised training.

[0066] This application utilizes an existing semantic segmentation mini-model (i.e., the first semantic segmentation model) to generate an initial version of the dialogue semantic segmentation results. Then, it filters simple samples based on confidence levels, and submits low-confidence erroneous samples to manual verification and modification. Finally, it integrates samples of different difficulty levels to construct a training set for dialogue segmentation training.

[0067] As an optional embodiment, the second semantic segmentation model is trained in stages using the training set to obtain the target semantic segmentation model, including: training the second semantic segmentation model using LORA with the first instruction set to obtain the third semantic segmentation model; training the third semantic segmentation model using LORA with the second instruction set to obtain the fourth semantic segmentation model; and training the fourth semantic segmentation model using LORA with the third instruction set to obtain the target semantic segmentation model.

[0068] The optimization training provided in this application is divided into three stages, using instruction sets corresponding to different stages as training data in sequence, and using the training results of the previous stage as the pre-trained model. The optimization training in each stage is implemented using the LoRA (Low-Rank Adaptation of Large Language Models) training method, which means that only some parameter weights are updated, which can effectively reduce the resource requirements during the training process.

[0069] It should be noted that the number of training phases can be adjusted according to actual needs. The three phases provided in this application are only for reference as the optimal embodiment.

[0070] This application employs the LoRA training method, combining training sets of varying difficulty levels to perform supervised tuning training on a lightweight language model. The core idea of ​​LoRA is to replace full weight updates with a low-rank approximation. Specifically, it freezes the original weights of the pre-trained model and introduces a low-rank matrix ΔW into the weight matrix of certain specific layers. This matrix can be represented as the product of two smaller trainable matrices A and B (i.e., ΔW = A·B). During training, only these two smaller matrices A and B need to be optimized, without updating the weights of the entire model.

[0071] Figure 2 The diagram illustrates the generation of the target semantic segmentation model provided in this application. As shown in the figure, a small semantic segmentation model is used to identify video dialogue data. Then, semantic segmentation instruction sets (including dialogue semantic segmentation instruction sets for stage 1, stage 2, and stage 3) are obtained through semantic paragraph reasoning, manual verification, and automatic hard example filtering. The semantic segmentation instruction sets are then used to fine-tune the lightweight language model in stages to obtain the dialogue semantic segmentation model (i.e., the target semantic segmentation model of this application).

[0072] After obtaining the target semantic segmentation model, the video dialogue sequences in the target dialogue data can be processed. The target semantic segmentation model can then output the sentence segmentation type corresponding to the dialogue sequence. Combining dialogue sequences that do not require sentence segmentation forms a dialogue semantic segment; dividing long video dialogue data into several dialogue semantic segments achieves the effect of automatic dialogue semantic segmentation.

[0073] This application provides a semantic segmentation method for dialogue, comprising: using a first semantic segmentation model to perform semantic segmentation on first and second dialogue data respectively, and generating a first dataset and a second dataset based on the segmentation results, wherein both the first and second datasets include sample data of different difficulty levels; integrating the first and second datasets according to the difficulty level to obtain a third dataset; generating training sets for different training stages based on the third dataset; using the training sets to perform staged optimization training on the second semantic segmentation model to obtain a target semantic segmentation model; and using the target semantic segmentation model to perform semantic segmentation on the target dialogue data to obtain semantic segmentation results. By first dividing the semantically segmented sample data according to different difficulty levels, then using sample data of different difficulty levels to perform staged optimization training on the semantic segmentation model to obtain an optimized semantic segmentation model, and finally using the optimized semantic segmentation model to perform semantic segmentation of dialogue, this method solves the problem of needing a large number of training samples to improve the accuracy of dialogue semantic segmentation.

[0074] According to another aspect of the embodiments of this application, this application provides a semantic segmentation device for dialogue, such as... Figure 3 As shown, it includes:

[0075] The first segmentation module 302 is used to perform semantic segmentation on the first dialogue data and the second dialogue data using the first semantic segmentation model, and generate a first dataset and a second dataset based on the segmentation results. The first dataset and the second dataset both include sample data of different difficulty levels.

[0076] Integration module 304 is used to integrate the first dataset and the second dataset according to the difficulty level to obtain the third dataset;

[0077] The generation module 306 is used to generate training sets for different training stages based on the third dataset;

[0078] Training module 308 is used to perform phased optimization training on the second semantic segmentation model using the training set to obtain the target semantic segmentation model;

[0079] The second segmentation module 310 is used to perform semantic segmentation on the target dialogue data using the target semantic segmentation model to obtain semantic segmentation results.

[0080] It should be noted that the first segmentation module 302 in this embodiment can be used to execute step 101 in this application embodiment, the integration module 304 in this embodiment can be used to execute step 103 in this application embodiment, the generation module 306 in this embodiment can be used to execute step 105 in this application embodiment, the training module 308 in this embodiment can be used to execute step 107 in this application embodiment, and the second segmentation module 310 in this embodiment can be used to execute step 109 in this application embodiment.

[0081] Optionally, the first segmentation module 302 includes:

[0082] The first segmentation submodule is used to perform semantic segmentation on the first dialogue data using the first semantic segmentation model, obtain the first segmentation result, and divide the first segmentation result into the first dataset with different difficulty levels.

[0083] The second segmentation submodule is used to perform semantic segmentation on the second dialogue data using the first semantic segmentation model, obtain the second segmentation result, and divide the second segmentation result into a second dataset with different difficulty levels.

[0084] Optionally, the first segmentation submodule is further configured to verify the first segmentation result, obtain a first sample that passes the verification and a second sample that fails the verification, and correct the second sample that fails the verification; extract the confidence scores of each sample in the first sample and the second sample, and compare each confidence score with the target confidence score; determine the samples with confidence scores greater than or equal to the target confidence score as the first simple samples, and determine the samples with confidence scores less than the target confidence score and belonging to the first sample as the first medium samples, and determine the samples with confidence scores less than the target confidence score and belonging to the second sample as the first hard samples; and determine the first simple samples, the first medium samples, and the first hard samples as the first dataset.

[0085] Optionally, the second segmentation submodule is further configured to extract the confidence level of each sample in the second dataset and compare each confidence level with the target confidence level; determine the samples with confidence levels greater than or equal to the target confidence level as the second simple samples; validate the samples with confidence levels less than the target confidence level to obtain the second medium samples that pass the validation and the second difficult samples that fail the validation; and define the second simple samples, the second medium samples, and the second difficult samples as the second dataset.

[0086] Optionally, the integration module 304 is also used to calculate the ratio of the number of samples with difficulty level of medium to hard in the first dataset to obtain a first ratio, and to calculate the ratio of the number of samples with difficulty level of medium to hard in the second dataset to obtain a second ratio; calculate the ratio difference between the first ratio and the second ratio; if the ratio difference is less than or equal to the target difference, then integrate the first dataset and the second dataset into a third dataset.

[0087] Optionally, the generation module 306 is further configured to transform the third dataset according to the training instruction structure corresponding to the second semantic segmentation model to obtain a training instruction set; and divide the training instruction set into a first instruction set corresponding to the early stage of training, a second instruction set corresponding to the middle stage of training, and a third instruction set corresponding to the late stage of training according to the difficulty level. Among them, the first instruction set has the highest proportion of simple difficulty instruction samples, the second instruction set has the highest proportion of medium difficulty instruction samples, and the third instruction set has the highest proportion of hard difficulty instruction samples.

[0088] Optionally, the training module 308 is further configured to perform LORA training on the second semantic segmentation model using the first instruction set to obtain the third semantic segmentation model; perform LORA training on the third semantic segmentation model using the second instruction set to obtain the fourth semantic segmentation model; and perform LORA training on the fourth semantic segmentation model using the third instruction set to obtain the target semantic segmentation model.

[0089] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.

[0090] According to another aspect of the embodiments of this application, this application provides an electronic device, such as... Figure 4 As shown, the system includes a memory 401, a processor 403, a communication interface 405, and a communication bus 407. The memory 401 stores a computer program that can run on the processor 403. The memory 401 and the processor 403 communicate through the communication interface 405 and the communication bus 407. When the processor 403 executes the computer program, it implements the steps of the above method.

[0091] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0092] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0093] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0094] According to another aspect of the embodiments of this application, a computer-readable medium having processor-executable non-volatile program code is also provided.

[0095] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0096] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0097] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0098] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0099] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0101] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0104] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A semantic segmentation method for dialogue, characterized in that, include: The first semantic segmentation model is used to perform semantic segmentation on the first dialogue data and the second dialogue data respectively, and the first dataset and the second dataset are generated based on the segmentation results. The first dataset and the second dataset both include sample data with different difficulty levels. The different difficulty levels are obtained by a dual mechanism of confidence verification and manual verification. The first and second datasets are integrated according to the difficulty level to obtain the third dataset; Training sets for different training stages are generated based on the third dataset; The second semantic segmentation model is trained and optimized in stages using the training set to obtain the target semantic segmentation model. The target semantic segmentation model is used to perform semantic segmentation on the target dialogue data to obtain semantic segmentation results; The process of integrating the first and second datasets according to the difficulty level to obtain a third dataset includes: Calculate the ratio of the number of samples with difficulty level of medium to hard in the first dataset to obtain a first ratio, and calculate the ratio of the number of samples with difficulty level of medium to hard in the second dataset to obtain a second ratio. Calculate the ratio difference between the first ratio and the second ratio; If the ratio difference is less than or equal to the target difference, then the first dataset and the second dataset are integrated into a third dataset.

2. The method according to claim 1, characterized in that, The process of using a first semantic segmentation model to perform semantic segmentation on the first dialogue data and the second dialogue data, and generating a first dataset and a second dataset based on the segmentation results, includes: The first semantic segmentation model is used to perform semantic segmentation on the first dialogue data to obtain a first segmentation result, and the first segmentation result is divided into the first dataset with different difficulty levels. The first semantic segmentation model is used to perform semantic segmentation on the second dialogue data to obtain a second segmentation result, and the second segmentation result is divided into a second dataset with different difficulty levels.

3. The method according to claim 2, characterized in that, The step of dividing the first segmentation result into the first dataset with different difficulty levels includes: The first segmentation result is verified to obtain a first sample that passes the verification and a second sample that fails the verification, and the second sample that fails the verification is corrected. Extract the confidence scores of each sample in the first sample and the second sample, and compare each confidence score with the target confidence score; Samples with a confidence level greater than or equal to the target confidence level are defined as first easy samples, samples with a confidence level less than the target confidence level and belonging to the first sample are defined as first medium samples, and samples with a confidence level less than the target confidence level and belonging to the second sample are defined as first difficult samples. The first simple sample, the first medium sample, and the first difficult sample are defined as the first dataset.

4. The method according to claim 2, characterized in that, The step of dividing the second segmentation result into a second dataset with different difficulty levels includes: Extract the confidence score of each sample in the second dataset, and compare each confidence score with the target confidence score; Samples with a confidence level greater than or equal to the target confidence level are identified as second simple samples; The samples with confidence levels lower than the target confidence level are validated to obtain a second medium sample that passes the validation and a second difficult sample that fails the validation. The second simple sample, the second medium sample, and the second difficult sample are identified as the second dataset.

5. The method according to claim 1, characterized in that, The step of generating training sets for different training stages based on the third dataset includes: The third dataset is transformed according to the training instruction structure corresponding to the second semantic segmentation model to obtain the training instruction set; The training instruction set is divided into a first instruction set corresponding to the early stage of training, a second instruction set corresponding to the middle stage of training, and a third instruction set corresponding to the late stage of training, based on the difficulty level. The first instruction set has the highest proportion of easy-difficulty instruction samples, the second instruction set has the highest proportion of medium-difficulty instruction samples, and the third instruction set has the highest proportion of hard-difficulty instruction samples.

6. The method according to claim 5, characterized in that, The step of using the training set to perform phased optimization training on the second semantic segmentation model to obtain the target semantic segmentation model includes: The second semantic segmentation model is trained using LORA using the first instruction set to obtain the third semantic segmentation model; The third semantic segmentation model is trained using LORA using the second instruction set to obtain the fourth semantic segmentation model; The fourth semantic segmentation model is trained using the third instruction set to obtain the target semantic segmentation model.

7. A semantic segmentation device for dialogue, characterized in that, include: The first segmentation module is used to perform semantic segmentation on the first dialogue data and the second dialogue data using the first semantic segmentation model, and generate a first dataset and a second dataset based on the segmentation results. The first dataset and the second dataset both include sample data with different difficulty levels, and the different difficulty levels are obtained by a dual mechanism of confidence verification and manual verification. An integration module is used to integrate the first dataset and the second dataset according to the difficulty level to obtain a third dataset; The generation module is used to generate training sets for different training stages based on the third dataset; The training module is used to perform phased optimization training on the second semantic segmentation model using the training set to obtain the target semantic segmentation model. The second segmentation module is used to perform semantic segmentation on the target dialogue data using the target semantic segmentation model to obtain semantic segmentation results. The integration module is further configured to calculate the ratio of the number of samples with difficulty level of medium to hard in the first dataset to obtain a first ratio, and to calculate the ratio of the number of samples with difficulty level of medium to hard in the second dataset to obtain a second ratio; calculate the ratio difference between the first ratio and the second ratio; if the ratio difference is less than or equal to a target difference, then integrate the first dataset and the second dataset into a third dataset.

8. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program executable on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method of any one of claims 1 to 6.

Images