Assessment method and system for traffic police to handle speech on site

Abstract

The invention discloses an assessment method and system for traffic police on-site speech processing, and belongs to the technical field of identification, and the method comprises the steps: obtaining the dialogue recording of the traffic police under a simulation condition; carrying out text transcription on the dialogue record and then carrying out preprocessing to obtain a preprocessed data set; constructing a speech scoring model based on a RoBERTa model and a long-short term memory network; and inputting the preprocessed data set into the speech scoring model for calculation, and generating a speech assessment score. According to the method, the semantic comprehension of RoBERTa and the time sequence modeling capability of LSTM are integrated, comprehensive evaluation features are formed through a full connection layer, and more comprehensive and accurate speech scores can be provided.

Description

Technical Field

[0001] The invention belongs to the field of recognition technology, and in particular relates to an assessment method and system for on-the-spot speech handling by traffic police. Background Art

[0002] In recent years, with the rapid development of the domestic economy, the number of motor vehicles and drivers has increased exponentially. Many new drivers have just learned to drive on the road, which has made the law enforcement work of the traffic police department face severe challenges of reform and development. At the same time, it has also put forward higher requirements for the training and management departments of the traffic police department under the new situation.

[0003] However, the current assessment process for on-the-spot speech of traffic police often relies on the subjective judgment of the assessor, resulting in a lack of consistency in the assessment results between different assessors. In addition, different regions or departments may have different assessment standards and methods, making it difficult to compare or standardize the assessment results on a wider scale. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for assessing the on-the-spot speech of traffic police to solve the problems existing in the above prior art.

[0005] To achieve the above object, the present invention provides a method for assessing the on-the-spot speech of traffic police, comprising:

[0006] Obtain recordings of conversations between traffic police officers in simulated situations;

[0007] Preprocessing the conversation recording after text transcription to obtain a preprocessed data set;

[0008] Construct a speech scoring model based on the RoBERTa model and long short-term memory network;

[0009] The preprocessed data set is input into the speech scoring model for calculation to generate a speech assessment score.

[0010] Preferably, the process of obtaining the recording of the conversation between the traffic police in the simulated situation includes:

[0011] Setting traffic situation simulation scenarios, including but not limited to traffic accidents, illegal parking, and drunk driving tests;

[0012] Importing the traffic situation simulation scene into the VR device;

[0013] The traffic police to be tested interacts with the simulated scene through the VR device and records the conversation during the interaction.

[0014] Preferably, the process of obtaining the preprocessed data set includes:

[0015] Cleaning and normalizing the conversation recording to generate cleaned data;

[0016] The cleaned data is segmented based on semantics to generate the preprocessed data set.

[0017] Preferably, the conversation recording is cleaned and normalized to generate cleaned data, including:

[0018] Smoothing the conversation recording through a Butterworth low-pass filter to obtain filtered data;

[0019] The filtered data is normalized to obtain normalized data, and semantic features of the normalized data are extracted based on an autoencoder to obtain the cleaned data.

[0020] Preferably, the process of segmenting the cleaned data based on semantics includes:

[0021] Using window segmentation technology to divide the cleaned data into several data segments;

[0022] The plurality of data segments are arranged based on a time axis to generate the preprocessed data set.

[0023] Preferably, the process of constructing a speech scoring model based on the RoBERTa model and the long short-term memory network includes:

[0024] Building a semantic recognition module based on the RoBERTa model, building a sequence processing module based on the long short-term memory network, and sequentially connecting the semantic recognition module, the long short-term memory network and the fully connected layer to generate an improved model;

[0025] Obtaining on-the-spot response corpora of traffic police in different situations, and constructing training sets and test sets based on the on-the-spot response corpora of traffic police in different situations;

[0026] The improved model is trained by using the training set, and the trained improved model is tested by using the test set to generate the speech scoring model.

[0027] Preferably, the process of inputting the preprocessed data set into the speech scoring model for calculation and generating a speech assessment score comprises:

[0028] Using the RoBERTa model as a feature extractor, converting the preprocessed dataset into a deep semantic representation vector;

[0029] Input the semantic representation vector into a plurality of LSTM layers, wherein the LSTM layers are used to process the sequence dependency in the semantic representation vector, recognize and memorize the contextual dynamics of the entire conversation;

[0030] The fully connected layer is used to give the speech assessment score according to the output of LSTM.

[0031] To achieve the above purpose, the present invention also provides an assessment system for on-the-spot speech of traffic police, comprising:

[0032] A data acquisition module is used to obtain recordings of conversations between traffic police officers in simulated situations;

[0033] A preprocessing module, used for preprocessing the conversation recording after text transcription to obtain a preprocessed data set;

[0034] A model building module for building a speech scoring model based on the RoBERTa model and the long short-term memory network;

[0035] The calculation module is used to input the preprocessed data set into the speech scoring model for calculation and generate a speech assessment score.

[0036] Compared with the prior art, the present invention has the following advantages and technical effects:

[0037] The present invention combines the powerful semantic understanding ability of the RoBERTa model and the time series processing ability of LSTM to more comprehensively capture and analyze the semantic information in the speech of traffic police. The fully connected layer can integrate and further process the features extracted from RoBERTa and LSTM to form the final scoring result.

[0038] The present invention integrates the semantic understanding of RoBERTa and the temporal modeling capability of LSTM, and forms a comprehensive evaluation feature through the fully connected layer, which can provide a more comprehensive and accurate speech score. It can also conduct a detailed analysis of the speech of traffic police, including semantic accuracy, language fluency, response timeliness, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 A flow chart of a method for assessing the on-the-spot speech of traffic police according to an embodiment of the present invention;

[0041] Figure 2 The present invention is a schematic diagram of an assessment system for on-the-spot speech handling by traffic police according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] Embodiment 1

[0045] like Figure 1 As shown, this embodiment provides a method for assessing the on-the-spot speech of traffic police, including:

[0046] Obtain recordings of conversations between traffic police officers in simulated situations;

[0047] Preprocessing the conversation recording after text transcription to obtain a preprocessed data set;

[0048] Construct a speech scoring model based on the RoBERTa model and long short-term memory network;

[0049] The preprocessed data set is input into the speech scoring model for calculation to generate a speech assessment score.

[0050] To further optimize the solution, the process of obtaining the recording of the conversation between the traffic police in the simulated situation includes:

[0051] Setting traffic situation simulation scenarios, including but not limited to traffic accidents, illegal parking, and drunk driving tests;

[0052] Importing the traffic situation simulation scene into the VR device;

[0053] The traffic police to be tested interacts with the simulated scene through the VR device and records the conversation during the interaction.

[0054] Further optimizing the scheme, the process of obtaining the preprocessed data set includes:

[0055] Cleaning and normalizing the conversation recording to generate cleaned data;

[0056] The cleaned data is segmented based on semantics to generate the preprocessed data set.

[0057] To further optimize the solution, the conversation recording is cleaned and normalized, and the process of generating cleaned data includes:

[0058] Smoothing the conversation recording through a Butterworth low-pass filter to obtain filtered data;

[0059] Since the inherent noise of the recorder itself and the unconscious shaking of the tester will cause noise in the collected data, the collected data needs to be filtered and denoised before feature extraction. Considering that the frequency of the human motion signal is mainly concentrated in the low-frequency section of 0 to 10 Hz, this embodiment selects a low-pass filter with a maximum flat amplitude response, namely a Butterworth low-pass filter. Considering that the acquisition frequency of the acceleration and gyroscope is 128 Hz, and the filter order is determined to be 4th order based on some given indicators such as the passband and stopband cutoff frequencies, a 4th-order Butterworth low-pass filter with a cutoff frequency of 10 Hz is selected to smooth the signal to obtain filtered data;

[0060] The calculation formula of Butterworth low-pass filter is:

[0061]

[0062] Among them, D o is the cutoff frequency of the filter, n is the order of the filter. In practical applications, D can be modified according to different filtering requirements. o and n value.

[0063] The filtered data is normalized to normalized data using a linear function. In order to improve the network convergence speed and avoid neuron oversaturation, the input data is processed using the maximum and minimum normalization method to map the original signal data to [0,1].

[0064] The calculation formula for normalization is:

[0065]

[0066] x is the input signal value, x max 、x min is the maximum and minimum value of all signals, x norm is the normalized value.

[0067] The filtered data is normalized to obtain normalized data, and semantic features of the normalized data are extracted based on an autoencoder to obtain the cleaned data.

[0068] Further optimizing the solution, the process of segmenting the cleaned data based on semantics includes:

[0069] Using window segmentation technology to divide the cleaned data into several data segments;

[0070] Window segmentation technology is used to divide the normalized data into several data segments. Most speech recognition methods divide the time series data into smaller time windows through window segmentation technology, and then extract features and recognize the corresponding speech in each time window. The window length and window overlap rate have a great impact on the accuracy and real-time performance of human speech recognition.

[0071] The data is segmented to obtain several data segments, and all the data segments together construct a sample set S, which is divided into a training set and a test set according to a ratio of 7:3. The specific process is: a sliding window with a width of 860ms and an overlap rate of 50% is used to segment the speech data. The segmented speech data is F m , F m =[x1,x2,...,x t ], m represents the number of action data, the length of each data segment is 128, and all data segments together constitute the sample set S, S = [F 1 ,F 2 ,…,F m ];

[0072] The plurality of data segments are arranged based on a time axis to generate the preprocessed data set.

[0073] Further optimizing the scheme, the process of constructing a speech scoring model based on the RoBERTa model and the long short-term memory network includes:

[0074] A semantic recognition module is constructed based on the RoBERTa model, and the model performance is enhanced by removing the Next Sentence Prediction (NSP) task of BERT and modifying the pre-training method. A sequence processing module is constructed based on the long short-term memory network, and the semantic recognition module, the long short-term memory network and the fully connected layer are sequentially connected to generate an improved model.

[0075] Obtaining on-the-spot response corpora of traffic police in different situations, and constructing training sets and test sets based on the on-the-spot response corpora of traffic police in different situations;

[0076] First, we need to collect the on-the-spot response data of traffic police in different situations. These data can come from simulation training, actual recordings or text records. The collected data will be annotated according to quality, such as scoring according to the response effect, professionalism, language logic, etc. of the traffic police.

[0077] Convert the collected text data into a format that the RoBERTa model can understand, such as performing word segmentation and removing irrelevant information. Use a pre-trained RoBERTa model as a basis, which can be obtained from libraries such as Hugging Face. Fine-tune on a specific on-site speech dataset. At this stage, the model will learn how to judge the effectiveness of the traffic police's response based on their speech.

[0078] Based on industry standards or expert opinions, define what type of response is ideal. For example, quick response, accuracy, organization, politeness, etc. may all be evaluation criteria. Convert these evaluation criteria into quantifiable scoring criteria, such as comprehensively evaluating the handling effect of traffic police by scoring different dimensions.

[0079] The improved model is trained by using the training set, and the trained improved model is tested by using the test set to generate the speech scoring model.

[0080] RoBERTa is a Transformer-based model that is very good at learning deep semantic information from large amounts of text.

[0081] LSTM is a recurrent neural network that excels at processing and memorizing time series information in input data.

[0082] Feature extraction layer: Use the pre-trained RoBERTa model as a feature extractor to convert text into a deep semantic representation vector.

[0083] Sequence processing layer: The output vector sequence of RoBERTa is input into one or more LSTM layers. The LSTM layer is responsible for processing the sequence dependencies in these vectors, identifying and memorizing the contextual dynamics of the entire conversation.

[0084] Output layer: A fully connected layer is added after LSTM to give the final score or classification based on the output of LSTM.

[0085] Finally, the features are input into the fully connected layer, which is a regression analysis layer. The collected gyroscope data contains both positive and negative numbers. If the ReLU activation function is used, the gradient value calculated for the part of the input less than zero during the back propagation process is 0, resulting in some ReLU neurons never being activated in the subsequent training process. Therefore, the SaveReLU activation function is used for activation;

[0086] The calculation formula of the SaveReLU activation function is:

[0087]

[0088] e is a constant with a value range of [0,0.5], which ensures that the derivative of SaveReLU is always non-zero, and then the gradient of the part of the input less than zero can be calculated during the back propagation process, avoiding the problem of neuron "death".

[0089] Configure an appropriate loss function and optimizer, for example, use the cross entropy loss and the Adam optimizer.

[0090] Fine-tune the entire model, including the RoBERTa and LSTM parts, on specific traffic police speech data to ensure that both are adapted to the needs of the specific task.

[0091] The performance of the model is evaluated by setting evaluation metrics such as precision, recall, and F1 score.

[0092] Precision, recall, average precision (AP) and mean average precision (mAP) are used as evaluation indicators.

[0093] Precision is the proportion of true positive samples among all predicted positive samples, and is calculated as:

[0094]

[0095] Among them, TP is the number of positive samples predicted by the model as positive samples, and FP is the number of negative samples predicted by the model as positive samples.

[0096] The recall rate is the proportion of all true positive samples that are predicted to be positive samples. The formula is as follows:

[0097]

[0098] Among them, TP is the number of positive samples predicted by the model as positive samples, and FN is the number of positive samples predicted by the model as negative samples.

[0099] The horizontal axis of the PR curve is the recall rate, and the vertical axis is the precision rate, which is also called the precision-recall curve. The average precision is the surface area enclosed under the PR curve. The formula is as follows:

[0100]

[0101] In the field of target detection, data sets usually contain multiple categories. AP is often used to measure the detection effect of a model for a certain category of targets. If the detection effect of multiple categories of targets is evaluated, the AP value of each category needs to be summed and averaged to obtain the mean average precision. mAP is a more comprehensive evaluation indicator that can reflect the overall performance of the model in multi-category target detection tasks. The formula is as follows:

[0102]

[0103] The IoU threshold affects the mAP value of the model. The commonly used mAP indicators are mAP@0.5 and mAP@0.5:0.95. mAP@0.5 represents the average precision of all categories when the IoU threshold is 0.5.

[0104] Deploy the trained model to the actual assessment system for real-time or batch speech assessment.

[0105] Further optimizing the scheme, the process of inputting the preprocessed data set into the speech scoring model for calculation and generating the speech assessment score includes:

[0106] Using the RoBERTa model as a feature extractor, converting the preprocessed dataset into a deep semantic representation vector;

[0107] Input the semantic representation vector into a plurality of LSTM layers, wherein the LSTM layers are used to process the sequence dependency in the semantic representation vector, recognize and memorize the contextual dynamics of the entire conversation;

[0108] The fully connected layer is used to give the speech assessment score according to the output of LSTM.

[0109] Embodiment 2

[0110] like Figure 2 As shown, in this embodiment, a system for assessing the speech of traffic police on the spot is provided, including:

[0111] A data acquisition module is used to obtain recordings of conversations between traffic police officers in simulated situations;

[0112] A preprocessing module, used for preprocessing the conversation recording after text transcription to obtain a preprocessed data set;

[0113] A model building module for building a speech scoring model based on the RoBERTa model and the long short-term memory network;

[0114] The calculation module is used to input the preprocessed data set into the speech scoring model for calculation and generate a speech assessment score.

[0115] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for assessing the on-the-spot speech of traffic police, characterized in that: The following steps are involved: Obtain recordings of conversations between traffic police officers in simulated situations; Preprocessing the conversation recording after text transcription to obtain a preprocessed data set; Construct a speech scoring model based on the RoBERTa model and long short-term memory network; The preprocessed data set is input into the speech scoring model for calculation to generate a speech assessment score.

2. The method for assessing the on-the-spot speech of traffic police according to claim 1 is characterized in that: The process of obtaining the recording of the conversation between the traffic police in the simulated situation includes: Setting traffic situation simulation scenarios, including but not limited to traffic accidents, illegal parking, and drunk driving tests; Importing the traffic situation simulation scene into the VR device; The traffic police to be tested interacts with the simulated scene through the VR device and records the conversation during the interaction.

3. The method for assessing the on-the-spot speech of traffic police according to claim 1 is characterized in that: The process of obtaining the preprocessed data set includes: Cleaning and normalizing the conversation recording to generate cleaned data; The cleaned data is segmented based on semantics to generate the preprocessed data set.

4. The method for assessing the on-the-spot speech of traffic police according to claim 3 is characterized in that: The process of cleaning and normalizing the conversation recording to generate cleaned data includes: Smoothing the conversation recording through a Butterworth low-pass filter to obtain filtered data; The filtered data is normalized to obtain normalized data, and semantic features of the normalized data are extracted based on an autoencoder to obtain the cleaned data.

5. The method for assessing the on-the-spot speech of traffic police according to claim 3 is characterized in that: The process of segmenting the cleaned data based on semantics includes: Using window segmentation technology to divide the cleaned data into several data segments; The plurality of data segments are arranged based on a time axis to generate the preprocessed data set.

6. The method for assessing the on-the-spot speech of traffic police according to claim 1 is characterized in that: The process of constructing a speech scoring model based on the RoBERTa model and the long short-term memory network includes: Building a semantic recognition module based on the RoBERTa model, building a sequence processing module based on the long short-term memory network, and sequentially connecting the semantic recognition module, the long short-term memory network and the fully connected layer to generate an improved model; Obtaining on-the-spot response corpora of traffic police in different situations, and constructing training sets and test sets based on the on-the-spot response corpora of traffic police in different situations; The improved model is trained by using the training set, and the trained improved model is tested by using the test set to generate the speech scoring model.

7. The method for assessing the on-the-spot speech of traffic police according to claim 6 is characterized in that: The process of inputting the preprocessed data set into the speech scoring model for calculation and generating a speech assessment score includes: Using the RoBERTa model as a feature extractor, converting the preprocessed dataset into a deep semantic representation vector; Input the semantic representation vector into a plurality of LSTM layers, wherein the LSTM layers are used to process the sequence dependency in the semantic representation vector, recognize and memorize the contextual dynamics of the entire conversation; The fully connected layer is used to give the speech assessment score according to the output of LSTM.

8. A system for assessing the speech of traffic police on the spot, characterized in that: include: A data acquisition module is used to obtain recordings of conversations between traffic police officers in simulated situations; A preprocessing module, used for preprocessing the conversation recording after text transcription to obtain a preprocessed data set; A model building module for building a speech scoring model based on the RoBERTa model and the long short-term memory network; The calculation module is used to input the preprocessed data set into the speech scoring model for calculation and generate a speech assessment score.

Images