A speech recognition method and system for use in driving test (Part 3)

Abstract

This invention discloses a speech recognition method and system applied to the driving test (Part 3). Based on video and audio data, this invention filters out suspected cheating audio segments. Then, through a constructed speech recognition model, the speech data is processed as a simple image classification. While maintaining accuracy, the Squeeze-and-Excitatio module in MobileNetV3, which has a poor effect on speech classification, is removed from the speech recognition model's network structure to improve speed, balancing accuracy and speed. The ReLU activation function is used in the front part of the network, and the Hardwish function is used in the back part. Furthermore, the Focal Loss function is used to mitigate the imbalance problem in the speech data. This invention also addresses the problem in existing technologies that cannot monitor voice cheating behavior emitted by the safety officer in the driving test vehicle.

Description

Technical Field

[0001] This invention belongs to the field of speech recognition technology, specifically relating to a speech recognition method and system for use in the driving test (Part 3). Background Technology

[0002] With the continuous improvement of people's living standards, convenient transportation has become a major demand. Consequently, motor vehicle driving skills tests are receiving increasing attention and popularity, with tens of thousands of students obtaining driver's licenses every year. However, due to the large number of people needing to take the driving test, cheating inevitably occurs. For example, cheating frequently occurs in the road driving test (Subject 3). The frequent occurrence of such cheating increases the risk of traffic accidents, which not only affect the vehicle itself but also pose a serious threat to people's economic well-being and safety.

[0003] Therefore, it is essential to supervise the driving test (Part 3) to further improve and standardize the driver's license examination process. Currently, my country's driver's license skills examination uses a combination of computer-based and human assessment. Only some test items have implemented data collection and automatic scoring; others still require invigilators. Due to a shortage of invigilators for Part 3, driving instructors or staff from driving test centers are used as auxiliary safety officers. This leads to instances of cheating, such as safety officers providing voice prompts, which are difficult to detect and eliminate, thus affecting the fairness and impartiality of the entire driving test industry. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a speech recognition method and system for the driving skills test (Part 3), which solves the problem that the existing evaluation methods for motor vehicle driver's skills tests only achieve data collection and automatic assessment and scoring for some test items, and cannot effectively judge cheating behaviors such as voice prompts from safety officers during the Part 3 test.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A speech recognition method for use in the driving test (Part 3) comprises the following steps:

[0007] Step 1: Collect audio data from the test vehicle for the third subject within a preset time period in real time;

[0008] Step 2: Convert the audio data from the test vehicle within the preset duration into corresponding spectrograms;

[0009] Step 3: Based on the spectrogram corresponding to the audio data in the test vehicle within the preset duration, use a speech recognition model to determine whether the safety officer has engaged in pre-set voice cheating behavior, and identify the voice cheating behavior of the safety officer in the test vehicle.

[0010] Preferably, the speech recognition model uses a network that removes the Squeeze-and-Excitatio module from MobileNetV3, which has an insignificant effect on speech classification. The network uses the ReLU activation function in the pre-preset part and the Hardwish function in the post-preset part. The loss function of the model uses the Focal Loss function to alleviate the data imbalance problem.

[0011] Preferably, the steps for constructing the speech recognition model are as follows:

[0012] Step 3.1: Obtain training data. The training data includes positive sample audio and negative sample audio. Positive sample audio is audio that belongs to the preset security officer's voice cheating behavior, and negative sample audio is audio that does not belong to the preset security officer's voice cheating behavior.

[0013] Step 3.2: Convert the positive and negative sample audio data in the training data into their corresponding spectrograms;

[0014] Step 3.3: Based on the spectrograms corresponding to the positive and negative sample audio samples in the training data, train the model to obtain a speech recognition model that takes the spectrogram as input and outputs the result of whether there is speech cheating behavior. This model is used to monitor the speech cheating behavior of the safety officer in the vehicle during the driver's license test (subject 3).

[0015] Preferably, the training data in step 3.1 is obtained through the following steps:

[0016] Step 3.1.1: Obtain the video data of the safety officer in the vehicle during the driving test (Part 3), as well as the audio data synchronized with the video data;

[0017] Step 3.1.2: Based on the safety officer video data, and combined with the safety officer's actions and facial expressions in the video data, label the synchronized audio data. Label the audio data that belongs to the preset safety officer voice cheating behavior as positive samples, and label the audio that does not belong to the preset safety officer voice cheating behavior as negative samples.

[0018] Preferably, in step 3.1.2, based on the safety officer's video data and combining the safety officer's actions and facial expressions in the video data, the following process is specifically performed: tagging the synchronized audio data:

[0019] Based on the safety officer video data, and combining the safety officer's actions and facial expressions in the video data, video segments in the video data whose safety officer actions correspond to the preset safety officer voice cheating behavior and whose safety officer facial expressions correspond to the preset safety officer voice cheating behavior are labeled; and the audio in the synchronized audio data that corresponds to the video segments is labeled as positive samples, and the remaining audio in the synchronized audio data is labeled as negative samples.

[0020] Preferably, the actions corresponding to the preset safety officer's voice cheating behavior include the safety officer tapping the vehicle body.

[0021] Preferably, the facial expressions corresponding to the preset security officer's voice cheating behavior include a fake cough facial expression and a speaking facial expression.

[0022] Preferably, the preset safety officer voice cheating behaviors include the safety officer's feigned cough, the safety officer's knocking on the vehicle body, and the safety officer's speaking voice.

[0023] A speech recognition system for use in driving test (Part 3), and the speech recognition method for use in driving test (Part 3) described above, comprising: a data acquisition module, a data conversion module, and a data recognition module;

[0024] The data acquisition module is used to collect audio data from the vehicle used for the driving test (subject 3) in real time within a preset duration.

[0025] The data conversion module is used to convert the audio data inside the test vehicle within a preset time period into corresponding spectrograms.

[0026] The data recognition module is used to determine whether the security officer has engaged in pre-set voice cheating behavior.

[0027] A speech recognition terminal for use in driving test (Part 3) includes a memory and a processor, which are communicatively connected. The memory stores computer instructions, and the processor executes the computer instructions to perform the speech recognition method for driving test (Part 3).

[0028] The beneficial effects of this invention are:

[0029] This invention first filters out suspected cheating audio segments based on video and audio data. Then, it processes the audio data into a simple image classification method using a constructed speech recognition model. While maintaining accuracy, the Squeeze-and-Excitatio module in MobileNetV3, which has a poor effect on speech classification, is removed from the speech recognition model's network structure to improve speed, balancing accuracy and speed. The ReLU activation function is used in the front part of the network, and the Hardwish function is used in the back part. Furthermore, the Focal Loss function is used to mitigate the problem of imbalanced audio data. This invention also addresses the problem in existing technologies that cannot monitor voice cheating behavior issued by safety officers in vehicles during driving tests. Attached Figure Description

[0030] Figure 1 This is a flowchart of the speech recognition method in an embodiment of the present invention;

[0031] Figure 2 This is a flowchart of the generation of spectrograms according to an embodiment of the present invention. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings. The following embodiments will enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way.

[0033] This invention designs a speech recognition system for the driving test (Part 3), and a speech recognition method for the driving test (Part 3), comprising a data acquisition module, a data conversion module, and a data recognition module;

[0034] The data acquisition module is used to collect audio data from the vehicle used for the driving test (subject 3) in real time within a preset duration.

[0035] The data conversion module is used to convert the audio data inside the test vehicle within a preset time period into corresponding spectrograms.

[0036] The data recognition module is used to determine whether the safety officer has engaged in pre-set voice cheating behavior based on the spectrogram corresponding to the audio data in the test vehicle within a preset time period.

[0037] In this embodiment, the data acquisition module includes a video acquisition module and an audio acquisition module. The video acquisition module collects video data of the safety officer inside the vehicle during the driving test (subject 3), and the audio acquisition module collects audio data synchronized with the video data. The video acquisition module includes a camera, and the audio acquisition module includes a microphone. In this embodiment, two cameras are used for the video acquisition device. One is installed on top of the passenger side of the vehicle, and the other is installed on top of the passenger side windshield. By installing the cameras at different positions and angles, complete image data of the passenger's body movements and facial expressions can be collected, avoiding missed judgments. Furthermore, the number and location of the cameras can be adjusted according to the needs of the actual application scenario, and no restrictions are imposed here.

[0038] like Figure 1 As shown, a speech recognition method applied to the driving test (Part 3) includes the following steps:

[0039] Step 1: Collect audio data from the test vehicle for the third subject within a preset time period in real time;

[0040] Step 2: Convert the audio data from the test vehicle within the preset duration into corresponding spectrograms;

[0041] Since spectrograms can reveal features such as intonation and speech rate, the audio above was converted into a spectrogram and used as image input. This transforms the speech problem into an image-based problem, breaking the limitation of treating speech solely as speech. For this project, image algorithms are simpler, more convenient, faster, and more efficient. Figure 2 As shown, the process of generating a spectrogram includes:

[0042] (1) Pre-emphasize, frame, and window the acquired audio;

[0043] Pre-emphasis: Low-frequency components in audio data are filtered out by a high-pass filter with transfer function s[n]-a*s[n], making high-frequency characteristics more prominent;

[0044] S preemp [n] = S[n] - α*S[n-1]

[0045] Typical value of α = 0.95

[0046] Frame segmentation: N sampling points are grouped into one observation unit. In this embodiment, the time covered by each frame is set to 32ms, the sampling rate is 16000, and the number of sample points in each frame is 512.

[0047] To avoid excessive changes between two adjacent frames, an overlap region is set between two adjacent frames. The overlap region is set to 16ms, that is, a frame is taken every 16ms.

[0048] Windowing is applied to each frame: (During the conversion, the intra-frame signal is treated as a periodic signal, so abrupt changes occur at the two endpoints of the frame, resulting in a significant difference between the converted spectrum and the original signal spectrum.) Windowing is applied to each frame to prevent abrupt changes at the two endpoints when performing FFT on the intra-frame signal; the window used is a Hamming window: (M is the frame length, i.e., 512).

[0049] W[n] = 0.54 - 0.46cos(2πn / M)

[0050] In the formula, M is the frame length, with a value of 512;

[0051] Padding each frame with zeros: Perform an FFT on each frame of signal. The input data length must be 2^K. Now, a frame has 512 sampling points, so pad with zeros to the nearest 512 bits.

[0052] (2) For each short-time analysis window, the corresponding spectrum is obtained by FFT;

[0053] (3) The spectrogram of the audio signal is obtained by taking the square of the modulus of the audio signal spectrum. That is, a 512-point FFT transformation is performed on each frame signal after framing and windowing to obtain the spectrum of each frame, and the spectrogram of the audio signal is obtained by taking the square of the modulus of the audio signal spectrum.

[0054] In this embodiment, the preset duration is 3 seconds. Since cheating is judged once every 3 seconds and the sampling frequency is 16kHz, the sample points for 3 seconds are 48000. During the generation of the spectrogram, the audio data is printed and subjected to Fast Fourier Transform (FFT). The sampling point (NFFT) is 512 for easy divisibility. Therefore, the value of the sampling point is 48128, which means that the audio is cut into segments of 3.008 seconds each. Since the preset duration is 3 seconds, the audio length is insufficient, so zero values ​​are used to pad it.

[0055] Step 3: Based on the spectrogram corresponding to the audio data inside the test vehicle within a preset time period, the speech recognition model is used to determine whether the safety officer has engaged in pre-set voice cheating behavior, and the voice cheating behavior of the safety officer inside the test vehicle is monitored. Then, the audio data inside the test vehicle within the preset time period indicating pre-set voice cheating behavior, along with the safety officer's video data synchronized with the audio data, are sent to the examination supervision platform via the Internet.

[0056] In this embodiment, for the audio data collected in the test vehicle for the third subject within a preset time period in real time, the following steps are also performed: based on the safety officer's video data, feature extraction is performed on the synchronized audio data, and audio segments suspected of being used by the safety officer to cheat are obtained as objects for spectrogram conversion, and then steps 2-3 are performed.

[0057] Based on the security officer's video data, the following methods were used to extract features from the audio data to obtain audio segments suspected of being used by the security officer to cheat:

[0058] For each frame of the safety officer's video data, the system determines whether the safety officer's facial expression is a suspected speaking expression. If it is, audio data within a preset duration is used as the suspected cheating audio segment. If it is not, the system collects the safety officer's video data inside the test vehicle for the next preset duration, along with synchronized audio data. The suspected speaking expressions include feigned coughing and speaking expressions. Specifically, a pre-trained facial recognition model takes an image containing a human face as input and determines whether the facial expression is a speaking expression as output. This model uses a binary classification approach to judge the safety officer's facial expressions, further determining whether they are feigned coughing or speaking. If neither feigned coughing nor speaking is present, the expression is not considered a suspected speaking expression; if either feigned coughing or speaking is present, the expression is considered a suspected speaking expression.

[0059] Furthermore, for each frame of the safety officer's video data, the system determines whether the safety officer's hand is in contact with the vehicle based on the position of their hand. If the safety officer's hand is in contact with the vehicle, the audio data within a preset duration is used as a suspected cheating audio segment. If the safety officer's hand is not in contact with the vehicle, the system collects the safety officer's video data inside the vehicle for the next preset duration, along with the audio data synchronized with the video data. In this embodiment, a multi-target recognition method is used to detect the position of the hand in the image and preset positions inside the vehicle. Then, it is determined whether the bounding box of the hand's position overlaps with the bounding boxes of the preset positions inside the vehicle. If there is an overlap, it indicates that the safety officer's hand is in contact with the vehicle; if there is no overlap, it indicates that the safety officer's hand is not in contact with the vehicle. Based on the video data obtained from two cameras, recognition is performed simultaneously on each frame of the safety officer's video data. When a frame of the synchronized video data from both cameras simultaneously shows the safety officer's hand in contact with the vehicle, it is determined that the safety officer's hand is in contact with the vehicle. This accelerates the speed of speech recognition and improves the accuracy of speech recognition through dual recognition.

[0060] The speech recognition system of the present invention, which is applied to the driving test, also includes a data extraction module. Based on the video data of the safety officer, the data extraction module is used to extract features from the audio data and obtain audio segments in the audio data that are suspected of being used by the safety officer to cheat.

[0061] The aforementioned speech recognition model is used to monitor the voice cheating behavior of in-vehicle safety officers during the driver's license test (subject three). The speech recognition model employs a network that removes the Squeeze-and-Excitatio module from MobileNetV3, which has a poor performance in speech classification. The network uses the ReLU activation function in the pre-set part and the Hardwish function in the post-set part. The model's loss function uses the Focal Loss function to mitigate the data imbalance problem. The construction steps are as follows:

[0062] Step 3.1: Obtain training data. The training data includes positive sample audio and negative sample audio. Positive sample audio is audio that belongs to the preset security officer's voice cheating behavior, and negative sample audio is audio that does not belong to the preset security officer's voice cheating behavior.

[0063] The training data in step 3.1 is obtained through the following steps:

[0064] Step 3.1.1: Obtain the video data of the safety officer in the vehicle during the driving test (Part 3), as well as the audio data synchronized with the video data;

[0065] Step 3.1.2: Based on the safety officer video data, and combined with the safety officer's actions and facial expressions in the video data, label the synchronized audio data. Label the audio data that belongs to the preset safety officer voice cheating behavior as positive samples, and label the audio that does not belong to the preset safety officer voice cheating behavior as negative samples.

[0066] In step 3.1.2, based on the safety officer's video data, and combining the safety officer's actions and facial expressions in the video data, the following process is specifically performed: The synchronized audio data is labeled:

[0067] Based on safety officer video data, and combining the safety officer's actions and facial expressions in the video data, video segments in the video data where the safety officer's actions correspond to preset actions of voice cheating and the safety officer's facial expressions correspond to preset facial expressions of voice cheating are labeled. Audio clips in the synchronized audio data corresponding to the video clips are labeled as positive samples, and the remaining audio clips in the synchronized audio data are labeled as negative samples; that is, audio clips in the synchronized audio data that are not labeled as positive samples are labeled as negative samples. In other words, the audio data of the video is labeled using the posture or tags of each frame of the video combined with sound.

[0068] The actions corresponding to the preset voice cheating behavior of the safety officer include the action of the safety officer tapping on the vehicle body. The facial expressions corresponding to the preset voice cheating behavior of the safety officer include the facial expressions of faking a cough and speaking.

[0069] Step 3.2: Convert the positive and negative audio samples in the training data into their corresponding spectrograms. The spectrogram conversion method is the same as that used in a speech recognition method applied to the driving test. The acquired speech signal is pre-emphasized, framed, and windowed. For each short-time analysis window, the corresponding spectrum is obtained through FFT. The spectrogram of the speech signal is obtained by taking the square of the modulus of the spectrum.

[0070] Step 3.3: Based on the spectrograms corresponding to the positive and negative sample audio samples in the training data, a model is trained to obtain a speech recognition model that takes the spectrogram as input and outputs the result of whether speech cheating behavior exists. This model is used to monitor the speech cheating behavior of the safety officer in the vehicle during the driver's license test (subject 3). The preset speech cheating behaviors of the safety officer include the safety officer's feigned cough, the safety officer's knocking on the vehicle body, and the safety officer's speaking voice. In this embodiment, the input size is uniformly adjusted to 224*224, the spectrogram is normalized, and a MobileNet v3 network with the Squeeze-and-Excitatio module removed is used as the backbone network of the deep convolutional network for training the binary classification model to obtain the speech recognition model.

[0071] For speech recognition models, MobileNet v3 references three network architectures: MobileNetv1, MobileNetv2, and MNASNet. MobileNet v1 proposes depthwise separable convolutions, consisting of channel-wise 3x3 convolutions (depthwise convolutions) and cross-channel 1x1 convolutions (pointwise convolutions). MobileNet v2 proposes an inverted residual with a linear bottleneck. MNASNet adds the Squeeze-and-Excitation module from SENet to MobileNet v2. Different combinations are used in the residual structure: SENet places the Squeeze-and-Excitation module after the residual module, while MNASNet places it in the middle. Through comparative experiments, this invention found that the Squeeze-and-Excitation module has little impact on accuracy; therefore, the Squeeze-and-Excitation module in MobileNet v3 is removed, and only the MobileNetv1 and MobileNetv2 modules are used.

[0072] Additionally, a portion of the ReLU activation function is directly replaced with Hardswish, a stronger activation function optimized from a lightweight model. Mish is a stronger activation function than ReLU, providing a performance boost, but due to the lack of inplace, its memory usage almost doubles. A hard version of Mish, designed for mobile devices, also incorporates inplace, known as hardmish. The performance of hardswish and hardmish is similar, but hardswish is more stable. Therefore, in the network, Hardswish is chosen to directly replace part of the ReLU activation function. Firstly, ReLU6 can be implemented in many hardware and software frameworks; secondly, quantization avoids loss of numerical precision and is fast. This non-linear change increases the model latency by 15%. Compared to ReLU6, hardswish is slightly slower but offers higher accuracy. Considering the trade-off between model speed and accuracy, the faster ReLU (not ReLU6) is used in the first half of the network structure when the image size is large, while the more accurate hardswish is used in the second half. In this embodiment, reducing the image size to one-quarter of the original image size is used as the dividing point between the preset first half and the preset second half. After the image size is reduced to one-quarter of the original image size, the network presets the second half using the Hardswish function, while before that, the network presets the first half using the ReLU activation function. Here, we introduce ReLU6, which is related to Hardswish. The ReLU6 activation function is min(max(x,0),6), and its idea is simple: truncate the result obtained by ReLU to a maximum value of 6.

[0073] The Hardwish function formula is as follows:

[0074]

[0075] Here, h-swish is an abbreviation for hardswish, and x is the output of the previous layer in the network.

[0076] For the loss function of speech recognition models, Focal Loss is adopted to address single-stage scenarios where there is an extreme imbalance between foreground and background classes during training (e.g., 1:1000). Easily classified negative examples constitute the majority of the loss and dominate the gradient. To better distinguish between easy and hard examples, Focal Loss introduces a modulation factor, which effectively reduces the influence of easily negative examples, focusing all attention on hard negative examples. Intuitively, the modulation factor reduces the loss contribution of easy examples and expands the range of examples receiving low loss. This loss function is now widely used for class imbalance problems in binary classification, which are classic examples of imbalanced binary classification.

[0077] Based on the structure and loss function of the speech recognition model described above, audio data of the security guard's feigned coughing, knocking on the vehicle, and speaking are considered cheating data, i.e., positive sample data, labeled with the number 1. Other audio data are considered non-cheating data, i.e., negative sample data. Corresponding spectrograms are generated from the audio data and used as training samples. The speech recognition model is then trained, i.e., a binary classification model is trained to obtain the speech recognition model. Using the trained speech recognition model, real-time video and audio are analyzed to identify suspected cheating audio segments, which are then converted into spectrograms. These spectrograms are then used for identification by the speech recognition model to determine whether the security guard has engaged in any of the preset types of speech cheating behavior. Specifically, if the spectrograms corresponding to the suspected cheating audio segments do not correspond to any of the preset types of speech cheating behavior, the output is that the security guard has not engaged in any of the preset types of speech cheating behavior; if the spectrograms corresponding to the suspected cheating audio segments correspond to any of the preset types of speech cheating behavior, the output is that the security guard has engaged in any of the preset types of speech cheating behavior.

[0078] A speech recognition terminal for use in driving test part 3 includes a memory and a processor, which are communicatively connected. The memory stores computer instructions, and the processor executes the computer instructions to perform the speech recognition method for driving test part 3.

[0079] This invention first filters out suspected cheating audio segments based on video and audio data. Then, it processes the audio data into a simple image classification method using a constructed speech recognition model. While maintaining accuracy, the Squeeze-and-Excitatio module in MobileNetV3, which has a poor effect on speech classification, is removed from the speech recognition model's network structure to improve speed, balancing accuracy and speed. The ReLU activation function is used in the front part of the network, and the Hardwish function is used in the back part. Furthermore, the Focal Loss function is used to mitigate the problem of imbalanced audio data. This invention also addresses the problem in existing technologies that cannot monitor voice cheating behavior issued by safety officers in vehicles during driving tests.

[0080] The above are merely preferred embodiments of the present invention, but do not limit the patent scope of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of the present invention specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of the present invention.

Claims

1. A voice recognition method applied to a subject three test, characterized in that, The steps are as follows: Step 1: Real-time acquisition of audio data in the subject three test vehicle within a preset time length; Step 2: Convert the audio data in the subject three test vehicle within a preset time length into corresponding spectrograms; Step 3: Based on the spectrogram corresponding to the audio data in the subject three test vehicle within a preset time length, use the speech recognition model to determine whether the safety officer has a preset safety officer speech cheating behavior, and identify the speech cheating behavior of the safety officer in the subject three vehicle; The speech recognition model uses a network that removes the Squeeze-and-Excitation module in mobieNetV3 which has no obvious effect on speech classification, uses the relu activation function in the preset front part of the network, and uses the hardwish function in the preset rear part; The loss function of the model uses the Focal Loss function to alleviate the data imbalance problem; The construction steps of the speech recognition model are as follows: Step 3.1: Obtain training data, which includes positive sample audio and negative sample audio, the positive sample audio is audio belonging to the preset safety officer speech cheating behavior, and the negative sample audio is audio not belonging to the preset safety officer speech cheating behavior; Step 3.2: Convert the positive sample audio and negative sample audio included in the training data into corresponding spectrograms respectively; Step 3.3: Based on the spectrogram corresponding to the positive sample audio and negative sample audio included in the training data, model training is performed, and a speech recognition model taking spectrogram as input and taking whether there is speech cheating behavior as output is obtained, which is used for monitoring the speech cheating behavior of the safety officer in the subject three test vehicle; The training data in step 3.1 is obtained through the following steps: Step 3.1.1: Obtain safety officer video data in the subject three test vehicle, and audio data synchronized with the video data; Step 3.1.2: Based on the safety officer video data, in combination with the actions and facial expressions of the safety officer in the safety officer video data, label the synchronized audio data, and mark the audio belonging to the preset safety officer speech cheating behavior as positive samples, and mark the audio not belonging to the preset safety officer speech cheating behavior as negative samples; In step 3.1.2, based on the safety officer video data, in combination with the actions and facial expressions of the safety officer in the safety officer video data, the following process is specifically performed to label the synchronized audio data: Based on the safety officer video data, in combination with the actions and facial expressions of the safety officer in the safety officer video data, the video segment in which the safety officer's actions and facial expressions in the safety officer video data are marked as the actions and facial expressions corresponding to the preset safety officer speech cheating behavior; And mark the audio corresponding to the video segment in the synchronized audio data as a positive sample, and mark the remaining audio in the synchronized audio data as a negative sample.

2. The speech recognition method for a subject three test according to claim 1, wherein, The actions corresponding to the preset safety officer speech cheating behavior include the safety officer knocking the vehicle body action.

3. The speech recognition method for a subject three test according to claim 1, wherein, The facial expressions corresponding to the preset safety officer speech cheating behavior include the fake cough facial expression and the speaking facial expression.

4. The speech recognition method for a subject three test according to claim 1, wherein, The preset safety officer voice cheating behavior includes a safety officer pretending to cough, a safety officer knocking the vehicle body, and a safety officer speaking.

5. The speech recognition system applied to the subject three test, applied to the speech recognition method applied to the subject three test in any one of claims 1-4, characterized in that, The method comprises the following steps: A data acquisition module, a data conversion module, and a data identification module are provided. The data acquisition module is configured to collect audio data in a vehicle during a subject three test in real time. The data conversion module is configured to convert the audio data in the vehicle during the subject three test into a corresponding spectrogram. The data identification module is configured to determine whether the safety officer has a preset safety officer voice cheating behavior.

6. A voice recognition terminal applied to a subject three test, characterized by, The application comprises a memory and a processor, which are in communication connection with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the voice recognition method for the subject three test according to claims 1-4.

Images