Training examination simulation method and system based on language and action synchronization

Abstract

The invention discloses a training examination simulation method and system based on language and action synchronization. The method comprises the following steps: constructing a plurality of police and people communication teaching plan libraries and an expert library comprising standard police and people communication characteristics; according to the police and people communication teaching plan library, establishing a voice database, including a virtual alarm condition model of a face model and an action model, so that the virtual alarm condition model reproduces alarm processing performance; constructing a virtual alarm processing scene based on the voice database and the virtual alarm condition model; acquiring interaction information when the trained police interacts with the voice database and the virtual alarm condition model in the virtual alarm processing scene through acquisition equipment; and scoring the similarity between the interaction information and the communication standard characteristics in the expert database to obtain a training examination simulation result of the trained policeman. According to the invention, through systematic training and assessment, it is ensured that traffic police can rapidly and accurately execute tasks in a complex and changeable traffic environment.

Description

Technical Field

[0001] The invention belongs to the technical field of educational software development, and in particular relates to a training and assessment simulation method and system based on language and action synchronization. Background Art

[0002] On-the-job training for public security traffic police is a key way to learn new police knowledge and skills and instill new police concepts, which can effectively improve the professional quality and law enforcement level of the majority of public security traffic police. Only scientific and effective on-the-job training practices can meet the requirements of standardization and formalization of public security traffic police law enforcement.

[0003] At present, it is found that there are several problems in the on-the-job training of public security traffic police: first, the degree of attention paid to on-the-job training of public security traffic police is not enough; second, the public security traffic police training system lacks pertinence; third, the training methods are simple; fourth, the setting of training content is unreasonable; fifth, mental health education training is just a formality; sixth, the training assessment and evaluation do not play their due role. On the basis of summarizing the current training status and finding the existing problems, the reasons for these problems are summarized in combination with relevant theories: first, there is a lack of instructors; second, the courses are boring; third, the system is difficult to build; fourth, there are few training opportunities; fifth, there is little time and venue; sixth, there is a lack of pertinence. Finally, on the basis of analyzing the problems and summarizing the reasons, suggestions and feasible measures for improving the training of public security traffic police are put forward. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention proposes a training and assessment simulation method and system based on language and action synchronization, which ensures that traffic police can perform their tasks quickly and accurately in a complex and changeable traffic environment through systematic training and assessment.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A training and assessment simulation method based on language and action synchronization includes the following steps:

[0007] Construct several police-community communication teaching case libraries and an expert library including standard police-community communication features;

[0008] According to the teaching case library of police-civilian communication, a voice database and a virtual police situation model including a face model and an action model are established to enable the virtual police situation model to reproduce the police performance;

[0009] Construct virtual police handling scenarios based on voice database and virtual police situation model;

[0010] The interactive information of the trained police officers interacting with the voice database and the virtual police situation model in the virtual police scene is obtained through the acquisition equipment;

[0011] The similarity between the interactive information and the communication standard features in the expert database is scored to obtain the training and assessment simulation results of the trained police officers.

[0012] Preferably, the method for establishing a virtual police situation model including a face model and an action model comprises:

[0013] Obtain human skeleton data and joint data through a depth camera, establish a mapping relationship between joint data and human skeleton data, and build a motion model;

[0014] The facial feature points are mapped to correspond to each other, and the model is aligned by aligning the video face with the 3D animation virtual face. The face model is transformed using Laplace coordinates to construct the face model.

[0015] Build character models, bind character models, face models and action models to construct a virtual police situation model.

[0016] Preferably, the interactive information obtained by the acquisition device when the trained police officer interacts with the voice database and the virtual police situation model in the virtual police scene includes:

[0017] The language information of the students is collected through voice analysis technology, the body movement information of the students is collected through posture capture technology, and the facial expression information of the students is collected through expression analysis technology.

[0018] Preferably, the method for scoring the similarity between the interaction information and the communication standard features in the expert database includes:

[0019] Analyze language information;

[0020] Analyze body movement information;

[0021] Analyze facial expression information;

[0022] The similarity between the analysis results of language information, body movement information and facial expression information and the communication standard features in the expert database is scored.

[0023] Preferably, the method for analyzing language information includes:

[0024] The trained police officers were analyzed by using the intelligent speech analysis technology based on the auditory attention model with BiLSTM and CNN;

[0025] The trained police officers’ analysis is compared with the standard analysis in the expert database to obtain the analysis matching rate.

[0026] Preferably, the method for analyzing body movement information includes:

[0027] The original video data is decomposed into continuous frames by using the frame extraction method and then sent to the action feature extraction network that integrates human body area information for training, so as to identify the body language information of the trained police officers during the interaction process;

[0028] The body language information is analyzed through action semantic analysis technology to obtain the semantics of the police trainees;

[0029] The semantics of the trained police officers are compared with the standard action categories in the expert database to obtain the action similarity.

[0030] Preferably, the method for analyzing facial expression information comprises:

[0031] The VGG-19 network in the improved expression classifier model was used to identify the facial expression features of the trained police officers during the interaction process;

[0032] The facial capture technology and expression analysis technology are used to extract the expression information of the trained police officers during the interaction process;

[0033] Compare the expression information with the standard expression categories in the expert database to obtain the expression similarity.

[0034] The present invention also discloses a training and assessment simulation system based on language and action synchronization, comprising: a first building module, a second building module, a third building module, an interaction module and an evaluation module;

[0035] The first construction module is used to construct several police-civilian communication teaching case libraries and an expert library including standard police-civilian communication features;

[0036] The second construction module is used to establish a voice database and a virtual police situation model including a face model and an action model according to the police-civilian communication teaching case library, so that the virtual police situation model can reproduce the police performance;

[0037] The third construction module is used to construct a virtual police scene based on the voice database and the virtual police situation model;

[0038] The interaction module is used to obtain the interaction information of the trained police officers when they interact with the voice database and the virtual police situation model in the virtual police scene through the acquisition device;

[0039] The evaluation module is used to score the similarity between the interactive information and the communication standard features in the expert database to obtain the training assessment simulation results of the trained police officers.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention discloses a training and assessment simulation method based on language and action synchronization, which constructs several police-civilian communication teaching case libraries and an expert library including standard police-civilian communication features; establishes a voice database and a virtual police situation model including a face model and an action model according to the police-civilian communication teaching case library, so that the virtual police situation model can reproduce the police handling performance; constructs a virtual police handling scene based on the voice database and the virtual police situation model; obtains the interactive information of the trained police officers when they interact with the voice database and the virtual police situation model in the virtual police handling scene through a collection device; scores the similarity between the interactive information and the communication standard features in the expert library, and obtains the training and assessment simulation results of the trained police officers. The present invention ensures that traffic police can perform tasks quickly and accurately in a complex and changeable traffic environment through systematic training and assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 The present invention is a flowchart of a training and assessment simulation method based on language and action synchronization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Embodiment 1

[0047] like Figure 1 As shown, the present invention provides a training and assessment simulation method based on language and action synchronization, comprising the following steps:

[0048] Construct several police-community communication teaching case libraries and an expert library including standard police-community communication features;

[0049] According to the teaching case library of police-civilian communication, a voice database and a virtual police situation model including a face model and an action model are established to enable the virtual police situation model to reproduce the police performance;

[0050] Construct virtual police handling scenarios based on voice database and virtual police situation model;

[0051] The interactive information of the trained police officers interacting with the voice database and the virtual police situation model in the virtual police scene is obtained through the acquisition equipment;

[0052] The similarity between the interactive information and the communication standard features in the expert database is scored to obtain the training and assessment simulation results of the trained police officers.

[0053] In this embodiment, the established voice database is: traffic police should respect the parties when enforcing the law and accepting requests from the masses, use civilized, polite, and standardized language, and speak in a solemn and peaceful tone. If the parties do not understand, they should patiently explain and should not scold or ridicule the parties. 1. When stating their identity: Hello, we are police officers from the XX Traffic Police Brigade. We are now conducting routine inspections in accordance with the law. Please cooperate. 2. When requiring motor vehicle drivers to turn off the engine or get off the vehicle for inspection: Please park the motor vehicle at XX and turn off the engine (turn off the engine and get off the vehicle) for inspection. 3. When checking the motor vehicle driver's license and vehicle license of motor vehicle drivers suspected of illegal behavior: Hello! Please show your driver's license and vehicle license. For those who refuse to show their driver's license and vehicle license: Failure to show their driver's license and vehicle license will be deemed as not being carried, and the motor vehicle will be detained according to law. 4. When correcting the illegal behavior of offenders (including motor vehicle drivers, non-motor vehicle drivers, pedestrians, and passengers, the same below), warning and educating them: Your (list specific illegal behaviors) violates road traffic safety laws and regulations, please abide by traffic regulations. Thank you for your cooperation. 5. When a pedestrian or non-motor vehicle driver is fined on the spot for illegal behavior: Your (list specific illegal behavior) violates the road traffic safety laws and regulations. According to Article XX of the Road Traffic Safety Law and Article XX of the Regulations for the Implementation of the Road Traffic Safety Law (or XX local regulations), you will be fined XX yuan on the spot. When a non-motor vehicle driver refuses to pay the fine: According to Article 89 of the Road Traffic Safety Law, if you refuse to accept the fine, your non-motor vehicle may be detained. 6. When a motor vehicle driver is fined on the spot or administrative compulsory measures are taken: Your (list specific illegal behavior) violates the road traffic safety laws and regulations. According to Article XX of the Road Traffic Safety Law and Article XX of the Regulations for the Implementation of the Road Traffic Safety Law (or XX local regulations), you will be fined XX yuan and XX points will be recorded (or your driver's license / motor vehicle will be detained). 7. Before the implementation of administrative penalties or administrative compulsory measures, when informing the offender of his or her rights: You have the right to make statements and defenses. 8. When raising an objection to an administrative penalty or administrative compulsory measure: If you have an objection to a penalty decision or administrative compulsory measure, you may apply for administrative reconsideration or initiate administrative litigation in accordance with the law. 9. When requiring the offender to sign the administrative penalty decision (or administrative compulsory measure certificate): Please read these contents of the legal document carefully and sign at the signature area. For those who refuse to sign or accept: According to the law, if you refuse to sign or accept, the legal document will still take effect and be served. 10. After the offender is dealt with in accordance with the law: Please keep the legal document (and certificate). 11. When no illegal behavior is found after inspection: Thank you for your cooperation. 12. For motor vehicle drivers who are required to pay fines to the bank according to regulations and propose to pay the fines on the spot: According to the law, we cannot collect the fines on the spot. Please go to XXX Bank or pay through online banking (or other payment methods).13. When the motor vehicle driver refuses to sign the penalty decision or administrative compulsory measure certificate: According to the law, if you refuse to sign or refuse to accept, the legal document will also take effect and be served. 14. When the party is required to stop the motor vehicle to the roadside for processing: Please stop the motor vehicle at (point out the parking location) to accept processing. 15. Implementing traffic control, performing traffic guard tasks, and maintaining traffic order at the scene of the traffic accident: Traffic control is being implemented ahead (there is a traffic guard task or a traffic accident has occurred), please detour XXX road (or wait patiently). 16. Crowds watching: Traffic police are enforcing the law, please do not watch, and irrelevant personnel please leave. 17. For those who are watching and taking pictures: Please keep a distance of more than 3 meters from the scene. You are not allowed to cross the cordon to enter the on-site inspection center area to take pictures. 18. If the person being inspected asks to check the "People's Police Certificate": I'm sorry, according to regulations, the "People's Police Certificate" can only be opened and shown to you, and cannot be handed over to you for inspection. I am a police officer of the XX Traffic Police Brigade, my name is XX, my police number is XX, if you have any questions, you can verify with my unit. 19. When the motor vehicle driver and other relevant personnel ask the traffic police to answer the phone: Hello, according to relevant regulations, you cannot answer the phone at will while on duty, please understand. 20. The party concerned questions what law has been violated: You are suspected of ×x, the traffic police are investigating, please cooperate. 21. The party concerned questions what evidence is there: please cooperate, we are investigating according to the law. 22. Summons and warning: You are suspected of xx, now summoned according to the law, please cooperate, if you do not accept the summons without a legitimate reason, we will force the summons. 23. Verbal summons: You are suspected of ×x, now summoned to the XX detachment and brigade to cooperate with the investigation. 24. Stop the non-violent implementation of crimes: I am a traffic police, immediately stop the illegal and criminal behavior and accept inspection. 25. Accuse or threaten to complain to the police: You can report to the relevant departments afterwards, this is your right; but the traffic police are enforcing the law now, please cooperate! 26. For those who interfere with normal law enforcement and disrupt the order on the scene: Your behavior has interfered with the police's law enforcement, and is suspected of obstructing the people's police from performing their duties according to law. It has been fully recorded by the law enforcement recorder. Please stop immediately. If you do not listen to the warning, you will bear legal responsibility according to law. 27. Use violent acts such as pushing, pulling, kicking and beating to hinder law enforcement: Don't move! Your behavior is suspected of obstructing the performance of duties. Stop immediately, otherwise, compulsory measures will be taken against you according to law. 28. When there are reporters interviewing on the scene: Thank you for your attention and support to our work. Sorry, I am enforcing the law. You must obtain the consent of our unit's publicity department if you want to interview. This is our discipline. Please understand. Thank you! For those who do not listen to the advice and continue to request an interview: Sorry. I cannot accept your interview without the approval of our unit's publicity department. Your behavior will interfere with the normal law enforcement of the police. Please go outside the cordon. Please cooperate.

[0054] Thanks!

[0055] In this embodiment, the method for establishing a virtual police situation model including a face model and an action model includes:

[0056] Obtain human skeleton data and joint data through a depth camera, establish a mapping relationship between joint data and human skeleton data, and build a motion model;

[0057] The facial feature points are mapped to correspond to each other, and the model is aligned by aligning the video face with the 3D animation virtual face. The face model is transformed using Laplace coordinates to construct the face model.

[0058] Build character models, bind character models, face models and action models to construct a virtual police situation model.

[0059] In this embodiment, based on the voice database and the virtual police situation model, the method for constructing a virtual police scene includes:

[0060] Based on the voice database and virtual police situation model, obtain standardized law enforcement training videos;

[0061] The police trainees will watch the standardized law enforcement training video from a third-person perspective. By watching the demonstration video, the police trainees will have a preliminary understanding of the entire police situation;

[0062] The police trainees will face the law enforcement officers from a first-person perspective. At the same time, the system will provide corresponding prompts for law enforcement language and action essentials. Keywords will be displayed in a special font color, and the police trainees can learn according to the corresponding language and action prompts.

[0063] The system will turn on the camera and record audio and video of the entire training process of the police trainees. At the same time, it will automatically score the training results to generate test scores. The police trainees can check the scores through the scoring details.

[0064] In this embodiment, the interactive information obtained by the acquisition device when the trained police officer interacts with the voice database and the virtual police situation model in the virtual police scene includes:

[0065] The language information of the students is collected through voice analysis technology, the body movement information of the students is collected through posture capture technology, and the facial expression information of the students is collected through expression analysis technology.

[0066] In this embodiment, the method for scoring the similarity between the interaction information and the communication standard features in the expert database includes:

[0067] Analyze language information;

[0068] Analyze body movement information;

[0069] Analyze facial expression information;

[0070] The similarity between the analysis results of language information, body movement information and facial expression information and the communication standard features in the expert database is scored.

[0071] Among them, the method for analyzing language information includes:

[0072] The trained police officers were analyzed by using the intelligent speech analysis technology based on the auditory attention model with BiLSTM and CNN;

[0073] The trained police officers’ analysis is compared with the standard analysis in the expert database to obtain the analysis matching rate.

[0074] Specifically, an end-to-end classifier CNN-BiLSTM is introduced, taking the original image as input. The CNN network is regarded as a feature extractor, which extracts the high-level representation of the input; then BiLSTM is used to obtain the temporal information. The entire structure is trained in an end-to-end manner.

[0075] In the bidirectional long short-term memory (BLSTM) network deep learning model, the attention mechanism and emotional symbols are introduced to build a dual attention model. The convolutional neural network model is used to extract local features. When the convolutional neural network is used for text classification, word2vec is first used to convert the word w(i) into the corresponding word vector v(w(i)), and the sentence composed of the word w(i) is mapped into the sentence matrix S j .

[0076] V(W(i))∈R k Represents the sentence matrix S j The i-th word vector in is a K-dimensional word vector, S j ∈R m×k ,m represents the sentence matrix S j The number of sentences in the sentence matrix S j The vector matrix used as the embedding layer of the convolutional neural network language model. The sentence matrix S j = {V (w(1)) , V (w(2)) ,…,V (w(m))}. The convolutional layer uses a filter of size r×k to S j Perform convolution operation to extract S j local features.

[0077] C i =f(F·V(W(i:i+r-1))+b) (1)

[0078] Among them, F represents the r×k filter; b represents the bias; f represents the function of nonlinear operation through RELU; V(W(i:i+r-1)) represents S jThere are r row vectors from i to i+r-1; C i Represents the local features obtained by the convolution operation. As the filter slides from top to bottom with a step size of 1, it passes through the entire S j , and finally get the local feature vector set C:

[0079] C={c1,c2,…c r-h+1} (2)

[0080] The maximum pooling method is used to extract the feature with the largest value from the local features obtained by the convolution operation to replace the entire local feature. The pooling operation can greatly reduce the size of the feature vector. All the features obtained after pooling are combined in the fully connected layer to output the vector U:

[0081] U={d1,d2,…,d n} (3)

[0082] Finally, the output U of the fully connected layer is input into the softmax classifier for classification. The model uses the labels in the actual classification and optimizes the parameters through the back propagation algorithm.

[0083] P{y|U, W, b}=softmax(F·U+b) (4)

[0084] The method for analyzing body movement information includes:

[0085] The original video data is decomposed into continuous frames by using the frame extraction method and then sent to the action feature extraction network that integrates human body area information for training, so as to identify the body language information of the trained police officers during the interaction process;

[0086] The body language information is analyzed through action semantic analysis technology to obtain the semantics of the police trainees;

[0087] The semantics of the trained police officers are compared with the standard action categories in the expert database to obtain the action similarity.

[0088] Specifically, the present invention selects C3D network to perform recognition and extraction of action information. Each convolution layer of C3D network is followed by a pooling layer, two fully connected layers are used to flatten the high-dimensional feature vector, and finally a Softmax classifier is used to obtain the final video label.

[0089] The original video data is decomposed into continuous frames by using the frame extraction method and then sent to the network for training, and then the action categories are identified. The frame extraction operation in the action recognition process is carried out simultaneously with the network operation process, and the network inputs 16 frames of images at a time. Each video has 6 possible results, and the probability of identifying each action category is (0, 1). The category with the highest confidence score in the Softmax layer is taken as the final output category, and the final recognition result is visualized on the input video.

[0090] The present invention requires the network to pay more attention to the foreground information and appropriately reduce the weight of the background information. Therefore, in order to explore more effective features of body movements and prevent the network from paying too much attention to background information and interfering with the recognition accuracy, the present invention proposes a motion feature extraction network that integrates human body area information. The network has two inputs, one is the original input of the C3D network, and the other is the video frame image that only retains the human body area obtained by processing the video data through the YOLOv3 human body detection network. Both types of input data are sent to the C3D network for training, and the two inputs are fused when passing through the first fully connected layer of the C3D network. The fused features are subjected to a 3D convolution operation again, and finally connected to a fully connected layer to output the features.

[0091] Among them, the method for analyzing facial expression information includes:

[0092] The VGG-19 network in the improved expression classifier model was used to identify the facial expression features of the trained police officers during the interaction process;

[0093] The facial capture technology and expression analysis technology are used to extract the expression information of the trained police officers during the interaction process;

[0094] Compare the expression information with the standard expression categories in the expert database to obtain the expression similarity.

[0095] Specifically, the present invention makes two adjustments to the VGG-19 network in the expression classifier model. First, a random inactivation mechanism (dropout mechanism) is introduced between the last convolutional layer and the fully connected layer, so that the deep neural network no longer activates all neurons during the forward propagation process, but randomly discards a part of the neurons with a certain probability and activates another part of the neurons. By adding the dropout mechanism, the probability of overfitting in the neural network can be greatly reduced, the generalization ability of the neural network can be improved, and the operation speed of the network can be greatly accelerated. Second, the network structure of VGG-19 is modified, and the last multiple fully connected layers of the original network are reduced to 1, and the Softmax loss function is used in the last fully connected layer to speed up the network convergence speed, and finally complete the seven classifications of facial expressions. The calculation formula of the Softmax loss function is:

[0096]

[0097] Among them, N represents the size of the batch imported data, n represents the number of sample categories, W represents the weight value of the jth column, and b j represents the deviation value of the jth column, x i Represents the deep feature of the i-th sample, y i Represents the category of the i-th sample.

[0098] Embodiment 2

[0099] The present invention also discloses a training and assessment simulation system based on language and action synchronization, comprising: a first building module, a second building module, a third building module, an interaction module and an evaluation module;

[0100] The first building module is used to build several police-civilian communication teaching case libraries and an expert library including standard police-civilian communication features;

[0101] The second construction module is used to establish a voice database and a virtual police situation model including a face model and an action model based on the police-civilian communication teaching case library, so that the virtual police situation model can reproduce the police performance;

[0102] The third building module is used to build a virtual police scene based on the voice database and the virtual police situation model;

[0103] The interactive module is used to obtain the interactive information of the trained police officers interacting with the voice database and the virtual police situation model in the virtual police scene through the acquisition equipment;

[0104] The evaluation module is used to score the similarity between the interactive information and the communication standard features in the expert database to obtain the training assessment simulation results of the trained police officers.

[0105] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A training and assessment simulation method based on language and action synchronization, characterized in that: The following steps are involved: Construct several police-community communication teaching case libraries and an expert library including standard police-community communication features; According to the teaching case library of police-civilian communication, a voice database and a virtual police situation model including a face model and an action model are established to enable the virtual police situation model to reproduce the police performance; Construct virtual police handling scenarios based on voice database and virtual police situation model; The interactive information of the trained police officers interacting with the voice database and the virtual police situation model in the virtual police scene is obtained through the acquisition equipment; The similarity between the interactive information and the communication standard features in the expert database is scored to obtain the training and assessment simulation results of the trained police officers.

2. The training and assessment simulation method based on language and action synchronization according to claim 1 is characterized in that: The method for establishing a virtual police situation model including a face model and an action model includes: Obtain human skeleton data and joint data through a depth camera, establish a mapping relationship between joint data and human skeleton data, and build a motion model; The facial feature points are mapped to correspond to each other, and the model is aligned by aligning the video face with the 3D animation virtual face. The face model is transformed using Laplace coordinates to construct the face model. Build character models, bind character models, face models and action models to construct a virtual police situation model.

3. The training and assessment simulation method based on language and action synchronization according to claim 1 is characterized in that: The interactive information obtained by the acquisition device when the trained police officers interact with the voice database and the virtual police situation model in the virtual police scene includes: The language information of the students is collected through voice analysis technology, the body movement information of the students is collected through posture capture technology, and the facial expression information of the students is collected through expression analysis technology.

4. The training and assessment simulation method based on language and action synchronization according to claim 3 is characterized in that: Methods for scoring the similarity between the interaction information and the communication standard features in the expert database include: Analyze language information; Analyze body movement information; Analyze facial expression information; The similarity between the analysis results of language information, body movement information and facial expression information and the communication standard features in the expert database is scored.

5. The training and assessment simulation method based on language and action synchronization according to claim 1 is characterized in that: Methods for analyzing language information include: The trained police officers were analyzed by using the intelligent speech analysis technology based on the auditory attention model with BiLSTM and CNN; The trained police officers’ analysis is compared with the standard analysis in the expert database to obtain the analysis matching rate.

6. The training and assessment simulation method based on language and action synchronization according to claim 1 is characterized in that: Methods for analyzing body movement information include: The original video data is decomposed into continuous frames by using the frame extraction method and then sent to the action feature extraction network that integrates human body area information for training, so as to identify the body language information of the trained police officers during the interaction process; The body language information is analyzed through action semantic analysis technology to obtain the semantics of the police trainees; The semantics of the trained police officers are compared with the standard action categories in the expert database to obtain the action similarity.

7. The training and assessment simulation method based on language and action synchronization according to claim 1 is characterized in that: Methods for analyzing facial expression information include: The VGG-19 network in the improved expression classifier model was used to identify the facial expression features of the trained police officers during the interaction process; The facial capture technology and expression analysis technology are used to extract the expression information of the trained police officers during the interaction process; Compare the expression information with the standard expression categories in the expert database to obtain the expression similarity.

8. A training and assessment simulation system based on language and action synchronization, characterized in that: include: A first building module, a second building module, a third building module, an interaction module and an evaluation module; The first construction module is used to construct several police-civilian communication teaching case libraries and an expert library including standard police-civilian communication features; The second construction module is used to establish a voice database and a virtual police situation model including a face model and an action model according to the police-civilian communication teaching case library, so that the virtual police situation model can reproduce the police performance; The third construction module is used to construct a virtual police scene based on the voice database and the virtual police situation model; The interaction module is used to obtain the interaction information of the trained police officers when they interact with the voice database and the virtual police situation model in the virtual police scene through the acquisition device; The evaluation module is used to score the similarity between the interactive information and the communication standard features in the expert database to obtain the training assessment simulation results of the trained police officers.

Images