A non-contact virtual-real interactive simulation system and method for GMP pharmaceutical production

Abstract

This invention belongs to the field of intelligent training technology in the pharmaceutical industry, and provides a non-contact virtual-real interactive simulation system and method for GMP drug production. It includes: a login authentication module for authenticating trainees' identities based on the virtual-real interactive simulation platform; after successful authentication, trainees can select a training mode; a startup module for activating a motion capture module and a real-time evaluation module after the virtual-real interactive simulation platform detects the selected training mode; a recognition module for capturing and recognizing trainees' actions in real time based on the motion capture module, and synchronizing with the virtual scene of a high-fidelity drug production line model in the training mode to obtain virtual environment feedback; a compliance judgment module for determining whether the trainee's operational compliance based on the virtual environment feedback exceeds a preset compliance level based on the real-time evaluation module; and a report generation module for recording training data under virtual environment feedback and generating an evaluation report stored in a database. This improves training efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent training technology in the pharmaceutical industry, and proposes a non-contact virtual-real interactive simulation system and method for GMP drug production. Background Technology

[0002] Currently, in the field of GMP simulation training systems, the main products use 3D virtual simulation technology to build virtual simulation work scenarios for pharmaceutical manufacturing enterprises to achieve functions such as equipment cognition, pharmaceutical process learning, and process assessment. However, the following defects exist: the system relies on traditional input devices such as keyboards and mice for operation, the interaction method is mechanical, lacks natural body movement interaction, and it is difficult to provide an immersive experience. Existing optical motion capture systems require the placement of complex marker points, the equipment deployment is cumbersome, and it is not suitable for rapid application in training scenarios. It lacks the ability to recognize refined gestures for the special needs of pharmaceutical production, and cannot meet the requirements for accurate recognition of professional actions in GMP compliance operations, resulting in low training efficiency.

[0003] Therefore, the present invention provides a non-contact virtual-real interactive simulation system and method for GMP pharmaceutical production. Summary of the Invention

[0004] This invention provides a non-contact virtual-real interactive simulation system and method for GMP pharmaceutical production, in order to solve the aforementioned technical problems.

[0005] This invention provides a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, comprising:

[0006] The login authentication module is used to authenticate the student's identity based on the virtual-real interactive simulation platform. After successful authentication, the student can select a training mode, which includes: equipment disassembly and assembly training mode, operation process training mode, and abnormal handling training mode.

[0007] The startup module is used to start the motion capture module and the real-time evaluation module when the virtual-real interaction simulation platform captures the training mode selected by the trainee. The real-time evaluation module is related to the GMP operation specification knowledge base and the temporal convolutional network.

[0008] The recognition module is used to capture and recognize the trainee's behavior in real time based on the motion capture module, and synchronize with the virtual scene of the high-fidelity pharmaceutical production line model in the training mode to obtain virtual environment feedback. The behavior parsing is based on the gesture recognition model.

[0009] The compliance judgment module is used to determine, based on the real-time evaluation module, whether the compliance of the student's operation based on feedback from the virtual environment is greater than the preset compliance.

[0010] If yes, continue the process; if no, provide error messages and corrections.

[0011] The report generation module is used to record the training data under the feedback of the virtual environment and generate an evaluation report stored in the database. The training data is the initial behavioral data of each training stage under the training mode and the behavioral data adjusted according to the correction information after each update.

[0012] Preferably, the login authentication module includes:

[0013] The information acquisition unit is used to acquire the waiting login information set of the student's user terminal based on the virtual interactive simulation platform, wherein the waiting login information set includes: the device version of the user terminal, the user account for logging into the virtual interactive platform, and a two-dimensional authentication array, wherein the two-dimensional authentication array includes a global authentication vector and a random local authentication vector;

[0014] The version determination unit is used to determine whether the device version is the latest version. If so, it determines that the device version verification has passed.

[0015] The account determination unit is used to match the stored account in the account database to see if it is completely consistent with the user account. If it exists, the user account is determined to have passed the verification.

[0016] An array authentication unit is used to preprocess the two-dimensional authentication array and determine whether the two-dimensional authentication array has passed verification.

[0017] Once all verifications are successful, the student is allowed to log in to the virtual interactive simulation platform.

[0018] Preferably, the array authentication unit includes:

[0019] The graph construction subunit is used to obtain the historical successful authentication set of verified user accounts, and to determine the average range of input habit probability and the range of input probability variance based on the authentication information input habits under each historical successful authentication, and to construct an account login behavior graph. The authentication information input habits include: input speed and input force.

[0020] The habit matching subunit is used to capture the student's current input habits and match them with the account login behavior graph. If the match is successful, the verification of the two-dimensional authentication array continues.

[0021] The encoding subunit is used to encode and transform each layer of authentication information in the global authentication vector according to the layered authentication strategy, and to randomly arrange all the transformation encoding orders involved in the global authentication vector to obtain the first encoding group. The encoding transformation is related to the encoding length and transformation symbol set according to the number of characters in each layer of authentication information and the layer weight of the authentication layer.

[0022] The encoding and parsing subunit is used to perform encoding and parsing on the random local authentication vector to obtain supplementary encoding;

[0023] A combination subunit is used to combine the supplementary code with the first code group to obtain a second code group;

[0024] The encoding matching subunit is used to match the second encoding group with the storage encoding group in the login storage database. If the match is successful, it is determined that the global authentication vector and the random local authentication vector have been verified.

[0025] Otherwise, output a verification error message;

[0026] If the matching fails, the function to generate a QR code is sent to the virtual interactive simulation platform.

[0027] The face verification subunit is used to send the QR code function to the user terminal for face verification when the virtual interactive simulation platform receives a QR code verification request from the user terminal within a preset time period.

[0028] Preferably, the identification module includes:

[0029] The process acquisition unit is used to acquire the training process in the training mode selected by the trainee, wherein the training process includes at least one training step.

[0030] A non-touch display unit is used to guide the trainee to perform corresponding actual actions according to the training steps in the virtual scene of the high-fidelity pharmaceutical production line model in the training mode, and to display the actual actions guided at each captured guidance time point on the non-touch screen.

[0031] The action judgment unit is used to identify the displayed action based on the gesture recognition model and compare it with the standard action of the current pre-embedded trigger point in the corresponding training session. The comparison analysis result is the virtual environment feedback.

[0032] Preferably, the compliance judgment module includes:

[0033] The sequence extraction unit is used to extract the subsequence Xr={d} within a time window ΔT before and after the action time at the current pre-embedded point. t-ΔT ,...,dt ,...,d t+ΔT}, where d t-ΔT d t d t+ΔT These represent the datasets based on the current pre-embedded trigger point at time t-ΔT, time t, and time t+ΔT, respectively, and d t =s t A t}, and A t ={a1 t ,...,an t}, s t A represents the spatial coordinates at time t; t a1 represents the action feature vector at time t; t 、an t Let represent the features based on the first dimension and the nth dimension at time t, respectively;

[0034] The network analysis unit is used to input the subsequence into a temporal convolutional network and output an action difference feature vector E1.

[0035] The compliance calculation unit is used to retrieve the standard action feature matrix Bs corresponding to the current pre-embedded trigger point from the GMP operation specification knowledge base and calculate the operation compliance Qs.

[0036]

[0037] Where ∝1 and α2 represent weights respectively; Mu(Xr,Cs) represents the matching degree function between Xr and the trigger condition set Cs of the current pre-embedded trigger point; Np represents the number of trigger conditions that Xr completely matches the trigger condition set Cs of the current pre-embedded trigger point; N0 represents the total number of trigger conditions existing in the trigger condition set Cs of the current pre-embedded trigger point; P i1 This represents the probability of Xr successfully matching the i1th trigger condition; 0.8 is the preset probability threshold, and the trigger condition set Cs is related to the combination of action speed, force, and device status parameters;

[0038] The advancement unit is used to ignore the current pre-embedded trigger point and continue to advance the next pre-embedded trigger point of the training session when the compliance of the operation is greater than the preset compliance.

[0039] The correction unit is used to mark the current pre-embedded trigger point when the compliance of the operation is not greater than the preset compliance, and to determine the correction information and correction reminder based on the current pre-embedded trigger point according to the comparison analysis results.

[0040] Preferably, the compliance judgment module further includes:

[0041] The temporal correlation unit is used to determine the temporal correlation characteristics based on the set independence coefficient and set difficulty index of each marked trigger point under the corresponding training stage;

[0042] The number of times determination unit is used to obtain the historical operation information of the trainee in the corresponding training session, calculate the attention weight of each marked trigger point through the attention mechanism, and generate the initial number of additional training sessions for the corresponding marked trigger point.

[0043] The feature extraction unit is used to input the historical operation information into the convolutional neural network, extract the operation sequence features of the trainee, and input them into the classifier to determine the training mode. The training modes include: repetitive mode, individual reinforcement mode, and hybrid mode.

[0044] The number of times update unit is used to collect the action data of each labeled trigger point in real time during the training process, and dynamically adjust the remaining number of times of the corresponding labeled trigger point through the temporal correlation characteristics. When the corresponding labeled trigger point is detected to have failed to trigger for m0 consecutive times, the training number of the trigger point is automatically increased by m0+1 times. At the same time, action decomposition teaching is performed.

[0045] Among them, the maximum number of initial additional training sessions under the corresponding training stage is used as the number of cycles in the loop mode;

[0046] The hybrid mode uses the minimum number of retentions among all retention counts in the corresponding training phase as the first n1 cycles, and uses the remaining counts after n1 cycles based on the retention count as separate reinforcements, with the retention counts obtained by relying on the synergy coefficient.

[0047] Preferred options also include:

[0048] The collaboration determination module is used to determine the historical collaboration coefficients of adjacent annotation trigger points under H random historical training tests, starting from the first annotation trigger point under the corresponding training stage.

[0049]

[0050] in, Represents the single-test collaboration coefficient between the z-th annotation trigger point and the z+1-th annotation trigger point in the j1-th historical training test; Count(R) j1,z ∩R j1,z+1 ) represents the number of overlapping trigger events between the z-th and z+1-th labeled trigger points in the j1-th historical training test; Count(R) j1,z ∪R j1,z+1 () represents the sum of the number of all trigger events under the z-th and z+1-th labeled trigger points in the j1-th historical training test; Nk represents the total number of labeled trigger points in the corresponding training stage;

[0051] The retention judgment module is used to retain the pair of annotation trigger points with the larger initial training count if the historical coordination coefficient is greater than the preset coefficient; otherwise, it determines that the two annotation trigger points under the combination with the historical coordination coefficient not greater than the preset coefficient will not be combined, and the corresponding initial training count will be retained independently.

[0052] This invention provides a non-contact virtual-real interactive simulation method for pharmaceutical production, comprising:

[0053] Step 1: Log in and authenticate the student's identity based on the virtual-real interactive simulation platform. After successful authentication, the student can select a training mode, which includes: equipment disassembly and assembly training mode, operation process training mode, and abnormal handling training mode.

[0054] Step 2: After the virtual-real interaction simulation platform captures the training mode selected by the trainee, it starts the motion capture module and the real-time evaluation module. The real-time evaluation module is related to the GMP operation specification knowledge base and the temporal convolutional network.

[0055] Step 3: The motion capture module captures and identifies the trainee's actions in real time, and synchronizes them with the virtual scene of the high-fidelity pharmaceutical production line model in the training mode to obtain virtual environment feedback. The action parsing is based on a gesture recognition model.

[0056] Step 4: Determine whether the compliance of the student's operation based on feedback from the virtual environment, as determined by the real-time evaluation module, is greater than the preset compliance.

[0057] If yes, continue the process; if no, provide error messages and corrections.

[0058] Step 5: Record the training data under the feedback of the virtual environment and generate an evaluation report to be stored in the database. The training data is the initial behavioral data of each training stage under the training mode and the behavioral data adjusted according to the correction information after each update.

[0059] Compared with the prior art, the beneficial effects of this application are as follows:

[0060] Based on non-contact gesture operation, trainees can complete the assembly and disassembly of virtual devices without their eyes, reducing operation latency and enhancing the training immersion. Combined with the GMP knowledge base and TCN network, it can effectively identify typical violations in real time. By recording the entire process operation data, it can generate targeted evaluation reports and support repeated training, effectively improving training efficiency.

[0061] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0062] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0063] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0064] Figure 1 This is a structural diagram of a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, as described in an embodiment of the present invention.

[0065] Figure 2 This is a flowchart of a non-contact virtual-real interactive simulation method for GMP pharmaceutical production in an embodiment of the present invention;

[0066] Figure 3 This is a structural diagram of the system in an embodiment of the present invention. Detailed Implementation

[0067] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0068] This invention provides a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, such as... Figure 1 As shown, it includes:

[0069] The login authentication module is used to authenticate the student's identity based on the virtual-real interactive simulation platform. After successful authentication, the student can select a training mode, which includes: equipment disassembly and assembly training mode, operation process training mode, and abnormal handling training mode.

[0070] Equipment disassembly and assembly training mode, such as disassembling the main components of a fluidized bed granulator:

[0071] (1) Start disassembly: Extend your index finger forward and bring your other limbs together. The hand movement position can be displayed on the non-touch screen. After the index finger moves to the [Start Disassembly] button on the screen, the simulation system enters the pelleting workshop.

[0072] (2) Equipment inspection: With your five fingers together, slide your hand left and right in the air to rotate and inspect the appearance of the fluidized bed granulator;

[0073] (3) Take tools: After finding the air inlet, you can select the screw at the connection position between the air inlet and the main body of the equipment by gesture. Understand the specifications and size of the screw according to the prompts in the simulation scenario. Then, take the screwdriver of the corresponding specification in the simulation scenario by spreading your five fingers and then making a fist.

[0074] (4) Component disassembly: Move the screwdriver to the screw position of the air inlet, rotate it with a fist, and complete the disassembly of the air inlet screws in the simulation scenario.

[0075] (5) The disassembly of components such as air inlet filter, exhaust pipe, dust bag or cyclone separator can be completed in the above manner.

[0076] Practical training mode for operational processes, such as lyophilization process operation for lyophilized powder injections:

[0077] (1) Pre-production inspection: With palms together, move horizontally in one direction. Move with hand operation in the simulation scenario to complete the indoor cleanliness inspection. Select the PLC operation screen in the virtual environment by making a fist and extending the index finger. The system will pop up a complete operation interface. Check in detail whether the instrument and meter values ​​of the equipment on the interface are normal.

[0078] (2) Start freeze drying: Locate the freeze drying chamber door by moving, grasp the door handle in the virtual environment by extending your arm forward and making a fist, rotate to open, put the filled intermediate product into the freeze drying chamber, close the door, and start the freeze dryer remotely by making a fist and extending your index finger on the PLC operation screen.

[0079] (3) Pre-freezing: Set the temperature drop time and low temperature holding time of the control shelf by selecting remotely;

[0080] (4) One-time drying: The condenser is turned on remotely by making a fist and extending the index finger on the PLC operation screen to cool down. The cooling temperature, vacuum pressure inside the chamber, number of times the shelf temperature is raised and the duration of each temperature rise, and the duration of temperature holding after the temperature rise are set.

[0081] (5) Secondary drying: On the PLC operation screen, the temperature of the plate is set remotely by making a fist and extending the index finger, setting the temperature rise time, the heating time, and the holding time after heating, etc., to control the temperature of the sample. After drying, the plate is pressed.

[0082] The startup module is used to start the motion capture module and the real-time evaluation module when the virtual-real interaction simulation platform captures the training mode selected by the trainee. The real-time evaluation module is related to the GMP operation specification knowledge base and the temporal convolutional network.

[0083] The recognition module is used to capture and recognize the trainee's behavior in real time based on the motion capture module, and synchronize with the virtual scene of the high-fidelity pharmaceutical production line model in the training mode to obtain virtual environment feedback. The behavior parsing is based on the gesture recognition model.

[0084] The compliance judgment module is used to determine, based on the real-time evaluation module, whether the compliance of the student's operation based on feedback from the virtual environment is greater than the preset compliance.

[0085] If yes, continue the process; if no, provide error messages and corrections.

[0086] The report generation module is used to record the training data under the feedback of the virtual environment and generate an evaluation report stored in the database. The training data is the initial behavioral data of each training stage under the training mode and the behavioral data adjusted according to the correction information after each update.

[0087] In this embodiment, the virtual-real interactive simulation platform is a software system that integrates functions such as virtual simulation, motion capture, and data processing. It is used to construct virtual scenes of drug production and realize human-computer interaction. Specifically, it is developed based on the Unity3D / UnrealEngine4 engine, integrates 3D modeling technology to construct a high-fidelity virtual environment for drug production lines, and connects to hardware devices (such as LeapMotion) through API interfaces. For example, the platform can simulate the three-dimensional structure of a fluidized bed granulator, and trainees can rotate the equipment in the virtual scene to view details through gesture operations.

[0088] In this embodiment, the training mode is divided into different scenario types according to the training objectives, including three categories: equipment disassembly and assembly, process operation, and abnormal handling. These are presented as visual buttons on the platform interface. After clicking, the virtual model and operation process of the corresponding scenario are loaded. For example, after selecting the equipment disassembly and assembly training mode, the virtual scenario switches to the lyophilized powder injection equipment disassembly interface, which displays a list of operable parts.

[0089] In this embodiment, the motion capture module collects the student's limb movement data through hardware devices (such as LeapMotion) and converts it into computer-recognizable instructions. LeapMotion captures the coordinates of key points of the hand at a sampling rate of 200Hz and recognizes the gesture posture through a deep learning model (such as MediaPipe). For example, if the student makes a fist gesture, the module recognizes it as a tool grabbing instruction, and a screwdriver model in the virtual scene is selected.

[0090] In this embodiment, the real-time evaluation module is a system that evaluates and scores trainees' operations in real time based on GMP specifications. It is linked to a knowledge base and a temporal convolutional network (TCN). For example, the GMP operation specification knowledge base stores data such as GMP standard operation procedures and parameter thresholds (e.g., the pre-freezing temperature of the freeze-drying process needs to be controlled between -40℃ and -35℃). The temporal convolutional network processes action sequence data and identifies the compliance of the time sequence of operation steps (e.g., disassembling screws must be done in a diagonal order). For example, if a trainee does not operate in the correct order when disassembling the screws of the fluidized bed granulator, the TCN detects the time sequence abnormality and determines compliance.

[0091] In this embodiment, action recognition involves parsing the captured action data and mapping it to operation instructions in a virtual scene. Specifically, based on a gesture recognition model specifically designed for pharmaceutical production (such as training 15 standard gestures), a CNN network is used to classify the coordinates of hand skeletal points. For example, if a student spreads their five fingers and then makes a fist, the model recognizes it as picking up a tool, and a screwdriver of the corresponding specification is invoked in the virtual scene.

[0092] In this embodiment, the high-fidelity pharmaceutical production line model is a three-dimensional virtual model that accurately restores the size, structure, and interaction logic of real equipment. Specifically, it involves using 3D modeling software (such as Maya) to scan real equipment, importing it into Unity3D, and setting physical engine parameters (such as screw rotation torque). For example, the model of the freeze-drying box for freeze-dried powder injections can simulate the resistance to opening the real box door, and the force of the gesture rotation needs to match the actual operation.

[0093] In this embodiment, the virtual environment feedback is based on the comparative analysis results of the trainee's actions between the virtual scene and standard actions.

[0094] Operational compliance is the result of evaluating trainees' actions in a virtual scenario. For example, if a trainee ends the cooling process prematurely during the pre-freezing stage, the operational compliance score is 0.4, and the system determines that the actions taken during the pre-freezing time do not meet the standards.

[0095] A preset compliance threshold is set as the standard value for measuring whether an operation is qualified (e.g., setting compliance ≥ 80% as passing). For example, in the abnormal handling assessment, if the trainee's operation compliance is 75%, which is higher than the preset 70% (preset compliance threshold), the process continues.

[0096] In this embodiment, the evaluation report is a visual analysis document generated based on training data, which includes scores, weaknesses, and improvement suggestions. Specifically, it uses a Python data visualization library (such as Matplotlib) to generate charts and combines natural language processing to generate suggested text. For example, the report indicates that the student scored 60 points in the equipment disassembly and assembly sequence and suggests strengthening training on the diagonal disassembly principle.

[0097] In this embodiment, the specific framework diagram is as follows: Figure 3 As shown.

[0098] The beneficial effects of the above technical solution are as follows: Based on non-contact gesture operation, trainees can complete the assembly and disassembly of virtual devices without their eyes, reducing operation latency and enhancing the sense of immersion in training. Combined with the GMP knowledge base and TCN network, typical violations can be effectively identified in real time. By recording the entire process operation data, targeted evaluation reports can be generated. At the same time, repeated training is supported, which can effectively improve training efficiency.

[0099] This invention provides a contactless virtual-real interactive simulation system for pharmaceutical production, wherein the login authentication module includes:

[0100] The information acquisition unit is used to acquire the waiting login information set of the student's user terminal based on the virtual interactive simulation platform, wherein the waiting login information set includes: the device version of the user terminal, the user account for logging into the virtual interactive platform, and a two-dimensional authentication array, wherein the two-dimensional authentication array includes a global authentication vector and a random local authentication vector;

[0101] The version determination unit is used to determine whether the device version is the latest version. If so, it determines that the device version verification has passed.

[0102] The account determination unit is used to match the stored account in the account database to see if it is completely consistent with the user account. If it exists, the user account is determined to have passed the verification.

[0103] An array authentication unit is used to preprocess the two-dimensional authentication array and determine whether the two-dimensional authentication array has passed verification.

[0104] Once all verifications are successful, the student is allowed to log in to the virtual interactive simulation platform.

[0105] In this embodiment, the virtual interactive simulation platform user terminal is the terminal software operated by the student, used to connect to the virtual simulation system and send interactive commands. Specifically, it is a client program developed based on Unity3D, supporting Windows / macOS systems, and communicating with the server through network protocols (such as WebSocket). For example, when the student opens the drug production simulation client on the computer, the interface displays the login entrance.

[0106] In this embodiment, the device version is the version number of the user-end software. For example, the client program has a built-in version number field (such as AssemblyVersion), which is automatically read and sent to the server when it starts. If the latest version of the server is V2.3.5 and the student's client version is V2.3.4, an upgrade prompt will be given.

[0107] In this embodiment, the user account is a unique identifier for the student in the system. It is stored in the user_account table of the MySQL database in string format (such as employee ID / student ID). For example, the account PHARM2024001 corresponds to student Zhang San.

[0108] In this embodiment, the two-dimensional authentication array is a combined array containing global and local authentication vectors to enhance authentication security. Specifically, the global authentication vector (GAV) is a fixed authentication code generated based on account registration information, and the random local authentication vector (RLA) is a random string dynamically generated each time you log in. For example, the two-dimensional authentication array is [GAV_abc123, RLA_def456], where GAV is fixed and RLA is different each time you log in.

[0109] In this embodiment, the account database stores student accounts, passwords, and other information.

[0110] The beneficial effects of the above technical solution are: preventing unauthorized login based on multi-dimensional security verification.

[0111] This invention provides a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, wherein the array authentication unit includes:

[0112] The graph construction subunit is used to obtain the historical successful authentication set of verified user accounts, and to determine the average range of input habit probability and the range of input probability variance based on the authentication information input habits under each historical successful authentication, and to construct an account login behavior graph. The authentication information input habits include: input speed and input force.

[0113] The habit matching subunit is used to capture the student's current input habits and match them with the account login behavior graph. If the match is successful, the verification of the two-dimensional authentication array continues.

[0114] The encoding subunit is used to encode and transform each layer of authentication information in the global authentication vector according to the layered authentication strategy, and to randomly arrange all the transformation encoding orders involved in the global authentication vector to obtain the first encoding group. The encoding transformation is related to the encoding length and transformation symbol set according to the number of characters in each layer of authentication information and the layer weight of the authentication layer.

[0115] The encoding and parsing subunit is used to perform encoding and parsing on the random local authentication vector to obtain supplementary encoding;

[0116] A combination subunit is used to combine the supplementary code with the first code group to obtain a second code group;

[0117] The encoding matching subunit is used to match the second encoding group with the storage encoding group in the login storage database. If the match is successful, it is determined that the global authentication vector and the random local authentication vector have been verified.

[0118] Otherwise, output a verification error message;

[0119] If the matching fails, the function to generate a QR code is sent to the virtual interactive simulation platform.

[0120] The face verification subunit is used to send the QR code function to the user terminal for face verification when the virtual interactive simulation platform receives a QR code verification request from the user terminal within a preset time period.

[0121] In this embodiment, the historical successful authentication set is a collection of authentication records of the student's past successful logins, used to analyze behavioral habits. Specifically, the database login_history table stores data such as the time, input speed, and intensity of each login. For example, the input speed of student STU001 in the past 10 logins was between 8 and 12 characters / second, forming a historical set.

[0122] In this embodiment, the authentication information input habit refers to the behavioral characteristics of the user when inputting authentication information. For example, when a student enters a password, they tend to type quickly and continuously (10 characters / second) with moderate force (60% pressure).

[0123] In this embodiment, the average range of the input habit probability is the mean of the input speed ± standard deviation (μ ± σ).

[0124] The variance of the input habit probability ranges from 0 to 3σ.

[0125] In this embodiment, the account login behavior graph is a user input habit model presented in a visual way for real-time behavior matching. Specifically, it uses multi-dimensional vector representation (such as speed, intensity, and input order) and builds a machine learning model through TensorFlow. For example, the graph shows that the input speed of student STU001 is concentrated in 8-12 characters / second and the intensity is concentrated in 50%-70%, forming a behavioral feature range.

[0126] In this embodiment, the current input habit is the real-time input behavior data of the student during this login.

[0127] In this embodiment, the hierarchical authentication strategy splits the global authentication vector into layers, with each layer using different encoding rules. Specifically, the layers are divided according to the importance of the authentication information, and independent encoding rules are set for each layer. For example, the global authentication vector GAV_abc123 is split into two layers: the first layer is abc (3 characters, layer weight 0.6), and the second layer is 123 (3 characters, layer weight 0.4).

[0128] Encoding conversion is based on the number of characters and the layer weight to calculate the encoding length, and then selects the conversion symbol (such as ASCII offset or hash function). Specifically, the encoding length is calculated as follows: Encoding length = number of characters × layer weight (e.g., if the layer weight is 0.6 and the number of characters is 3, the encoding length is 3 × 0.6 = 1.8, which is rounded down to 2).

[0129] The conversion symbol is used to convert the character 'a' to the ASCII value of the a+ layer weight (a→97+0.6=97.6, rounded down to 98→'b'). For example, in the first layer, 'abc' is converted to 'bdf' (each character +0.6 and rounded down), with an encoding length of 2, resulting in 'bd'; in the second layer, '123' is converted to '234', with an encoding length of 2, resulting in '23'.

[0130] In this embodiment, random order arrangement means shuffling the order of the converted code to increase the difficulty of cracking (equivalent to improving login security). For example, if the converted code is [bd,23], the first code group [23,bd] is obtained after random arrangement.

[0131] In this embodiment, the supplementary encoding is to decode the random local authentication vector to generate supplementary verification information. Specifically, the RLA is parsed using Base64 decoding or a custom algorithm (such as XOR operation). For example, RLAdef456 is decoded as 100101102525354 (ASCII value) as supplementary encoding.

[0132] In this embodiment, the second coding group is a combination of the first coding group and the supplementary coding, used for final verification, for example: first coding group [23,bd] + supplementary coding [100,101] → second coding group [23,bd,100,101].

[0133] In this embodiment, the login storage database is a database that stores legitimate code groups for comparison and verification.

[0134] The QR code function is a program function that generates a QR code image containing verification information. After scanning the QR code, the user is redirected to a face verification page.

[0135] The beneficial effects of the above technical solution are as follows: By constructing a behavior graph through historical login data, abnormal operations (such as sudden changes in input speed or abnormal force) can be identified, and the system can still allow normal login through probability range tolerance. The hierarchical encoding and random arrangement of the global authentication vector, combined with dynamic RLA supplementary encoding, makes the verification sequence unique for each login. When verification fails, a QR code is automatically triggered for quick face verification. The scanning is completed within a preset time period (such as 30 seconds), reducing repeated input steps and improving login efficiency.

[0136] This invention provides a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, wherein the identification module includes:

[0137] The process acquisition unit is used to acquire the training process in the training mode selected by the trainee, wherein the training process includes at least one training step.

[0138] A non-touch display unit is used to guide the trainee to perform corresponding actual actions according to the training steps in the virtual scene of the high-fidelity pharmaceutical production line model in the training mode, and to display the actual actions guided at each captured guidance time point on the non-touch screen.

[0139] The action judgment unit is used to identify the displayed action based on the gesture recognition model and compare it with the standard action of the current pre-embedded trigger point in the corresponding training session. The comparison analysis result is the virtual environment feedback.

[0140] In this embodiment, for example, when a trainee selects the equipment disassembly and assembly training mode, the system loads a file that defines the training process for disassembling a fluidized bed granulator, and the training steps include: starting disassembly, tool selection, screw removal, etc.

[0141] The training session guidance uses visual and auditory prompts to guide trainees in performing the operational tasks of the current session. For example, in the tool retrieval session, a yellow arrow appears next to the air inlet screw in the virtual scene, prompting the trainee to click here to select the screwdriver.

[0142] The guidance time points are the key operation time nodes preset in the training process, which are used to trigger motion capture and feedback. For example, the guidance time points for the screw disassembly process include: t0 = aligning with the screw hole, t1 = starting to rotate, and t2 = disassembly completed.

[0143] The actual motion display involves mapping the limb movements captured by the trainee through motion capture equipment to the movement of the hand / tool ​​model in the virtual scene in real time. Specifically, LeapMotion captures the coordinates of key points of the hand (such as the position of the fingertip of the index finger), and the InverseKinematics algorithm calculates the joint angles of the virtual arm to drive the model's movement. For example, when the trainee extends and moves his index finger in reality, the index finger model in the virtual scene synchronously displays the movement trajectory on a non-touchscreen, pointing to the start disassembly button.

[0144] The gesture recognition model is a deep learning-based algorithm used to classify captured motion data into standard gestures (such as grasping, rotating, and sliding). Specifically, it uses the MediaPipe framework to train a hand keypoint detection model. The input is the coordinates of 21 hand joints, and the output is the gesture category (such as grasping a tool with a confidence of 0.95). For example, if a student spreads their five fingers and then makes a fist, the model recognizes it as a grasping gesture, which corresponds to the operation of grasping a screwdriver in a virtual scene.

[0145] The pre-embedded trigger points are key operation positions or states preset in the training process, used to trigger action verification (such as screw hole position, parameter setting threshold). For example, in the screw disassembly process, a trigger point is pre-embedded at the screw hole position. When the tip of the virtual screwdriver enters the area, the system begins to compare the trainee's rotation action with the standard action.

[0146] In this embodiment, the standard action is an operation action template that conforms to GMP specifications, including parameters such as action trajectory, force, and timing. For example, the standard action is to rotate the screwdriver clockwise by 90° in 1.5 seconds with uniform rotation force, which is stored in file 1 in the database.

[0147] Comparative analysis quantifies and compares various parameters of the trainee's real-time actions with those of the standard actions, generating virtual environment feedback, including trajectory comparison and time sequence comparison. For example, if the trainee rotates the screwdriver at an angle of 85° and takes 1.8 seconds, compared with the standard action (90°, 1.5 seconds), the distance deviation is 5° and the time deviation is 0.3 seconds, which is judged as basically qualified.

[0148] The beneficial effects of the above technical solution are as follows: by comparing pre-embedded trigger points with standard actions, millimeter-level action accuracy calibration is achieved, ensuring that trainees' operations comply with GMP standards. The complex training process is broken down into independent links, and each link provides real-time action visualization and feedback. Trainees can gradually master skills such as equipment disassembly and assembly, and process operation. Based on the gesture recognition model and standard action library, the system can automatically identify typical operational errors. By recording the action comparison data at each guidance time point, a trainee operation weakness analysis report is generated, providing a basis for personalized training.

[0149] This invention provides a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, wherein the compliance judgment module includes:

[0150] The sequence extraction unit is used to extract the subsequence Xr={d} within a time window ΔT before and after the action time at the current pre-embedded point. t-ΔT ,...,d t ,...,d t+ΔT}, where d t-ΔT d t d t+ΔT These represent the datasets based on the current pre-embedded trigger point at time t-ΔT, time t, and time t+ΔT, respectively, and d t ={s t A t}, and A t ={a1 t ,...,an t}, s t A represents the spatial coordinates at time t; t a1 represents the action feature vector at time t;t 、an t Let represent the features based on the first dimension and the nth dimension at time t, respectively;

[0151] The network analysis unit is used to input the subsequence into a temporal convolutional network and output an action difference feature vector E1.

[0152] The compliance calculation unit is used to retrieve the standard action feature matrix Bs corresponding to the current pre-embedded trigger point from the GMP operation specification knowledge base and calculate the operation compliance Qs.

[0153]

[0154] Where ∝1 and α2 represent weights respectively; Mu(Xr,Cs) represents the matching degree function between Xr and the trigger condition set Cs of the current pre-embedded trigger point; Np represents the number of trigger conditions that Xr completely matches the trigger condition set Cs of the current pre-embedded trigger point; N0 represents the total number of trigger conditions existing in the trigger condition set Cs of the current pre-embedded trigger point; P i1 This represents the probability of Xr successfully matching the i1th trigger condition; 0.8 is the preset probability threshold, and the trigger condition set Cs is related to the combination of action speed, force, and device status parameters;

[0155] The advancement unit is used to ignore the current pre-embedded trigger point and continue to advance the next pre-embedded trigger point of the training session when the compliance of the operation is greater than the preset compliance.

[0156] The correction unit is used to mark the current pre-embedded trigger point when the compliance of the operation is not greater than the preset compliance, and to determine the correction information and correction reminder based on the current pre-embedded trigger point according to the comparison analysis results.

[0157] In this embodiment, the boundary between valid and invalid matches is set at 0.8 as a probability threshold. This threshold is used to determine whether the overall matching degree of the triggering conditions is close to acceptable. The threshold of 0.8 implies that key conditions must be met first, and the overall matching degree must reach more than 80% to be considered a valid operation. This is consistent with the management requirement in actual production that if key constraints are not met, the operation is considered a violation.

[0158] Drug manufacturing operations allow for a certain margin of error. Adjusting e-0.5 ensures that when some conditions are not met but the function is close to the threshold, the matching degree will not drop sharply to 0, but will decay slowly (e.g., when the matching probability difference is 0.1, the matching degree drops from 0.8 to 0.6). This is more in line with human tolerance for errors when approaching compliant operations. The logarithmic term becomes ln(e-0.5+1 / 2)=ln(e)=1, ensuring that the function outputs a reasonable value when in the critical state (the matching probability just reaches the threshold), avoiding computational overflow or logical contradictions.

[0159] In this embodiment, for example, in the pre-freezing stage of lyophilized powder injection, the pre-embedded trigger point is when the temperature of the shelf drops to -40°C. When the virtual system detects that the temperature has reached this value, it triggers the action verification.

[0160] The action moment is the point in time when the student's operation is captured and is associated with the pre-embedded trigger point. For example, if the student completes the action of aligning the screwdriver with the screw hole at t=10.5s, this moment is marked as the trigger point action moment.

[0161] A time window is a time interval centered on the moment of an action and extending before and after it, used to capture continuous action data sequences.

[0162] A subsequence is a collection of motion data within a time window, containing dimensional information such as spatial coordinates and motion feature vectors, and is related to hand coordinates, speed, force, and angle of finger joints.

[0163] The motion difference feature vector E1 is a vector output by TCN, representing the degree of difference between the trainee's motion and the standard motion. Each dimension corresponds to the deviation of different features. For example, E1 = [0.2, 0.1, 0.3, ...] represents the difference values ​​of dimensions such as speed, force, and angle.

[0164] In this embodiment, the GMP operation specification knowledge base is a database that stores data such as GMP standard operation procedures, parameter thresholds, and action templates, providing a benchmark for compliance assessment. It adopts a relational database (such as MySQL) to store standard action feature matrices, trigger condition sets, etc. For example, the standard action feature matrix Bs for disassembling the screws of the boiling granulator in the knowledge base includes parameters such as rotation angle (90°), speed (5cm / s), and force (70%).

[0165] The standard action feature matrix Bs is a standard action template corresponding to the pre-embedded trigger point. It stores the standard values ​​and tolerance ranges of multi-dimensional features in matrix form. For example, the first column of Bs corresponds to the standard value of rotation angle 90° with a tolerance of ±5°; the second column corresponds to the standard value of speed 5cm / s with a tolerance of ±1cm / s.

[0166] Operational compliance is a quantitative indicator that the trainee's actions conform to GMP standards. It is calculated from the action difference feature vector and the standard matrix, and the trigger condition set Cs is used. For example, the trigger condition set Cs for screw removal is: speed 5-10cm / s, force ≥60%, rotation angle = 90°.

[0167] The probability of a successful match is Pi1. For example, if the standard speed is 5-10 cm / s and the student's speed is 6 cm / s, then Pi1...

[0168] = (6-5) / (10-5) = 0.2; If the student's speed is 4cm / s, then Pi1 = 0.

[0169] The preset compliance threshold is the standard value for judging whether an operation is qualified. It can be adjusted according to the training stage, for example, 0.7.

[0170] Ignoring the current pre-embedded trigger point means that when the operation is compliant, the current trigger point verification is skipped and the next step is continued. When the operation is non-compliant, the problem trigger point is marked and specific correction suggestions are generated. For example, if the trainee rotates the angle only 80°, the compliance rate is 65% < 70%. The system marks the screw part in red and prompts that the current angle is 80°, the standard is 90° ± 5°, and the rotation angle should be increased.

[0171] The beneficial effects of the above technical solution are as follows: By capturing the time dependency of action sequences through the TCN network, subtle violation patterns such as fast at the beginning and slow at the end can be identified. By combining the action difference vector with the matching degree of trigger conditions, quantitative verification of GMP specifications can be achieved. Non-compliant operations are immediately marked with problem trigger points and provided with concrete corrective suggestions. Trainees can intuitively understand the cause of errors in a virtual scenario. Training sessions can be automatically advanced or paused based on compliance, preventing trainees from skipping key operation steps. By recording the compliance data of each trigger point, a trainee competency map is generated, providing data support for personalized training plans.

[0172] This invention provides a non-contact virtual-real interactive simulation system for GMP pharmaceutical production, wherein the compliance judgment module further includes:

[0173] The temporal correlation unit is used to determine the temporal correlation characteristics based on the set independence coefficient and set difficulty index of each marked trigger point under the corresponding training stage;

[0174] The number of times determination unit is used to obtain the historical operation information of the trainee in the corresponding training session, calculate the attention weight of each marked trigger point through the attention mechanism, and generate the initial number of additional training sessions for the corresponding marked trigger point.

[0175] The feature extraction unit is used to input the historical operation information into the convolutional neural network, extract the operation sequence features of the trainee, and input them into the classifier to determine the training mode. The training modes include: repetitive mode, individual reinforcement mode, and hybrid mode.

[0176] The number of times update unit is used to collect the action data of each labeled trigger point in real time during the training process, and dynamically adjust the remaining number of times of the corresponding labeled trigger point through the temporal correlation characteristics. When the corresponding labeled trigger point is detected to have failed to trigger for m0 consecutive times, the training number of the trigger point is automatically increased by m0+1 times. At the same time, action decomposition teaching is performed.

[0177] Among them, the maximum number of initial additional training sessions under the corresponding training stage is used as the number of cycles in the loop mode;

[0178] The hybrid mode uses the minimum number of retentions among all retention counts in the corresponding training phase as the first n1 cycles, and uses the remaining counts after n1 cycles based on the retention count as separate reinforcements, with the retention counts obtained by relying on the synergy coefficient.

[0179] Preferred options also include:

[0180] The collaboration determination module is used to determine the historical collaboration coefficients of adjacent annotation trigger points under H random historical training tests, starting from the first annotation trigger point under the corresponding training stage.

[0181]

[0182] in, Represents the single-test collaboration coefficient between the z-th annotation trigger point and the z+1-th annotation trigger point in the j1-th historical training test; Count(R) j1,z ∩R j1,z+1 ) represents the number of overlapping trigger events between the z-th and z+1-th labeled trigger points in the j1-th historical training test; Count(R) j1,z ∪R j1,z+1 () represents the sum of the number of all trigger events under the z-th and z+1-th labeled trigger points in the j1-th historical training test; Nk represents the total number of labeled trigger points in the corresponding training stage;

[0183] The retention judgment module is used to retain the pair of annotation trigger points with the larger initial training count if the historical coordination coefficient is greater than the preset coefficient; otherwise, it determines that the two annotation trigger points under the combination with the historical coordination coefficient not greater than the preset coefficient will not be combined, and the corresponding initial training count will be retained independently.

[0184] In this embodiment, the marked trigger point is a key operation node in the training process that is judged to be non-compliant and requires intensive training, including device interaction points, parameter verification points, etc.

[0185] The independence coefficient is a parameter used to measure the independent training value of labeled trigger points, reflecting the strength of the correlation between the trigger point and other nodes (a higher independence coefficient indicates a weaker correlation). Specifically, it is assigned by expert experience or calculated through historical data (such as the frequency of the trigger point appearing alone), ranging from 0 to 1 (an independence coefficient of 0.8 indicates a weak correlation). For example, the emergency stop button trigger point has strong independence (independence coefficient = 0.9) because it has low correlation with other operations; the screw removal trigger point has weak independence (independence coefficient = 0.3) because it is often associated with tool selection.

[0186] The difficulty index is a metric that quantifies the difficulty of the operation at the labeled trigger point. It is calculated based on the historical error rate and operational complexity, specifically: Difficulty Index = Number of Historical Errors / Total Number of Historical Training Sessions.

[0187] The temporal correlation characteristic marks the temporal order dependency between trigger points, reflecting the sequential logic and correlation of the operation process. Specifically, it constructs a directed acyclic graph (DAG), where nodes are trigger points and edge weights are the product of independence coefficients and difficulty indices. The larger the weight, the stronger the temporal correlation. For example, the edge weight of tool selection (TP002) → screw removal (TP001) = 0.3 (TP001 independence coefficient) × 3 (TP001 difficulty index) = 0.9, indicating that the two trigger points are closely temporally correlated.

[0188] Historical operation information is the operation data generated during the trainee's past training, including trigger point interaction records, error types, operation time, etc. For example, trainee A's historical operation information: TP001 3 times, 2 times compliant; TP002 5 times, 1 time compliant.

[0189] The attention mechanism is an algorithm that simulates human attention allocation, highlighting the influence of key trigger points in historical operations (e.g., high error rate trigger points have high weights). Specifically, it uses the attention layer of Transformer, with the input being the sequence of historical operations and the output being the attention weights of the trigger points (e.g., Softmax normalization). For example, if TP001 has a historical error rate of 67% (2 / 3) and TP002 has an error rate of 80% (4 / 5), the attention mechanism calculates that the weight of TP002 is 0.6 and the weight of TP001 is 0.4.

[0190] In this embodiment, the attention weight is the importance of the trigger point in the historical operation and is used to allocate training resources (a higher weight means more training iterations). Specifically, it is the normalized coefficient of the attention mechanism output, ranging from 0 to 1 (weight = 0.6 means that the trigger point needs 60% of the training resources). For example, TP001 weight = 0.4, TP002 weight = 0.6, and when the total training resources are 10 times, the initial training iterations of TP001 = 4 and TP002 = 6.

[0191] The initial number of training sessions is the number of trigger point reinforcement training sessions initially determined based on the attention weight, without considering the synergistic relationship. Specifically, the initial number of training sessions = total number of training sessions × attention weight, rounded down (e.g., if the total number of sessions is 10 and the weight is 0.6, then the number of sessions = 6).

[0192] A convolutional neural network is a 3-layer CNN. The input is a time-feature matrix of historical operation sequences (such as operation timestamps and compliance tags), and the output is a 128-dimensional feature vector.

[0193] In this embodiment, the operation sequence features represent the abstract features of the student's operation habits and error patterns, which are used to determine the training mode. For example, feature vector clustering shows that the operation sequence of student A is similar to the repeated error type sample, and it is determined that the loop mode is required.

[0194] The classifier is a pattern discrimination model based on machine learning, which outputs training modes (recurrent, individual reinforcement, and hybrid). Specifically, it trains an SVM classifier with operation sequence features as input and pattern labels as output (0 = recurrent, 1 = individual reinforcement, 2 = hybrid).

[0195] In this embodiment, the motion data is the trigger point operation data collected in real time during the training process, including motion trajectory, force, timing, etc., capturing the coordinates of key hand points at 200 frames per second, and storing it as a sequence of [x,y,z,timestamp].

[0196] The remaining number of times is the number of additional training attempts that were not completed at the trigger point, and is a dynamically adjusted value (e.g., 6 times initially, 4 times after adjustment).

[0197] The number of consecutive failed triggers, m0, is the number of times the trigger point fails compliance verification consecutively. Action decomposition teaching is a visual teaching method that breaks down complex operations into sub-steps to help students correct errors.

[0198] The randomized H historical training tests are randomly selected historical training records used to calculate the collaborative relationships between trigger points (e.g., H = 100 tests) to ensure coverage of different trainees and scenarios.

[0199] Historical co-operation coefficient measures the frequency with which two labeled trigger points co-occur and coordinate in historical tests. A preset coefficient (e.g., 0.6) is a threshold used to determine if the co-operation coefficient is high enough to decide whether to merge trigger point training. By calculating the historical co-operation coefficient between trigger points, the operational nodes in virtual training are bound to the logical connections of real production. This avoids isolated practice by trainees (e.g., practicing screw disassembly without understanding the importance of tool selection) and makes the training process more closely resemble the continuous connections in actual production.

[0200] The beneficial effects of the above technical solution are as follows: by using temporal correlation characteristics and attention mechanisms, training resources are prioritized for trigger points with high difficulty and high error rate; based on real-time action data and continuous failure detection, the number of training iterations is automatically adjusted and the teaching is decomposed; by filtering through historical coordination coefficients, training of trigger points with strong correlation is merged to reduce repetitive training; CNN+SVM is used to identify trainees' operation patterns to achieve personalized training; all operation data is fed back to the model; and historical coordination coefficients are updated every quarter to ensure that the training strategy fits the actual production needs.

[0201] This invention provides a non-contact virtual-real interactive simulation method for GMP pharmaceutical production, such as... Figure 2 As shown, it includes:

[0202] Step 1: Log in and authenticate the student's identity based on the virtual-real interactive simulation platform. After successful authentication, the student can select a training mode, which includes: equipment disassembly and assembly training mode, operation process training mode, and abnormal handling training mode.

[0203] Step 2: After the virtual-real interaction simulation platform captures the training mode selected by the trainee, it starts the motion capture module and the real-time evaluation module. The real-time evaluation module is related to the GMP operation specification knowledge base and the temporal convolutional network.

[0204] Step 3: The motion capture module captures and identifies the trainee's actions in real time, and synchronizes them with the virtual scene of the high-fidelity pharmaceutical production line model in the training mode to obtain virtual environment feedback. The action parsing is based on a gesture recognition model.

[0205] Step 4: Determine whether the compliance of the student's operation based on feedback from the virtual environment, as determined by the real-time evaluation module, is greater than the preset compliance.

[0206] If yes, continue the process; if no, provide error messages and corrections.

[0207] Step 5: Record the training data under the feedback of the virtual environment and generate an evaluation report to be stored in the database. The training data is the initial behavioral data of each training stage under the training mode and the behavioral data adjusted according to the correction information after each update.

[0208] The beneficial effects of the above technical solution are as follows: Based on non-contact gesture operation, trainees can complete the assembly and disassembly of virtual devices without their eyes, reducing operation latency and enhancing the sense of immersion in training. Combined with the GMP knowledge base and TCN network, typical violations can be effectively identified in real time. By recording the entire process operation data, targeted evaluation reports can be generated. At the same time, repeated training is supported, which can effectively improve training efficiency.

[0209] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A non-contact virtual-real interactive simulation system for GMP pharmaceutical production, characterized in that, include: The login authentication module is used to authenticate the student's identity based on the virtual-real interactive simulation platform. After successful authentication, the student can select a training mode, which includes: equipment disassembly and assembly training mode, operation process training mode, and abnormal handling training mode. The startup module is used to start the motion capture module and the real-time evaluation module when the virtual-real interaction simulation platform captures the training mode selected by the trainee. The real-time evaluation module is related to the GMP operation specification knowledge base and the temporal convolutional network. The recognition module is used to capture and recognize the trainee's behavior in real time based on the motion capture module, and synchronize with the virtual scene of the high-fidelity pharmaceutical production line model in the training mode to obtain virtual environment feedback. The behavior parsing is based on the gesture recognition model. The compliance judgment module is used to determine, based on the real-time evaluation module, whether the compliance of the student's operation based on feedback from the virtual environment is greater than the preset compliance. If yes, continue the process; if no, provide error messages and corrections. The compliance assessment module includes: The sequence extraction unit is used to extract the action moments before and after the current pre-embedded trigger point. Subsequence within the next time window ,in, , These respectively represent the time based on the currently embedded trigger point. Time t, time t The following dataset, and ,and , Represents the spatial coordinates at time t; This represents the action feature vector at time t; Let represent the features based on the first dimension and the nth dimension at time t, respectively; The network analysis unit is used to input the subsequence into a temporal convolutional network and output an action difference feature vector E1. The compliance calculation unit is used to retrieve the standard action feature matrix Bs corresponding to the current pre-embedded trigger point from the GMP operation specification knowledge base and calculate the operation compliance. ; ； ； in, , These represent the weights respectively; express The trigger condition set of the currently pre-embedded trigger point The matching degree function; This represents the number of trigger conditions that Xr successfully matches the trigger condition set Cs of the currently embedded trigger point; This represents the total number of trigger conditions existing in the trigger condition set Cs of the currently embedded trigger point; express The probability of a successful match with the i1th triggering condition; The preset probability threshold and the triggering condition set It is related to the combination of action speed, force, and equipment status parameters; The advancement unit is used to ignore the current pre-embedded trigger point and continue to advance to the next pre-embedded trigger point in the training process when the compliance of the operation is greater than the preset compliance. The correction unit is used to mark the current pre-embedded trigger point when the compliance of the operation is not greater than the preset compliance, and to determine the correction information and correction reminder based on the current pre-embedded trigger point according to the comparison analysis results. The report generation module is used to record the training data under the feedback of the virtual environment and generate an evaluation report stored in the database. The training data is the initial behavioral data of each training stage under the training mode and the behavioral data adjusted according to the correction information after each update.

2. The GMP pharmaceutical production non-contact virtual-real interactive simulation system according to claim 1, characterized in that, The login authentication module includes: The information acquisition unit is used to acquire the waiting login information set of the student's user terminal based on the virtual reality interaction simulation platform. The waiting login information set includes: the device version of the user terminal, the user account for logging into the virtual reality interaction simulation platform, and a two-dimensional authentication array. The two-dimensional authentication array includes a global authentication vector and a random local authentication vector. The version determination unit is used to determine whether the device version is the latest version. If so, it determines that the device version verification has passed. The account determination unit is used to match the stored account in the account database to see if it is completely consistent with the user account. If it exists, the user account is determined to have passed the verification. An array authentication unit is used to preprocess the two-dimensional authentication array and determine whether the two-dimensional authentication array has passed verification. Once all verifications are successful, the student is allowed to log in to the virtual-real interaction simulation platform.

3. The GMP pharmaceutical production non-contact virtual-real interactive simulation system according to claim 2, characterized in that, The array authentication unit includes: The graph construction subunit is used to obtain the historical successful authentication set of verified user accounts, and to determine the average range of input habit probability and the range of input probability variance based on the authentication information input habits under each historical successful authentication, and to construct an account login behavior graph. The authentication information input habits include: input speed and input force. The habit matching subunit is used to capture the student's current input habits and match them with the account login behavior graph. If the match is successful, the verification of the two-dimensional authentication array continues. The encoding subunit is used to encode and transform each layer of authentication information in the global authentication vector according to the layered authentication strategy, and to randomly arrange all the transformation encoding orders involved in the global authentication vector to obtain the first encoding group. The encoding transformation is related to the encoding length and transformation symbol set according to the number of characters in each layer of authentication information and the layer weight of the authentication layer. The encoding and parsing subunit is used to perform encoding and parsing on the random local authentication vector to obtain supplementary encoding; A combination subunit is used to combine the supplementary code with the first code group to obtain a second code group; The encoding matching subunit is used to match the second encoding group with the storage encoding group in the login storage database. If the match is successful, it is determined that the global authentication vector and the random local authentication vector have been verified. Otherwise, output a verification error message; If the matching fails, the QR code generation function is sent to the virtual-real interaction simulation platform. The face verification subunit is used to send the QR code function to the user terminal for face verification when the virtual-real interaction simulation platform receives a QR code verification request from the user terminal within a preset time period.

4. The GMP pharmaceutical production non-contact virtual-real interactive simulation system according to claim 1, characterized in that, The identification module includes: The process acquisition unit is used to acquire the training process in the training mode selected by the trainee, wherein the training process includes at least one training step. A non-touch display unit is used to guide the trainee to perform corresponding actual actions according to the training steps in the virtual scene of the high-fidelity pharmaceutical production line model in the training mode, and to display the actual actions guided at each captured guidance time point on the non-touch screen. The action judgment unit is used to identify the displayed action based on the gesture recognition model and compare it with the standard action of the current pre-embedded trigger point in the corresponding training session. The comparison analysis result is the virtual environment feedback.

5. The GMP pharmaceutical production non-contact virtual-real interactive simulation system according to claim 1, characterized in that, The compliance assessment module also includes: The temporal correlation unit is used to determine the temporal correlation characteristics based on the set independence coefficient and set difficulty index of each marked trigger point under the corresponding training stage; The number of times determination unit is used to obtain the historical operation information of the trainee in the corresponding training session, calculate the attention weight of each marked trigger point through the attention mechanism, and generate the initial number of additional training sessions for the corresponding marked trigger point. The feature extraction unit is used to input the historical operation information into the convolutional neural network, extract the operation sequence features of the trainee, and input them into the classifier to determine the training mode. The training modes include: repetitive mode, individual reinforcement mode, and hybrid mode. The number of times update unit is used to collect the action data of each labeled trigger point in real time during the training process, and dynamically adjust the remaining number of times of the corresponding labeled trigger point through the temporal correlation characteristics. When the corresponding labeled trigger point is detected to have failed to trigger for m0 consecutive times, the training number of the trigger point is automatically increased by m0+1 times. At the same time, action decomposition teaching is performed. Among them, the maximum number of initial additional training sessions under the corresponding training stage is used as the number of cycles in the loop mode; The hybrid mode uses the minimum number of retentions among all retention counts in the corresponding training phase as the first n1 cycles, and uses the remaining counts after n1 cycles based on the retention count as separate reinforcements, with the retention counts obtained by relying on the synergy coefficient.

6. The GMP pharmaceutical production non-contact virtual-real interactive simulation system according to claim 5, characterized in that, Also includes: The collaboration determination module is used to determine the historical collaboration coefficients of adjacent annotation trigger points under H random historical training tests, starting from the first annotation trigger point under the corresponding training stage. ； in, This represents the single-time collaboration coefficient between the z-th annotation trigger point and the z+1-th annotation trigger point under the j1-th historical training test; This represents the number of overlapping trigger events between the z-th and z+1-th labeled trigger points in the j1-th historical training test. This represents the sum of the number of trigger events at the z-th marked trigger point and the (z+1)-th marked trigger point under the j1-th historical training test; This indicates the total number of marked trigger points in the corresponding training session; The retention judgment module is used to retain the pair of annotation trigger points with the larger initial training count if the historical coordination coefficient is greater than the preset coefficient; otherwise, it determines that the two annotation trigger points under the combination with the historical coordination coefficient not greater than the preset coefficient will not be combined, and the corresponding initial training count will be retained independently.

7. A non-contact virtual-real interactive simulation method for GMP pharmaceutical production, based on the system described in claim 1, characterized in that, include: Step 1: Log in and authenticate the student's identity based on the virtual-real interactive simulation platform. After successful authentication, the student can select a training mode, which includes: equipment disassembly and assembly training mode, operation process training mode, and abnormal handling training mode. Step 2: After the virtual-real interaction simulation platform captures the training mode selected by the trainee, it starts the motion capture module and the real-time evaluation module. The real-time evaluation module is related to the GMP operation specification knowledge base and the temporal convolutional network. Step 3: The motion capture module captures and recognizes the trainee's actions in real time, and synchronizes them with the virtual scene of the high-fidelity pharmaceutical production line model in the training mode to obtain virtual environment feedback. The action parsing is based on a gesture recognition model. Step 4: Determine whether the compliance of the student's operation based on feedback from the virtual environment, as determined by the real-time evaluation module, is greater than the preset compliance. If yes, continue the process; if no, provide error messages and corrections. Step 5: Record the training data under the feedback of the virtual environment and generate an evaluation report to be stored in the database. The training data is the initial behavioral data of each training stage under the training mode and the behavioral data adjusted according to the correction information after each update.

Images