Non-contact operator state judgment method based on fuzzy neural network

Abstract

The invention relates to the technical field of man-machine interaction state perception and intelligent monitoring, and discloses a fuzzy neural network-based non-contact operator state judgment method, which comprises the following steps of: acquiring pupil reaction data by using a non-contact pupil measurement device, covering quantitative and qualitative parameters, and constructing an original data set; carrying out abnormal value elimination, normalization processing and core parameter screening on the original data, and dividing training, testing and verifying subsets; an initial fuzzy rule is generated based on the core parameters, effective rules are reserved through triple screening, a five-layer fuzzy neural network is built, and rule weights and membership function parameters are optimized through a two-stage training algorithm; and finally, inputting the preprocessed real-time pupil data into the trained model, traversing all effective rules to calculate a truth value degree and a reliability score, and outputting an operator state judgment result and natural language interpretation. The problems that a traditional method is single in data adaptation, invisible in judgment result and poor in model generalization ability are solved.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction state perception and intelligent monitoring technology, specifically a non-contact operator state determination method based on fuzzy neural networks. Background Technology

[0002] In human-machine collaborative systems, the operator's functional status directly determines the operational safety of the technical system. Especially in high-risk operational scenarios, abnormal operator status can lead to irreversible consequences. Existing operator status assessment methods are mainly divided into two categories: invasive and non-invasive. Invasive methods require contact sensing devices, are complex to operate, and can easily interfere with the operator's normal work. Among non-invasive methods, non-contact technology based on pupil measurement has become a research hotspot due to its convenience and lack of interference, but it still has significant technical drawbacks.

[0003] Currently, existing operator status assessment methods are mainly divided into two categories: invasive and non-invasive. Invasive methods require contact sensing devices, are complex to operate, and interfere with the operator's normal work. Among non-invasive methods, contactless technology based on pupil measurement has become a research hotspot due to its convenience, but it has significant drawbacks. Traditional non-invasive methods can only process quantitative data and cannot integrate qualitative parameters such as pupil response level and race, resulting in low data utilization. The classification results lack interpretability, making it difficult for operators to understand the system's judgment basis, leading to insufficient trust. Training data often suffers from class imbalance, and traditional sampling methods cannot guarantee dataset balance, resulting in poor model generalization ability. Furthermore, the classic fuzzy neural network structure is difficult to adapt to the needs of heterogeneous data processing, making it difficult to balance inference efficiency and judgment accuracy. To address this, this invention proposes a non-contact operator status judgment method based on fuzzy neural networks. By integrating the advantages of fuzzy logic and neural networks, it processes heterogeneous data and provides interpretable results, generating highly accurate and diverse status judgment results. This provides a reliable solution for operator status monitoring and promotes the development of related technologies. Summary of the Invention

[0004] The purpose of this invention is to provide a non-contact operator status determination method based on fuzzy neural networks to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a non-contact operator status determination method based on a fuzzy neural network, comprising the following steps:

[0006] Step S1: Acquisition of pupillary response data. Using a non-contact pupillary measurement device, pupillary response data of operators of different ages and races are collected under two typical conditions: normal and fatigued.

[0007] Step S2: Data preprocessing. The original pupil response dataset is preprocessed in multiple dimensions to obtain a standardized dataset.

[0008] Step S3: Construct a balanced dataset based on grouped random sampling with replacement. The preprocessed standardized dataset is divided into a balanced training subset, a test subset, and a validation subset using a grouped random sampling algorithm with replacement.

[0009] Step S4: Improved fuzzy inference algorithm;

[0010] Step S5: Construction of the improved fuzzy neural network. By building a core framework of a 5-layer improved fuzzy neural network, the function, computational logic and output format of each layer are clarified.

[0011] Step S6: Improved fuzzy neural network training, using a two-stage training algorithm to optimize network parameters, combining rule-state connection path logic and training effectiveness criteria;

[0012] Step S7: Use the trained fuzzy neural network to determine the operator's status in real time and interpret the results.

[0013] Preferably, the data collected in step S1 is in discrete time series form, containing quantitative and qualitative parameters; the output label is the operator's functional status; the collection process covers all typical states; and the total number of samples in the original dataset is no less than 1500.

[0014] Preferably, step S2 adopts... The criteria identify and remove outliers from the quantitative parameters to ensure data reliability. Next, the filtered quantitative parameters are normalized, mapping the parameter values ​​to the [0,1] interval. Excluding dimensional differences, the normalization formula is as follows:

[0015]

[0016] in, These are the original values ​​of the quantitative parameters. , These are the maximum and minimum values ​​of the parameter in the original dataset, respectively. Finally, the preprocessed core parameter data are stored in a unified format to establish a standardized operator pupil response dataset for subsequent model training and validation.

[0017] As a preferred embodiment, step S3 specifically involves the following steps:

[0018] Initialization parameter: N is the total number of objects in the dataset. ; Number of object states; Define the object's state category; initialize the training subset. test subset Validation subset ;

[0019] According to state B k Divide the preprocessed dataset into Groups Each group has a capacity of N. k ,satisfy ;

[0020] Training subset sampling, for each group Execute in a loop To N k Randomly select objects Copy to training subset middle: ;from Remove those already selected Object: ;

[0021] Test and validation subset sampling, when At that time, randomly select objects Copy to test subset middle: ;from Remove those already selected Object: And randomly select objects. Copy to the verification subset middle: ;from Remove those already selected Object: until ;

[0022] The subsets corresponding to each state group are merged to obtain the final dataset: , , .

[0023] Preferably, step S4 is as follows: First, define the feature representation of the object to be judged: The feature of the object to be judged is represented by an n-dimensional vector. Characterization, where the first g elements are qualitative features, from The elements up to n are quantitative features, and then regular weights are introduced. The larger the weight value, the higher the reliability and priority of the corresponding rule. For rule R... p The improved fuzzy rule design to adapt to heterogeneous data is as follows:

[0024]

[0025] in, The categorical value for a qualitative variable; For a fuzzy set of quantitative variables, its membership function is: ; For object state categories; For the state corresponding to rule Rp The weights; ( (For the total number of rules), define the membership function analogy form for the qualitative variable:

[0026]

[0027] in, That is, when the input qualitative variable value matches the qualitative classification value in the rule, the membership degree is 1; otherwise, it is 0. Rule R p The truth value represents the degree of matching between the input vector and the rule conditions, and the calculation formula is as follows:

[0028]

[0029] in , By taking the minimum value between the membership degrees of the qualitative and quantitative variables, based on rule R... p The truth value and rule weight are used to calculate the solution obtained by applying the rule. The reliability score is calculated using the following formula:

[0030]

[0031] in, The total number of operator status categories, and the rule weight. The system response, i.e., the operator state of the input object, is generated through adaptive calculation via a fuzzy neural network training process, and is obtained by selecting a reliability score. The largest rule serves as the core decision rule, and its output state is... This refers to the judgment state of the input object; simultaneously, the reliability score of this core rule. As a confidence factor, it is output along with the decision state to characterize the reliability of the decision result. Sort in descending order, and the resulting list of rules serves as the core basis for natural language interpretation.

[0032] As a preferred embodiment, the specific implementation process of step S5 is as follows:

[0033] Input layer: Receives the preprocessed core feature vectors ,in This corresponds to four core parameters: D min t c Dhc PuRe has the same number of neurons as the number of core parameters, and no additional data transformation operations are required. It directly passes the feature vector to the next layer.

[0034] Fuzzification layer: Performs fuzzification processing on qualitative and quantitative features, and calculates the membership degree for qualitative features. This achieves binary fuzzy mapping of qualitative classification values; for quantitative features, a trapezoidal membership function is used to calculate the membership degree. Mapping quantitative parameters to Fuzzy sets of intervals;

[0035] Rule Precondition Layer: Calculates R for each fuzzy rule based on the membership degree output by the fuzzification layer. p truth value Truth value characterizes the degree of matching between input features and rule conditions;

[0036] Rule successor layer: Introduces a rule weight matrix to realize reliability score calculation and rule ranking. for A 3D matrix, in the following form:

[0037]

[0038] in, For rule R p Corresponding state The weights are assigned, and a reliability score is calculated for each rule. This comprehensively reflects both the rule matching degree and the reliability of the rule itself, and then constructs a conflict rule set. , and according to Sort in descending order and generate a list of sorted rules. ;

[0039] Deblurring layer: Completes state determination and result output; the network outputs a value pair. Calculated according to the following rules:

[0040]

[0041]

[0042]

[0043]

[0044] in, This is the optimal rule index. For operator status determination results, To determine the credibility of the results, the output includes both a status label and a credibility score.

[0045] As a preferred embodiment, the specific process of step S6 is as follows:

[0046] The first phase of training involves rule weighting. Initialization calculation, in a fuzzy neural network, neuron rule R p With object state The connection corresponds to the rule-state connection path in the network. For each state There are several rule-state connection paths, and each path corresponds to one fuzzy inference rule. The format is as follows:

[0047]

[0048] in , For the output state is Fuzzy rules Total number, if there is no corresponding rule-state connection path in the network. According to the rules, the weighting coefficients If a corresponding rule exists, the weight is calculated using the following formula:

[0049]

[0050] in, For training subset The status label is The number of samples; Qualitative characteristics Corresponding rules Membership degree; For quantitative characteristics Corresponding rules The membership degree is used to calculate the initial rule weight matrix. Then, fix the matrix and proceed to the next training phase;

[0051] The second stage of training optimizes the membership function parameters and fixes the regular weight matrix. Then, the membership function parameters of the quantitative features are iteratively optimized using a genetic algorithm. The optimization process aims at achieving a comprehensive criterion for training effectiveness.

[0052] Introducing a comprehensive criterion for training effectiveness As the optimization objective, it takes the following form:

[0053]

[0054] Simultaneously satisfy the constraints:

[0055]

[0056] in, For training rounds; , Training subsets and test subset The total number of samples; , The first The number of training samples and the number of test samples that are correctly classified in each round of training.

[0057] 8. A non-contact operator status determination method based on a fuzzy neural network according to claim 7, characterized in that: step S7 specifically involves processing the real-time acquired pupil response data through steps S2 and S3, inputting it into the trained model, and obtaining the operator status through the inference algorithm in step S4. and credibility If the state is fatigue, then based on Generate natural language explanation text; save data and inference results, issue anomaly alerts and provide intervention suggestions; if the status is normal, directly output the status and continuously collect data for cyclical judgment.

[0058] Compared with the prior art, the beneficial effects of this invention are as follows:

[0059] 1. The non-contact operator status determination method based on fuzzy neural network described in this invention is proposed for the first time, and combines fuzzy production rules with heterogeneous data fusion technology for real-time monitoring of operator status in complex and dangerous scenarios;

[0060] 2. Compared with the prior art, the judgment method based on the present invention can process quantitative and qualitative data simultaneously, and the judgment results support natural language interpretation, effectively solving the problems of single data adaptation and invisible judgment basis in traditional methods;

[0061] 3. The determination method described in this invention does not require contact sensing devices, is suitable for various high-risk operation scenarios such as aircraft piloting and heavy vehicle operation, significantly improves the practicality and safety of operator status monitoring, and further promotes the development of safety technology for "human-machine" collaborative systems. Attached Figure Description

[0062] Figure 1 This is the overall algorithm flowchart of the method of the present invention;

[0063] Figure 2 This is a diagram of the improved fuzzy neural network architecture of the present invention;

[0064] Figure 3 This is the improved fuzzy neural network path diagram of the present invention.

[0065] Figure 4 This is a schematic diagram illustrating the classification accuracy of the improved fuzzy neural network of this invention on a dataset subset. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example

[0068] Please see Figure 1 The diagram illustrates a non-contact operator status determination method based on a fuzzy neural network, which includes the following steps:

[0069] Step S1: Pupil response data acquisition. Using a non-contact pupil measurement device, pupil response data of operators of different ages and ethnicities are collected under two typical conditions: normal and fatigued. The collected data is in discrete time series format and includes quantitative and qualitative parameters.

[0070] The output labels represent the operator's functional status. The data collection process covers all typical states, and the total number of samples in the original dataset is no less than 1500, ensuring uniform sample distribution and guaranteeing data diversity and representativeness.

[0071] Step S2, data preprocessing: Perform multi-dimensional preprocessing on the original pupillary response dataset to obtain a standardized dataset, and then... The criteria identify and remove outliers from quantitative parameters to ensure data reliability; next, the filtered quantitative parameters are normalized, mapping the parameter values ​​to... For intervals, excluding differences in dimensions, the normalization formula is as follows:

[0072]

[0073] in, These are the original values ​​of the quantitative parameters. , These are the maximum and minimum values ​​of the parameter in the original dataset, respectively. Finally, the preprocessed core parameter data are stored in a unified format to establish a standardized operator pupil response dataset for subsequent model training and validation.

[0074] Step S3: Construct a balanced dataset based on grouped random sampling with replacement. Using a grouped random sampling algorithm with replacement, the preprocessed standardized dataset is divided into balanced training subsets S. tr Test subset Stest With the verification subset S val The specific steps are as follows:

[0075] Initialization parameter: N is the total number of objects in the dataset. ; Number of object states; Define the object's state category; initialize , , ;

[0076] According to state B k Divide the preprocessed dataset into Groups Each group has a capacity of N. k ,satisfy ;

[0077] Training subset sampling, for each group Execute in a loop To N k Randomly select objects Copy to training subset middle: ;from Remove those already selected Object: ;

[0078] Test and validation subset sampling, when At that time, randomly select objects Copy to test subset middle: ;from Remove those already selected Object: And randomly select objects. Copy to the verification subset middle: ;from Remove those already selected Object: until ;

[0079] The subsets corresponding to each state group are merged to obtain the final dataset: , , ;

[0080] The sampling method achieves dataset balance through probabilistic selection: individual objects are selected. The probability is Therefore Contains approximately 63.2% of unique objects in the corresponding group; the remaining objects are evenly distributed to... and Each subset contains approximately 18.4% unique objects, ensuring consistency in the state distribution of the training, testing, and validation subsets, thereby improving the model's training stability and generalization ability.

[0081] Step S4, the improved fuzzy inference algorithm, addresses the need for fusion processing of heterogeneous data. First, it defines the feature representation of the object to be determined: the feature of the object to be determined is represented by an n-dimensional vector. Characterization, where the first g elements are qualitative features, from The elements up to n are quantitative characteristics;

[0082] To improve the reliability and priority differentiation ability of rule-based reasoning, rule weights are introduced. The larger the weight value, the higher the reliability and priority of the corresponding rule. For rule R... p The improved fuzzy rule design to adapt to heterogeneous data is as follows:

[0083]

[0084] in, The categorical value for a qualitative variable; For a fuzzy set of quantitative variables, its membership function is: ; For object state categories; For the state corresponding to rule Rp The weights; ( (Total number of rules);

[0085] To achieve fuzzification of qualitative data, the membership function of the qualitative variables is defined in analogical form:

[0086]

[0087] in, That is, when the input qualitative variable value is consistent with the qualitative classification value in the rule, the membership degree is 1, otherwise it is 0, to ensure accurate matching and fuzzification of qualitative data;

[0088] Rule R p The truth value represents the degree of matching between the input vector and the rule conditions, and the calculation formula is as follows:

[0089]

[0090] in , By taking the minimum value of the membership degree of the qualitative variable and the membership degree of the quantitative variable, the overall matching degree between the input vector and the rule conditions is comprehensively reflected.

[0091] Based on rule Rp The truth value and rule weight are used to calculate the solution obtained by applying the rule. The reliability score is calculated using the following formula:

[0092]

[0093] in, The score represents the total number of operator status categories. It comprehensively considers the rule matching degree and the reliability of the rule itself, providing a quantitative basis for rule ranking.

[0094] Rule weight The neural network is adaptively calculated through the training process of the fuzzy neural network. This neural network is a neural network representation of the above-mentioned improved fuzzy rules. It can automatically optimize the weight parameters based on the training data to ensure the adaptability of the rule weights to the actual data distribution.

[0095] The system response (i.e., the operator status of the input object) is generated by selecting a reliability score. The largest rule serves as the core decision rule, and its output state is... This refers to the judgment state of the input object; simultaneously, the reliability score of this core rule. As a confidence factor, it is output along with the decision state to characterize the credibility of the decision result;

[0096] As a confidence factor, it is output along with the decision state to characterize the reliability of the decision result. Sorting in descending order, the resulting list of rules serves as the core basis for natural language interpretation, clearly presenting the logical chain of the judgment process, enabling decision-makers to understand the origin of the judgment results, and enhancing the system's trustworthiness.

[0097] Step S5: Construction of the improved fuzzy neural network. By building a core framework of a 5-layer improved fuzzy neural network, the function, computational logic, and output format of each layer are clarified, providing a foundation for subsequent parameter optimization. The network structure is as follows: Figure 1 As shown, the specific implementation is as follows:

[0098] Input layer: Receives the preprocessed core feature vectors ,in (Corresponding to 4 core parameters: D) min t c D hc PuRe), the number of neurons is the same as the number of core parameters, there is no additional data transformation operation, and the feature vector is directly passed to the next layer;

[0099] Fuzzification layer: Performs fuzzification processing on qualitative and quantitative features, and calculates the membership degree for qualitative features. This achieves binary fuzzy mapping of qualitative classification values; for quantitative features, a trapezoidal membership function is used to calculate the membership degree. Mapping quantitative parameters to The fuzzy set of intervals provides a foundation for subsequent rule matching;

[0100] Rule Precondition Layer: Calculates R for each fuzzy rule based on the membership degree output by the fuzzification layer. p truth value Truth value characterizes the degree of matching between input features and rule conditions;

[0101] Rule successor layer: Introduces a rule weight matrix to realize reliability score calculation and rule ranking. for A 3D matrix, in the following form:

[0102]

[0103] in, For rule R p Corresponding state The weights are assigned, and a reliability score is calculated for each rule. This comprehensively reflects both the rule matching degree and the reliability of the rule itself, and then constructs a conflict rule set. , and according to Sort in descending order and generate a list of sorted rules. ;

[0104] Deblurring layer: Completes state determination and result output; the network outputs a value pair. Calculated according to the following rules:

[0105]

[0106]

[0107]

[0108]

[0109] in, This is the optimal rule index. For operator status determination results, To assess the reliability of the results, the output includes both a status label and a reliability score, thereby enhancing the reference value of the assessment results.

[0110] Step S6: Improved fuzzy neural network training. A two-stage training algorithm is used to optimize network parameters. Combining rule-state connection path logic and training effectiveness criteria, the model's convergence and generalization ability are ensured. The specific process is as follows:

[0111] The first phase of training involves rule weighting. Initialization calculation, in a fuzzy neural network, neuron rule R p With object state The connection, such as Figure 2 As shown, this corresponds to the rule-state connection path in the network. For each state There are several rule-state connection paths, and each path corresponds to one fuzzy inference rule. The format is as follows:

[0112]

[0113] in , For the output state is Fuzzy rules Total number, if there is no corresponding rule-state connection path in the network. According to the rules, the weighting coefficients If a corresponding rule exists, the weight is calculated using the following formula:

[0114]

[0115] in, For training subset The status label is The number of samples; Qualitative characteristics Corresponding rules Membership degree; For quantitative characteristics Corresponding rules Membership degree;

[0116] The initial rule weight matrix is ​​calculated. Afterwards, the matrix is ​​fixed and the training proceeds to the next stage. The second stage of training optimizes the membership function parameters and fixes the regular weight matrix. Then, the membership function parameters of the quantitative features are iteratively optimized using a genetic algorithm. The optimization process aims at achieving a comprehensive criterion for training effectiveness.

[0117] Introducing a comprehensive criterion for training effectiveness As the optimization objective, it takes the following form:

[0118]

[0119] Simultaneously satisfy the constraints:

[0120]

[0121] in, For training rounds; , Training subsets and test subset The total number of samples; , The first The number of training samples and test samples correctly classified in each training round. The physical meaning of this criterion is: to maximize the classification accuracy of the training set in all rounds while ensuring that the classification accuracy of the test set does not decrease with each round.

[0122] Step S7: The trained fuzzy neural network is used for real-time determination and interpretation of the operator's status. The real-time pupil response data, after being processed in steps S2 and S3, is input into the trained model, and the operator's status is obtained through the inference algorithm in step S4. and credibility If the state is fatigue, then based on Generate natural language explanation text; save data and inference results, issue anomaly alerts and provide intervention suggestions; if the status is normal, directly output the status and continuously collect data for cyclical judgment.

[0123] The primary purpose of the non-contact operator state determination method based on fuzzy neural networks proposed in this invention is to accurately determine and interpretably output the operator's normal / fatigue state by non-contactly collecting the operator's pupil reaction data and combining it with an improved fuzzy neural network and a two-stage training algorithm.

[0124] This embodiment prepares an operator pupil response dataset containing 1,500 raw data entries from operators of different ages and ethnicities (750 in normal state and 750 in fatigue state). Each data entry includes 12 input parameters (10 quantitative parameters and 2 qualitative parameters) and 1 output label.

[0125] Quantitative parameters: Initial normalized pupil diameter D0, minimum normalized pupil diameter D min Minimum normalized pupil diameter D T Pupil contraction time t c Pupil dilation time t d Pupil contraction rate V c Pupil dilation rate V d Half-pupil contraction rate D hc Half-contraction time t hc Half-expansion time t hd .

[0126] Qualitative parameters: Pupil reactivity (graded as follows: Grade 1 - sluggish response, Grade 2 - normal response, Grade 3 - sensitive response), subject race (Caucasian, Mongoloid, Negroid, Australian, Native American).

[0127] The output label is the operator function status Class (0-normal, 1-fatigue).

[0128] The original pupil response dataset underwent multi-dimensional preprocessing to obtain a standardized dataset. First, based on expert analysis, the core parameters that contributed most to operator status judgment were selected, redundant parameters were removed, and finally four key parameters were retained: minimum normalized pupil diameter D. min Pupil contraction time t c Half-pupil contraction rate D hc PuRe (Pupillary Reactivity)

[0129] Based on this, the preprocessed dataset is divided by the grouped random sampling with replacement algorithm in step S3 to obtain a training subset of 1500 samples (containing 948 unique objects), a test subset of 276 samples (containing 51 unique objects), and a validation subset of 276 samples (containing 51 unique objects), effectively solving the data imbalance problem.

[0130] Subsequently, four core parameters (D) were selected. min t c D hc PuRe constructs fuzzy production rules: Three fuzzy levels ("low," "medium," and "high") are set as quantitative parameters. A trapezoidal membership function is used to achieve fuzzy mapping, and its membership function formula is expressed as:

[0131]

[0132] in, The inflection point parameters of the trapezoidal membership function need to be optimized through subsequent training.

[0133] Qualitative parameters are classified into 1-3 levels to achieve binary fuzzy matching, and their membership functions are as follows:

[0134]

[0135] Combining the fuzzy hierarchical and classification dimensions of the four core parameters, 81 initial rules for all permutations are generated. The basic form of each initial rule is as follows:

[0136]

[0137] To ensure the effectiveness and practicality of the rules, the initial rules are further optimized through a triple screening mechanism of "physical meaning validity verification," "data-driven relevance verification," and "rule conflict merging," eliminating meaningless, valueless, and logically contradictory rules.

[0138] Physical validity verification: Based on the physiological response mechanism of the pupil, rules that violate physiological laws are eliminated; such as those involving highly responsive operators like PuRe. At that time, the minimum normalized pupil diameter D min It will not be at a "high" level;

[0139] Data-driven correlation verification: Count the number of matches of the initial rule condition combination in 1500 original samples, and remove rules with no sample match, that is, the rule conditions have no corresponding instances in the dataset and have no actual judgment value;

[0140] Rule conflict merging: For rules with the same combination of conditions but contradictory judgment states, retain the rule with a higher degree of matching with the sample distribution; if 80% of the samples under the same combination of conditions correspond to the "fatigue" state, then only the rule that is judged as "fatigue" is retained.

[0141] After the above three-stage screening, 52 valid fuzzy inference rules were ultimately retained. Some of the rules are shown in the table below:

[0142] Table 1: Fragments of fuzzy inference rules used to evaluate operator functional status

[0143] Rules Class CF IF High AND High AND 1 AND Average THEN Tired 0.519 IF Low AND Average AND 2 AND Low THEN Norm 0.326 IF High AND High AND 2 AND Average THEN Tired 0.281 IF Low AND Low AND 1 AND Average THEN Norm 0.140 IF Low AND High AND 3 AND Average THEN Norm 0.138 IF High AND High AND 1 AND Low THEN Tired 0.122

[0144] A 5-layer fuzzy neural network was built based on the TensorFlow framework, and the two-stage training algorithm in step S6 was adopted: first, the regular weight matrix was calculated according to the formula. Complete weight initialization; fix rule weights, optimize the trapezoidal membership function parameters of quantitative parameters based on genetic algorithm, and set the training rounds. Training effectiveness criteria It eventually converged to 0.98.

[0145] The trained model is then used for inference. In this invention, 52 effective fuzzy inference rules are constructed. When a single input feature vector is passed into the model, the truth value of all 52 rules needs to be calculated one by one. and reliability score Then, the rule with the highest reliability score is selected from all the rules.

[0146] When inputting preprocessed operator pupil data: D min : D hc : PuRe: “1”, t c : The model will execute the following inference process:

[0147] 1. The fuzzification layer calculates the membership degree of each parameter, and the rule precursor layer obtains the rules. truth value ;

[0148] 2. Combine the rule weights shown in the table. The reliability score for the rule-following layer is calculated using the following formula:

[0149]

[0150] 3. If after 52 iterations, If the value is the maximum among all rules, the deblurring layer outputs the final result: "The operator is in a state of fatigue. The judgment criteria are: the minimum normalized pupil diameter and the half-constricted pupil diameter are both at a high level, the pupil reactivity level is 1 (sluggish response), and the pupil constriction time is in the medium range. The judgment confidence is 0.462."

[0151] Through the complete technical solution of rule construction, network training, and inference judgment described above, this invention can accurately determine the normal and fatigued functional states of an operator from non-contact acquisition of operator pupil reaction data, and output credible natural language explanation text; the performance of the constructed fuzzy logic inference system is verified, and the results are as follows: Figure 4 As shown, the classification accuracy rate of the experimental data was 98%, the classification accuracy rate of the testing process was 95.3%, and the classification accuracy rate of the verification process was 96.7%, which fully demonstrates the reliability and superiority of the technical solution of this invention.

[0152] It should be noted that, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0153] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A non-contact operator status determination method based on fuzzy neural networks, characterized in that, Includes the following steps: Step S1: Acquisition of pupillary response data. Using a non-contact pupillary measurement device, pupillary response data of operators of different ages and races are collected under two typical conditions: normal and fatigued. Step S2: Data preprocessing. The original pupil response dataset is preprocessed in multiple dimensions to obtain a standardized dataset. Step S3: Construct a balanced dataset based on grouped random sampling with replacement. Using the grouped random sampling with replacement algorithm, the preprocessed standardized dataset is divided into a balanced training subset, a test subset, and a validation subset. Step S4: Improved fuzzy inference algorithm; Step S5: Construction of the improved fuzzy neural network. By building a core framework of a 5-layer improved fuzzy neural network, the functions, computational logic and output format of each layer are clarified. Step S6: Improved fuzzy neural network training, using a two-stage training algorithm to optimize network parameters, combining rule-state connection path logic and training effectiveness criteria; Step S7: Use the trained fuzzy neural network to determine the operator's status in real time and interpret the results.

2. The non-contact operator status determination method based on fuzzy neural network according to claim 1, characterized in that: The data collected in step S1 is in discrete time series form, containing quantitative and qualitative parameters: the output label is the operator's functional status, the collection process covers all typical states, and the total number of samples in the original dataset is no less than 1500.

3. The non-contact operator status determination method based on fuzzy neural network according to claim 2, characterized in that: Step S2 adopts The criteria identify and remove outliers from quantitative parameters to ensure data reliability. Next, the filtered quantitative parameters are normalized, mapping the parameter values ​​to the [0,1] interval. Excluding dimensional differences, the normalization formula is as follows: ， in, These are the original values ​​of the quantitative parameters. , These are the maximum and minimum values ​​of the parameter in the original dataset, respectively. Finally, the preprocessed core parameter data are stored in a unified format to establish a standardized operator pupil response dataset for subsequent model training and validation.

4. The non-contact operator status determination method based on fuzzy neural network according to claim 3, characterized in that: The specific steps of step S3 are as follows: Initialization parameter: N is the total number of objects in the dataset. ; Number of object states; Define the object's state category; initialize the training subset. test subset Validation subset ; According to state B k Divide the preprocessed dataset into Groups Each group has a capacity of N. k ,satisfy ; Training subset sampling, for each group Execute in a loop To N k Randomly select objects Copy to training subset middle: ;from Remove those already selected Object: ; Test and validation subset sampling, when At that time, randomly select objects Copy to test subset middle: ;from Remove those already selected Object: And randomly select objects. Copy to the verification subset middle: ;from Remove those already selected Object: until ; The subsets corresponding to each state group are merged to obtain the final dataset: , , .

5. The non-contact operator status determination method based on fuzzy neural network according to claim 4, characterized in that: The specific process of step S4 is as follows: First, define the feature representation of the object to be judged: the feature of the object to be judged is represented by an n-dimensional vector. Characterization, where the first g elements are qualitative features, from The elements up to n are quantitative features, and then regular weights are introduced. The larger the weight value, the higher the reliability and priority of the corresponding rule. For rule R... p The improved fuzzy rule design to adapt to heterogeneous data is as follows: ， in, The categorical values ​​for qualitative variables; For a fuzzy set of quantitative variables, its membership function is: ; For object state categories; For the state corresponding to rule Rp The weights; ( (For the total number of rules), define the membership function analogy form for the qualitative variable: ， in, That is, when the input qualitative variable value matches the qualitative classification value in the rule, the membership degree is 1; otherwise, it is 0. Rule R p The truth value represents the degree of matching between the input vector and the rule conditions, and the calculation formula is as follows: ， in , By taking the minimum value between the membership degrees of the qualitative and quantitative variables, based on rule R... p The truth value and rule weight are used to calculate the solution obtained by applying the rule. The reliability score is calculated using the following formula: ， in, The total number of operator status categories, and the rule weight. The system response, i.e., the operator state of the input object, is generated through adaptive calculation via a fuzzy neural network training process, and is obtained by selecting a reliability score. The largest rule serves as the core decision rule, and its output state is... This refers to the judgment state of the input object; simultaneously, the reliability score of this core rule. As a confidence factor, it is output along with the decision state to characterize the reliability of the decision result. Sort in descending order, and the resulting list of rules serves as the core basis for natural language interpretation.

6. The non-contact operator status determination method based on fuzzy neural network according to claim 5, characterized in that: The specific implementation process of step S5 is as follows: Input layer: Receives the preprocessed core feature vectors ,in This corresponds to four core parameters: D min t c D hc PuRe has the same number of neurons as the number of core parameters, and no additional data transformation operations are required. It directly passes the feature vector to the next layer. Fuzzification layer: Performs fuzzification processing on qualitative and quantitative features, and calculates the membership degree for qualitative features. To achieve binary fuzzy mapping of qualitative classification values; For quantitative features, the trapezoidal membership function is used to calculate the membership degree. Mapping quantitative parameters to Fuzzy sets of intervals; Rule Precondition Layer: Calculates R for each fuzzy rule based on the membership degree output by the fuzzification layer. p truth value Truth value characterizes the degree of matching between input features and rule conditions; Rule successor layer: Introduces a rule weight matrix to realize reliability score calculation and rule ranking. for A 3D matrix, in the following form: ， in, For rule R p Corresponding state The weights are assigned, and a reliability score is calculated for each rule. This comprehensively reflects both the rule matching degree and the reliability of the rules themselves, and then constructs a conflict rule set. , and according to Sort in descending order and generate a list of sorted rules. ; Deblurring layer: Completes state determination and result output; the network outputs a value pair. Calculated according to the following rules: ， ， ， ， in, This is the optimal rule index. For operator status determination results, To determine the credibility of the results, the output includes both a status label and a credibility score.

7. The non-contact operator status determination method based on fuzzy neural network according to claim 6, characterized in that: The specific process of step S6 is as follows: The first phase of training involves rule weighting. Initialization calculation, in a fuzzy neural network, neuron rule R p With object state The connection corresponds to the rule-state connection path in the network. For each state There are several rule-state connection paths, and each path corresponds to one fuzzy inference rule. The format is as follows: ， in , For the output state is Fuzzy rules Total number, if there is no corresponding rule-state connection path in the network. According to the rules, the weighting coefficients If a corresponding rule exists, the weight is calculated using the following formula: ， in, For training subset The status label is The number of samples; Qualitative characteristics Corresponding rules Membership degree; For quantitative characteristics Corresponding rules The membership degree is used to calculate the initial rule weight matrix. Then, fix the matrix and proceed to the next training phase; The second stage of training optimizes the membership function parameters and fixes the regular weight matrix. Then, the membership function parameters of the quantitative features are iteratively optimized using a genetic algorithm. The optimization process aims at achieving a comprehensive criterion for training effectiveness. Introducing a comprehensive criterion for training effectiveness The optimization objective takes the following form: ， Simultaneously satisfy the constraints: ， in, For training rounds; , Training subsets and test subset The total number of samples; , The first The number of training samples and the number of test samples that are correctly classified in each round of training.

8. The non-contact operator status determination method based on a fuzzy neural network according to claim 7, characterized in that: Specifically, step S7 involves processing the real-time pupil response data through steps S2 and S3, inputting it into the trained model, and obtaining the operator status through the inference algorithm in step S4. and credibility ; If the state is fatigue, then based on Generate natural language explanation text; save data and inference results, issue anomaly alerts and provide intervention suggestions; if the status is normal, directly output the status and continuously collect data for cyclical judgment.

Images