Multi-channel man-machine collaborative natural interaction method for explicit implicit knowledge

Abstract

The invention discloses an implicit knowledge explicit multichannel man-machine collaborative natural interaction method, which comprises the following steps of: (1) analyzing collaborative task sensing nodes taking an operator as a center to obtain a sensing information flow model in a man-machine-ring in a collaborative scene, and researching an explicit and implicit knowledge conversion chain in a man-machine collaborative combat process; step (2), constructing an SECI model of knowledge conversion, representing that there is explicit and implicit knowledge in a collaborative combat man-machine loop closed-loop system, and obtaining knowledge conversion node importance weight under a man-machine collaborative task; step (3), according to knowledge conversion node importance weights, carrying out an eye movement physiological evaluation experiment to obtain a behavior cognitive map fusing multiple perception channels, and explicitly expressing tacit knowledge in multichannel interaction behaviors; and step (4), outputting a multi-channel man-machine collaborative natural interaction method according to the behavior cognitive map. The method provides reference for man-machine collaborative interaction design of a multi-channel interaction system by mining behavior interaction implicit knowledge of an operator.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction, and in particular relates to a multi-channel human-computer collaborative natural interaction method for explicit knowledge of tacit knowledge. Background Art

[0002] The progress of technologies such as artificial intelligence and virtual reality has made single-channel human-computer interaction difficult to meet people's pursuit of interactive experiences with precise perception. Especially in the future collaborative environment with information explosion, multi-channel human-computer interaction can achieve multi-angle integration of human senses and has greater application potential and value.

[0003] With the exponential growth of information processed by users and the expansion of the demand for cross-screen information presentation and multi-channel interaction methods, the earliest proposed interface interaction paradigm can no longer meet the new needs in aspects such as system design, software architecture, and interface interaction technology. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-channel human-computer collaborative natural interaction method for explicit knowledge of tacit knowledge, which innovatively combines the transformation of explicit and tacit knowledge, behavior cognitive maps, and interaction methods, establishes a mapping relationship between combat mission knowledge representation and interaction behavior in a collaborative task scenario, and constructs a multi-channel behavior interaction method.

[0005] The technical solution for achieving the purpose of the present invention is as follows:

[0006] A multi-channel human-computer collaborative natural interaction method for explicit knowledge of tacit knowledge, comprising the following steps:

[0007] Step (1): Conduct feature analysis on the operator in the human-computer collaborative task, analyze the collaborative task perception nodes centered on the operator, obtain the perception information flow model in the human-machine-environment in the collaborative scenario, and study the explicit and tacit knowledge transformation chain in the process of the operator's human-computer collaborative combat;

[0008] Step (2): For the human-computer collaborative interaction mode, use the explicit and tacit knowledge transformation chain to construct the SECI model of knowledge transformation. The SECI model includes the explicit and tacit knowledge transformation chain. The SECI model is used to represent the existence of explicit and tacit knowledge in the human-machine-environment closed-loop system of collaborative combat, send a node importance questionnaire to the target user, recycle and analyze the questionnaire, and obtain the importance weight of the knowledge transformation node under the human-computer collaborative task;

[0009] Step (3): According to the importance weight of the knowledge transformation node, select multiple knowledge transformation nodes with the top-ranked importance weights, set the content corresponding to the top-ranked knowledge transformation nodes as the collaborative task, use a multi-channel human-computer interaction system to conduct an eye movement physiological evaluation experiment, and based on graph theory knowledge, obtain a behavior cognitive map that integrates multiple perception channels, and explicitly express the tacit knowledge in multi-channel interaction behavior;

[0010] Step (4): Output a multi-channel human-machine collaborative natural interaction method according to the behavior cognitive map.

[0011] Further, the said step (1) includes:

[0012] Step (1-1): Analyze the combat perception nodes centered on the operator to obtain the perception information flow model in the human-machine-loop in the collaborative scenario. The perception information flow model in the human-machine-loop includes an external stimulus unit, an operator perception unit, and a fighter control unit. The external stimulus unit is used to receive the transformation information and instructions of the battlefield situation and input them into the operator perception unit in the form of information. After receiving the information input from the external stimulus unit, the operator perception unit outputs corresponding behavior operations to the fighter control unit through an information perception module, an information processing module, and an information output module. After receiving the control operation of the operator, the fighter control unit affects the battlefield situation and returns to the external stimulus unit through information feedback.

[0013] Step (1-2): For the operator user group in the human-machine collaborative combat task, use the operator perception unit of the perception information flow model in the human-machine-loop to study the explicit and implicit knowledge transformation chain in the process of user human-machine collaboration.

[0014] Further, the said step (2) specifically includes:

[0015] Step (2-1): For the human-machine collaborative interaction mode, construct a SECI model for knowledge transformation using the explicit and implicit knowledge transformation chain. The SECI model includes four stages, namely socialization, externalization, combination, and internalization. The knowledge transformation in the four stages is in different fields, and the fields in the human-machine collaborative scenario correspond to the processing field, the output field, the situation field, and the perception field respectively. The processing field is used to express the internal transformation of tacit knowledge, including the information processing and output of the operator. The output field is used to express the transformation of tacit knowledge into explicit knowledge, including the decision output of the operator and the transmission of battlefield information. The situation field is used to express the internal transformation of explicit knowledge, including the change of the human-machine-loop situation information. The perception field is used to express the transformation of explicit knowledge into tacit knowledge, including the information reception and perception of the operator.

[0016] Step (2-2): Deconstruct the SECI model of the hierarchical deconstruction spiral closed-loop, construct four types of hierarchical structures: the target layer, the criterion layer, the sub-criterion layer, and the solution layer. The target layer includes the G node of the influencing factors in the human-machine collaborative perception stage. The criterion layer includes four first-level nodes, namely the information processing C1 node, the decision-making output node C2, the information sharing node C3, and the situation awareness node C4. The information processing node C1 includes three second-level nodes in the sub-criterion layer, namely channel allocation S1, knowledge transformation S2, and memory storage S3. The decision-making output node C2 includes two second-level nodes in the sub-criterion layer, namely voice control S4 and limb operation S5. The information sharing node C3 includes three second-level nodes in the sub-criterion layer, namely information sharing among operators S6, information sharing with unmanned aerial vehicles S7, and environmental situation sharing S8. The situation awareness node C4 includes two second-level nodes in the sub-criterion layer, namely sensory perception S9 and interface information perception S10. Use the analytic hierarchy process to design a questionnaire on the importance of nodes, send the questionnaire on the importance of nodes to the target users, conduct group decision-making on the questionnaire survey results, construct a weighted geometric mean set of judgment matrices, and analyze to obtain the importance weights of each node.

[0017] Further, the step (2-2) specifically includes:

[0018] Compare the relative importance of each pair of second-level nodes among the four first-level nodes, construct a judgment matrix, conduct a consistency test of the judgment matrix, calculate the consistency index CI, and the calculation formula is as shown in Equation (1); find the corresponding average consistency index RI; calculate the consistency ratio CR; the calculation formula is as shown in Equation (2):

[0019]

[0020] Among them, CI represents the consistency degree of the matrix; CR is the random consistency ratio. When CR < 0.1, it is considered that the judgment matrix passes the consistency test; λ max is the maximum eigenvalue of the judgment matrix; n is the dimension number of the judgment matrix, that is, the number of second-level nodes under each first-level node;

[0021] The set weight of expert experience is obtained by the weighted geometric mean method. The calculation method is as shown in Equation (3). First, multiply the elements in the judgment matrix by row to obtain a new column vector, then take the nth root of each element in the new column vector, and finally normalize the column vector to obtain the weight;

[0022]

[0023] i is the serial number of the second-level node, a nj and a kj are both elements in the judgment matrix.

[0024] Further, the step (3) specifically includes:

[0025] Step (3-1): According to the importance weights of knowledge transformation nodes, select multiple knowledge transformation nodes with top-ranked importance weights, set the content corresponding to the top-ranked knowledge transformation nodes as collaborative tasks, and conduct eye movement physiological evaluation experiments using a multi-channel human-computer interaction system to obtain an AOI set with temporal context relationships;

[0026] Step (3-2): Through the spectral clustering algorithm, cluster the AOI set with temporal context relationships into an AOI set with functional attributes;

[0027] Step (3-3): Using the attributes represented by the AOI set with functional attributes as graph nodes, establish a behavioral cognition sub-graph, merge the behavioral cognition sub-graphs to obtain a behavioral cognition graph, and calculate the closeness centrality of the graph nodes.

[0028] Furthermore, the AOI set with temporal context relationships in the step (3-1) is expressed as:

[0029] AOI t ={aoi1,aoi2,…,aoi i ,…} (4)

[0030] Divide the data collected on the multi-channel interaction system interface into Areas of Interest (AOIs) to obtain multiple regions, where aoi i represents an AOI region with temporal context, and this set has temporal and task context relationships;

[0031] Among them, the AOI set with functional attributes in the step (3-2) is expressed as:

[0032] {aoi|P(aoi)} (5)

[0033] where P(aoi) represents that the AOI region has functional attribute relationships after spectral clustering division, that is, an AOI functional attribute set is formed;

[0034] Among them, the behavioral cognition graph in the step (3-3) is expressed as:

[0035] H=(AOI′,ACT') (6)

[0036] G=(AOI,ACT) (7)

[0037] Among them, AOI'(H) is the set of points of the behavioral cognitive sub-graph, and AOI(H) is the set of points of the behavioral cognitive graph, that is, the set of AOIs with functional attributes. The size of the graph node represents the proportion of the fixation duration on this AOI; ACT'(H) is the set of edges of the behavioral cognitive sub-graph, and ACT(H) is the set of edges of the behavioral cognitive graph, that is, the set of interaction methods between each graph node. When there is an association relationship between graph nodes, they are connected by a directed edge, and the direction of the directed edge represents the jump order of the fixation. The thickness of the directed edge represents the proportion of the number of saccade times. The more times, the thicker the directed edge; the color distinction of the directed edge represents different interaction methods. H represents the behavioral cognitive sub-graph, and G represents the behavioral cognitive graph, satisfying

[0038] Among them, the formula for calculating the node closeness centrality in step (3-3) is:

[0039]

[0040] where s is the number of graph nodes, and d q is the average shortest path from the q-th graph node to the p-th graph node, and d q is smaller, which means the greater the closeness degree of this graph node, and d qp represents the path from the q-th graph node to the p-th graph node, and the reciprocal CC(q) of d q is defined as the closeness centrality of the graph node.

[0041] Compared with the prior art, the present invention has the following technical effects: The present invention innovatively combines the transformation of explicit and implicit knowledge, behavioral cognitive graphs, and interaction methods, establishes a mapping relationship between combat mission knowledge representation and interaction behavior in a collaborative task scenario, and constructs a multi-channel behavior interaction method. This method is aimed at the interaction system in a collaborative scenario. From the perspective of knowledge representation, by mining the implicit knowledge of the operator's behavior interaction, it provides a reference for the human-machine collaborative interaction design of related multi-channel interaction systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is the invention flow chart of the multi-channel human-machine collaborative natural interaction method for explicitizing implicit knowledge of the present invention.

[0043] Figure 2 is the information flow chart of the operator perception node in step (1).

[0044] Figure 3 is the SECI model diagram of knowledge transformation in step (2).

[0045] Figure 4 is the hierarchical analysis model diagram in step (2-2).

[0046] Figure 5 is the flow chart of constructing the behavioral cognitive graph in step (3).

[0047] Figure 6 They are the cognitive behavior sub-graph and the behavior cognition graph.

[0048] Figure 7 They are the experimental platform construction and experimental setup diagrams.

[0049] Figure 8 It is the experimental task time sequence diagram.

[0050] Figure 9 It is the experimental spectral clustering result diagram. Specific implementation manners

[0051] According to an embodiment of the present invention, a multi-channel human-machine collaborative natural interaction method for explicit knowledge of implicit knowledge is provided. The method flow is as Figure 1 shown, and specifically includes:

[0052] (1) Conduct feature analysis on the pilots in the human-machine collaborative tasks, analyze the collaborative task perception nodes centered on the pilots, and combine Figure 2 , the combat perception elements of the pilots penetrate the complex closed-loop system of the human-machine-environment. The essence of this process is the transmission and processing of information between the pilots and the machine-environment. The perception elements of the pilots are not only the combination of the information perception, processing, and output modules, but also the chain reaction of the entire information flow. In this information flow, the information is mainly stimulated and input by the battlefield situation, and also includes the external information feedback stimulation of manned / unmanned aircraft; in the information perception module, the pilots realize the perception of the external stimulus information source through sensory systems such as eyes, ears, and skin, and transmit it in the form of perception signals such as vision, hearing, and touch; in the information processing module, internal processing systems such as the human brain center identify, adapt to, and retrieve memories for the information, so as to form decision-making and execution plans. In parallel with the information processing is the process of knowledge transformation. Once the processing is completed, in the information output module, the pilots will make control decisions according to the current navigation mode and realize the control operation of the fighter through limbs, language, different control methods, etc. After the whole process, under the combined action of the closed-loop of the human-machine-environment, the change of the fighter operation parameters is fed back to the pilots again, forming a trend of information flow.

[0053] (2) For the human-machine collaborative interaction mode, construct a SECI model for knowledge transformation, analyze the explicit and implicit knowledge representations existing in the closed-loop system of the collaborative combat human-machine-environment, send out a node importance questionnaire to the target users, recycle and analyze the questionnaire, and obtain the importance weight ranking of the knowledge transformation nodes under the human-machine collaborative tasks. The specific steps are as follows:

[0054] (2-1)Construct the SECI model of knowledge transformation, which is mainly divided into four stages: socialization, externalization, combination, and internalization. The knowledge transformation in each stage is in a different field. According to the description of the "field" concept in the model, the fields in the man-machine collaboration scenario are abstracted as the processing field, output field, situation field, and perception field. The socialization stage corresponds to the processing field, that is, the internal transformation of tacit combat knowledge, which corresponds to the individual entering the information processing and information output stage through the recognition of battlefield situation signals in the man-machine collaboration combat scenario. During the information processing, the individual's brain encodes the received information internally, which has personalized characteristics and is difficult to share between organizations and systems. The intuitive judgment ability of the individual when facing the battlefield situation is the application of tacit knowledge. And in the information output stage, that is, the action stage, the action knowledge used is technical tacit knowledge, including the individual making decisions, formulating plans, and taking actions (including physical, language, etc.) in the battlefield; the externalization corresponds to the output field, that is, the transformation of tacit combat knowledge into explicit combat knowledge, and the individual's tacit knowledge is shared in the man-machine collaboration combat system, and the explicit description is in the form of combat data information, etc. In the man-machine collaboration combat scenario, it corresponds to the impact of the individual's decision output on the current battlefield situation, that is, the transmission of information among humans, machines, and the environment; the combination corresponds to the situation field, that is, the transformation between explicit combat knowledge, and the fragmented explicit knowledge in the current state is aggregated into a system to complete knowledge sharing. In the man-machine collaboration combat scenario, it corresponds to the information sharing among humans, machines, and the environment; the internalization corresponds to the perception field, that is, the transformation of explicit combat knowledge into tacit combat knowledge, and the systematic explicit knowledge is internalized into the individual's tacit understanding through sharing. In the man-machine collaboration combat scenario, it is expressed as the external stimulus input of battlefield situation information and the individual's internal information perception, so as to perform information processing and output operations, thus forming a closed-loop system. The SECI model of knowledge transformation is as Figure 3 shown.

[0055] (2-2)Combine all elements of the pilot's information perception in step (1) and hierarchically deconstruct the spiral model for importance assessment, such as Figure 4 . Deconstruct the SECI model of the spiral according to the analytic hierarchy process, construct four types of hierarchical structures: the target layer, criterion layer, sub-criterion layer, and solution layer. Compare the relative importance of the influencing factors in each layer of the hierarchical ladder model pairwise, construct a judgment matrix, conduct a consistency test of the judgment matrix, calculate the consistency index CI, and the calculation formula is as shown in Equation (1); find the corresponding average consistency index RI; calculate the consistency ratio CR; the calculation formula is as shown in Equation (2).

[0056]

[0057] Among them, CI represents the consistency degree of the matrix; CR is the random consistency ratio. When CR < 0.1, it is considered that the judgment matrix passes the consistency test; λ max is the maximum eigenvalue of the judgment matrix; n is the dimension number of matrix A.

[0058] The combined weight of the five experts' experience is obtained by the weighted geometric mean method. The calculation method is shown in Equation (3). First, multiply the elements in the matrix row by row to obtain a new column vector, then take the nth root of each element in the new vector, and finally normalize the column vector to obtain the weight vector.

[0059]

[0060] According to the results of the single sorting of the objective-criterion layer, the weight values of the criterion layer for the overall objective have been obtained. Table 1 contains the sub-criterion layer of the influencing factors in the human-machine collaborative combat perception stage and the relevant importance weight matrix of the solution layer.

[0061] Table 1 Hierarchical total sorting

[0062]

[0063] (3) According to the explicit and implicit knowledge transformation mechanism, set collaborative tasks for the multi-channel human-machine interaction system, conduct eye movement physiological evaluation experiments, and based on graph theory knowledge, obtain a behavioral cognitive map that integrates multiple perception channels to explicitly express the implicit knowledge in multi-channel interaction behaviors; specifically including:

[0064] (3-1) Design collaborative tasks for the multi-channel human-machine interaction system according to the importance degree and conduct eye movement physiological evaluation experiments. The eye movement physiological evaluation experiment is executed using the custom experiment program built into the Tobii X3-120 eye tracker and the supporting software Tobii Studio. All stimuli are presented on a 15.6-inch computer display with a resolution of 1920 pixels × 1080 pixels, and the viewing distance used is 50 cm, as Figure 7 shown. The experiment adopts a specific combat task process, and the time sequence diagram of the experimental task content is as Figure 8 shown. The experimental process includes the subjects reading the introduction to multi-channel interaction in collaborative combat and the operation instructions of this experiment, which takes about 3 minutes to understand the basic interface functions of the system and the meaning of the content expressed by the interface; then, system teaching is carried out, and the teaching content mainly includes the usage experience of 4 types of interaction tasks, aiming to guide the subjects to naturally use different interaction methods during the formal experiment, which takes about 3 minutes; the original data of the area of interest (AOI) with time context is obtained through the eye movement physiological evaluation experiment.

[0065] (3-2) Since different functional attributes are obtained by executing different tasks at different times, the spectral clustering method is used for the AOI sequence parameters with time context relationships to cluster the time series data, and the AOI sequence in time is converted into an AOI sequence with functional attributes. Figure 9 For the spectral clustering result, the AOI set with time context is clustered into an AOI set with 8 types of functional attributes.

[0066] (3-3) Combine Figure 5 , take the key attributes represented by the AOI set with functional attributes as nodes, establish a behavioral cognitive subgraph, merge and recognize the behavioral graph, and calculate the closeness centrality of the nodes.

[0067] Among them, the AOI set with time context relationships in step (3-1) is expressed as:

[0068] AOI t ={aoi1,aoi2,…,aoi i ,…} (4)

[0069] Divide the data collected on the multi-channel interaction system interface into Areas of Interest (AOI), where aoi i represents an AOI area with time context, and this set has context relationships in time and tasks.

[0070] The AOI set with functional attributes is expressed as:

[0071] {aoi|P(aoi)} (5)

[0072] Among them, P(aoi) represents that the elements have functional attribute relationships after spectral clustering division, that is, an AOI functional attribute sequence is formed.

[0073] The behavioral cognitive graph is expressed as:

[0074] H=(AOI,ACT′) (6)

[0075] G=(AOI,ACT) (7)

[0076] Among them, AOI′(H) is the set of points of the behavioral cognitive sub-graph, and AOI(H) is the set of points of the behavioral cognitive graph, that is, the set of AOIs with functional attributes. The size of the graph node represents the proportion of the duration of gazing at the AOI; ACT′(H) is the set of edges of the behavioral cognitive sub-graph, and ACT(H) is the set of edges of the behavioral cognitive graph, that is, the set of interaction methods between each graph node. When there is an association relationship between graph nodes, they are connected by directed edges. The direction of the directed edge represents the jump order of gazing. The thickness of the directed edge represents the proportion of the number of gaze jumps. The more times, the thicker the directed edge; the color distinction of the directed edge represents different interaction methods. H represents the behavioral cognitive sub-graph, and G represents the behavioral cognitive graph, satisfying such as Figure 6 shown

[0077] The calculation formula for the closeness centrality of a node is as follows:

[0078]

[0079] where d i is the average shortest path from a node to other nodes. The smaller d i is, the greater the closeness of the node. The reciprocal CC(i) of d i is defined as the closeness centrality of the node.

[0080] The following rules can be found through the cognitive behavior graph:

[0081] a. When facing a complex scene and multiple function switching operations are required in a short time, the voice interaction method can be tried to reduce the operation complexity of the pilot;

[0082] b. For the interface presenting a large amount of situation information, natural interaction methods such as voice or gestures can be tried to keep the pilot's perceptual attention highly concentrated.

[0083] (4) Based on the explicit and implicit knowledge transformation mechanism and the cognitive behavior graph, a reasonable interaction method is output.

[0084] The above is only the preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-channel human-machine collaborative natural interaction method for explicit manifestation of tacit knowledge, characterized in that, It includes the following steps: Step (1): Conduct feature analysis on the operator in the human-machine collaborative task, analyze the collaborative task perception nodes centered on the operator, obtain the perception information flow model in the human-machine-environment in the collaborative scenario, and study the explicit and implicit knowledge transformation chain in the process of the operator's human-machine collaborative operation; Step (2): For the human-machine collaborative interaction mode, construct the SECI model of knowledge transformation using the explicit and implicit knowledge transformation chain. The SECI model includes the explicit and implicit knowledge transformation chain, and the SECI model is used to characterize the existence of explicit and implicit knowledge in the human-machine-environment closed-loop system of collaborative operation. Send out a node importance questionnaire to the target users, collect and analyze the questionnaire to obtain the importance weights of the knowledge transformation nodes under the human-machine collaborative task; Step (3): According to the importance weights of the knowledge transformation nodes, select multiple knowledge transformation nodes with higher importance weights, set the content corresponding to the knowledge transformation nodes with higher rankings as the collaborative tasks, conduct an eye movement physiological evaluation experiment using a multi-channel human-computer interaction system, and based on graph theory knowledge, obtain a behavior cognitive map integrating multiple perception channels to explicitly express the implicit knowledge in the multi-channel interaction behavior; Step (4): Output a multi-channel human-machine collaborative natural interaction method according to the behavior cognitive map.

2. The multi-channel human-computer collaborative natural interaction method for explicit manifestation of tacit knowledge according to claim 1, wherein The said step (1) includes: Step (1-1): Analyze the combat perception nodes centered on the operator to obtain the perception information flow model in the human-machine-environment in the collaborative scenario. The perception information flow model in the human-machine-environment includes an external stimulus unit, an operator perception unit, and a fighter control unit. The external stimulus unit is used to receive the transformation information and instructions of the battlefield situation and input them into the operator perception unit in the form of information. After receiving the information input from the external stimulus unit, the operator perception unit outputs corresponding behavioral operations to the fighter control unit through an information perception module, an information processing module, and an information output module. After receiving the control operation of the operator, the fighter control unit affects the battlefield situation and returns to the external stimulus unit through information feedback; Step (1-2): For the operator user group of the human-machine collaborative combat task, use the operator perception unit of the perception information flow model in the human-machine-environment to study the explicit and implicit knowledge transformation chain in the process of the user's human-machine collaborative operation.

3. The multi-channel human-machine collaborative natural interaction method for explicit manifestation of tacit knowledge according to claim 2, wherein The said step (2) specifically includes: Step (2-1): For the human-machine collaborative interaction mode, a SECI model for knowledge transformation is constructed using an explicit and implicit knowledge transformation chain. The SECI model includes four stages, namely socialization, externalization, combination, and internalization. The knowledge transformation in these four stages occurs in different fields, which respectively correspond to a processing field, an output field, a situation field, and a perception field in the human-machine collaborative scenario. The processing field is used to express the internal transformation of tacit knowledge, including the information processing and output of the operator. The output field is used to express the transformation of tacit knowledge into explicit knowledge, including the decision-making output of the operator and the transmission of battlefield information. The situation field is used to express the internal transformation of explicit knowledge, including the changes in the human-machine-environment situation information. The perception field is used to express the transformation of explicit knowledge into tacit knowledge, including the information reception and perception of the operator; Step (2-2): Hierarchically decompose the spiral closed-loop SECI model to construct four types of hierarchical structures: a target layer, a criterion layer, a sub-criterion layer, and a solution layer. The target layer includes the influence factor G node of the human-machine collaborative perception stage. The criterion layer includes four first-level nodes, namely the information processing C1 node, the decision-making output node C2, the information sharing node C3, and the situation perception node C4. The information processing node C1 contains three second-level nodes in the sub-criterion layer, namely channel allocation S1, knowledge transformation S2, and memory storage S3. The decision-making output node C2 contains two second-level nodes in the sub-criterion layer, namely voice control S4 and limb operation S5. The information sharing node C3 contains three second-level nodes in the sub-criterion layer, namely information sharing among operators S6, information sharing with unmanned aerial vehicles S7, and environmental situation sharing S8. The situation perception node C4 contains two second-level nodes in the sub-criterion layer, namely sensory perception S9 and interface information perception S10. Use the analytic hierarchy process to design a node importance questionnaire, send the node importance questionnaire to the target users, conduct group decision-making on the questionnaire survey results, construct a judgment matrix weighted geometric mean set, and analyze to obtain the importance weights of each node.

4. The multi-channel human-machine collaborative natural interaction method for explicit manifestation of tacit knowledge according to claim 3, wherein The specific content of step (2-2) includes: Compare the relative importance of each pair of second-level nodes among the four first-level nodes to construct a judgment matrix, conduct a consistency test of the judgment matrix, calculate the consistency index CI, and the calculation formula is as shown in formula (1); find the corresponding average consistency index RI; calculate the consistency ratio CR; the calculation formula is as shown in formula (2): Among them, CI represents the consistency degree of the matrix; CR is the random consistency ratio. When CR < 0.1, it is considered that the judgment matrix passes the consistency test; λ max is the maximum eigenvalue of the judgment matrix; n is the dimension number of the judgment matrix, that is, the number of secondary nodes under each primary node; Use the weighted geometric mean method to obtain the set weight of expert experience. The calculation method is as shown in formula (3). First, multiply the elements in the judgment matrix by rows to obtain a new column vector, then take the nth root of each element in the new column vector, and finally normalize the column vector to obtain the weight; i is the serial number of the secondary node, a nj and a kj are both elements in the judgment matrix.

5. The multi-channel human-computer collaborative natural interaction method for explicit manifestation of tacit knowledge according to claim 4, wherein The specific content of step (3) includes: Step (3-1): According to the importance weights of the knowledge transformation nodes, select multiple knowledge transformation nodes with higher importance weights, set the content corresponding to the knowledge transformation nodes with higher rankings as collaborative tasks, and conduct an eye movement physiological evaluation experiment using a multi-channel human-machine interaction system to obtain an AOI set with time context relationships; Step (3-2): Cluster the AOI sets with temporal context relationships into AOI sets with functional attributes through the spectral clustering algorithm; Step (3-3): Use the attributes represented by the AOI sets with functional attributes as graph nodes to establish a behavioral cognitive subgraph, merge the behavioral cognitive subgraphs to obtain a behavioral cognitive graph, and calculate the closeness centrality of the graph nodes.

6. The multi-channel human-computer collaborative natural interaction method for explicit manifestation of tacit knowledge according to claim 5, wherein, The AOI set with temporal context relationships in the said step (3-1) is expressed as: AOI t ={aoi1,aoi2,…,aoi i ,…) (4) Divide the data collected on the multi-channel interaction system interface into multiple areas of interest (AOIs), where aoi i represents an AOI area with time context, and this set has context relationships in terms of time and tasks; Among them, the AOI set with functional attributes in step (3-2) is expressed as: {aoi|P(aoi)} (5) Where P(aoi) represents that the AOI area has functional attribute relationships after spectral clustering division, that is, a set of AOI functional attributes is formed; Among them, the behavioral cognitive graph in step (3-3) is expressed as: H = (AOI′, ACT') (6) G = (AOI, ACT) (7) Among them, AOI'(H) is the set of points of the behavioral cognitive sub-graph, and AOI(H) is the set of points of the behavioral cognitive graph, that is, the set of AOIs with functional attributes. The size of the graph nodes represents the proportion of the duration of fixation on the AOI; ACT'(H) is the set of edges of the behavioral cognitive sub-graph, and ACT(H) is the set of edges of the behavioral cognitive graph, that is, the set of interaction methods between each graph node. When there is an association relationship between graph nodes, they are connected by directed edges. The direction of the directed edge represents the jump order of fixation; the thickness of the directed edge represents the proportion of the number of saccade times. The more times, the thicker the directed edge; the color distinction of the directed edge represents different interaction methods. H represents the behavioral cognitive sub-graph, and G represents the behavioral cognitive graph, satisfying Among them, the calculation formula for the node closeness centrality in step (3-3) is: where s is the number of graph nodes, and d q is the average shortest path from the q-th graph node to the p-th graph node. The smaller d q is, the greater the closeness of the graph node. qp represents the path from the q-th graph node to the p-th graph node. q The reciprocal CC(q) of is defined as the closeness centrality of the graph node.

Images