Micro-robot morphology human-machine collaborative decision-making method, system, device, medium and product based on image and physiological information fusion perception

Abstract

The present application discloses a method, system, device, medium, and product for human-machine collaborative decision-making of microrobot morphology based on the fusion perception of image and physiological information, which relates to the field of microrobot morphology decision-making technology. The method comprises: obtaining physiological information of a designated area where the microrobot is located; based on actual posture state information and physiological information, the microrobot generates spontaneous deformation and obtains the posture state information after deformation; using a Kalman filter model, performing optimal state estimation on the posture state information after deformation to obtain the optimal state estimation of the posture state information; a control center determines the correctness of the spontaneous deformation result of the microrobot; if the deformation is correct, the posture state information after deformation is determined as the final posture state information; otherwise, an error correction instruction is issued; and using a CART classification tree model, the final posture state information is determined based on the error correction instruction and the deformation amount generated by the spontaneous deformation. The present application improves the accuracy of microrobot morphology decision-making.

Description

Technical Field

[0001] The present application relates to the field of micro-robot morphology decision-making technology, and in particular to a method, system, device, medium and product for human-machine collaborative decision-making of micro-robot morphology based on the fusion perception of image and physiological information. Background Art

[0002] Microrobots are small robots with dimensions ranging from a few nanometers to a few hundred micrometers. These robots can perform tasks at this scale with exceptional flexibility and adaptability. Microrobotics technology has rapidly developed into an emerging frontier research field, with crucial applications in clinical medicine and bioengineering, such as minimally invasive surgery, targeted therapy, cell manipulation, heavy metal detection, and pollutant degradation. The external drive and motion control system of a microrobot is crucial for achieving precise movement and executing multiple task sequences.

[0003] Limited by the sensing technology of closed living environments, traditional imaging technologies based on ultrasound, fluorescence, and magnetic fields all have problems such as weak signals, low sensitivity, and poor real-time performance. These problems pose huge challenges to the real-time adjustment of the microrobot's posture under human decision-making during living operations, such as movement, hovering, rotation, contraction, and stretching.

[0004] Targeted microrobot operations require the robot to be able to track and dynamically adjust its shape during movement, such as curling and contracting, to prevent damage to surrounding tissue. Furthermore, in certain strong alkaline and acidic environments, real-time deformation protects drugs from physical contact and inactivation by environmental substances. After reaching the target area, the robot's posture and shape must be adjusted to optimally contact the lesion. This places even higher demands on the real-time and precise dynamic decision-making in unstructured environments. Summary of the Invention

[0005] The purpose of this application is to provide a method, system, device, medium and product for human-machine collaborative decision-making of micro-robot morphology based on the fusion perception of image and physiological information, so as to solve the problem of low accuracy of micro-robot morphology decision-making.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a micro-robot morphology human-machine collaborative decision-making method based on image and physiological information fusion perception, comprising:

[0008] Acquiring physiological information of a designated area where the microrobot is located;

[0009] Based on the physiological information, the microrobot generates spontaneous deformation and obtains posture state information after deformation;

[0010] Using the Kalman filter model, the optimal state estimation of the deformed posture state information is performed to obtain the optimal state estimation of the posture state information;

[0011] The control center determines whether the spontaneous deformation of the microrobot is correct or incorrect; the correct or incorrect result is whether the deformation is correct or incorrect;

[0012] If the correct result is that the deformation is correct, the posture state information after the deformation is determined as the final posture state information of the micro robot;

[0013] If the correct result is a deformation error, the control center sends a correction instruction to the microrobot, and the microrobot uses the CART classification tree model to determine the final posture state information of the microrobot based on the correction instruction and the deformation amount generated by spontaneous deformation.

[0014] Optionally, before obtaining the physiological information of the designated area where the microrobot is located, the method further includes:

[0015] The microrobot obtains position information of the designated area and moves to the designated area based on the position information.

[0016] Optionally, a Kalman filter model is used to perform optimal state estimation on the deformed posture state information to obtain the optimal state estimation of the posture state information, specifically including:

[0017] The deformed posture state information is used as the measured value of the posture state information at the initial optimization moment, and the optimal state estimation is performed on the initial posture state information according to the set optimization time step to obtain the optimal state estimation of the posture state information.

[0018] Optionally, the Kalman filter model includes:

[0019]

[0020] in, is the optimal state estimate of the pose state information at the k-th optimization moment; is the predicted state estimate of the posture state information at the kth optimization moment determined based on the measured value of the posture state information at the k-1th optimization moment; K(k) is the Kalman gain at the kth optimization moment; Z(k) is the measured value of the posture state information at the kth optimization moment; H(k) is the observation matrix at the kth optimization moment; A(k) is the state transfer matrix at the kth optimization moment; It is the optimal state estimate of the pose state information at the k-1th optimization moment.

[0021] Optionally, determining whether the spontaneous deformation of the microrobot is correct or incorrect comprises:

[0022] Based on the posture state information after deformation and the optimal state estimation of the posture state information, the correctness or incorrectness of the spontaneous deformation of the micro robot is determined.

[0023] Optionally, determining whether the spontaneous deformation of the microrobot is correct or incorrect based on the posture state information after deformation and the optimal state estimation of the posture state information includes:

[0024] Determine whether the deformed posture state information is consistent with the optimal state estimate of the posture state information;

[0025] If so, the true or false result is that the deformation is correct;

[0026] If not, the true or false result is a deformation error.

[0027] In a second aspect, the present application provides a micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception, including:

[0028] A physiological information acquisition module, used to acquire physiological information of a designated area where the microrobot is located;

[0029] A spontaneous deformation module, configured to cause the microrobot to spontaneously deform based on the physiological information and to obtain posture state information after deformation;

[0030] A Kalman filter module is used to perform optimal state estimation on the deformed posture state information using a Kalman filter model to obtain the optimal state estimation of the posture state information;

[0031] A judgment module, configured to control the center to determine whether the spontaneous deformation of the microrobot is correct or incorrect; the correct or incorrect result is whether the deformation is correct or incorrect;

[0032] a first final posture determination module, configured to determine the posture state information after deformation as the final posture state information of the microrobot if the correctness result indicates that the deformation is correct;

[0033] The second final posture determination module is used to determine the final posture state information of the microrobot based on the error correction instruction and the deformation amount generated by the spontaneous deformation by the control center if the error result is a deformation error.

[0034] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned micro-robot morphology human-computer collaborative decision-making methods based on image and physiological information fusion perception.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned micro-robot morphology human-machine collaborative decision-making methods based on image and physiological information fusion perception.

[0036] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned micro-robot morphology human-machine collaborative decision-making methods based on image and physiological information fusion perception.

[0037] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0038] The present application discloses a method, system, device, medium and product for human-machine collaborative decision-making of micro-robot morphology based on the fusion perception of image and physiological information. First, the physiological information of the designated area where the micro-robot is located is obtained; based on the physiological information, the micro-robot generates spontaneous deformation and obtains the posture state information after deformation; secondly, the Kalman filter model is used to perform optimal state estimation on the posture state information after deformation to obtain the optimal state estimation of the posture state information; then, the control center determines the correctness or incorrectness of the spontaneous deformation of the micro-robot; the correctness or incorrectness result is that the deformation is correct or the deformation is incorrect; if the correctness or incorrectness result is that the deformation is correct, the posture state information after deformation is determined as the final posture state information of the micro-robot; if the correctness or incorrectness result is that the deformation is correct, the posture state information after deformation is determined as the final posture state information of the micro-robot; if the correctness or incorrectness result is that the deformation is correct, the posture state information after deformation is determined as the final posture state information of the micro-robot; if the correctness or incorrectness result is that the deformation is incorrect, the control center sends an error correction instruction to the micro-robot, and the micro-robot uses the CART classification tree model to determine the final posture state information of the micro-robot based on the error correction instruction and the deformation amount generated by the spontaneous deformation. The present application combines the autonomous deformation and error correction instructions of the micro-robot to realize human-machine collaborative decision-making of the micro-robot morphology and improve the accuracy of the micro-robot morphology decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic flow chart of a micro-robot morphology human-machine collaborative decision-making method based on image and physiological information fusion perception provided in one embodiment of the present application;

[0041] Figure 2 Schematic diagram of the architecture of the micro-robot morphology human-machine collaborative decision-making method based on the fusion perception of image and physiological information;

[0042] Figure 3A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0044] The purpose of this application is to provide a method, system, device, medium and product for micro-robot morphology human-machine collaborative decision-making based on the fusion perception of image and physiological information, aiming to improve the accuracy of micro-robot morphology decision-making.

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0046] In an exemplary embodiment, Figure 1 and Figure 2 As shown, the micro-robot human-machine collaborative decision-making method based on image and physiological information fusion perception in this embodiment includes:

[0047] Step 1: Obtain physiological information of the designated area where the microrobot is located.

[0048] Specifically, the microrobot is a microrobot that works in a human or animal body. The designated area in which the microrobot is located is a designated part in the body, so the designated area includes physiological information of hydrogen ions, sodium ions, and hydroxide ions.

[0049] As an optional implementation manner, before step 1, the method further includes:

[0050] The microrobot obtains the location information of the designated area and moves to the designated area based on the location information.

[0051] Step 2: Based on physiological information, the microrobot generates spontaneous deformation and obtains the posture state information after deformation.

[0052] Specifically, spontaneous deformation becomes expansion or contraction.

[0053] Step 3: Use the Kalman filter model to perform optimal state estimation on the deformed posture state information to obtain the optimal state estimation of the posture state information.

[0054] As an optional implementation, step 3 specifically includes:

[0055] The deformed posture state information is used as the measured value of the posture state information at the initial optimization moment. According to the set optimization time step, the optimal state estimation of the initial posture state information is performed to obtain the optimal state estimation of the posture state information.

[0056] As an optional implementation, the Kalman filter model includes:

[0057]

[0058] in, is the optimal state estimate of the pose state information at the k-th optimization moment; is the predicted state estimate of the posture state information at the kth optimization moment determined based on the measured value of the posture state information at the k-1th optimization moment; K(k) is the Kalman gain at the kth optimization moment; Z(k) is the measured value of the posture state information at the kth optimization moment; H(k) is the observation matrix at the kth optimization moment; A(k) is the state transfer matrix at the kth optimization moment; It is the optimal state estimate of the pose state information at the k-1th optimization moment.

[0059] Step 4: The control center determines whether the microrobot's spontaneous deformation is correct or incorrect. The correct or incorrect result is whether the deformation is correct or incorrect.

[0060] As an optional embodiment, determining whether the spontaneous deformation of the microrobot is correct or incorrect includes:

[0061] Based on the posture state information after deformation and the optimal state estimation of the posture state information, the correctness of the spontaneous deformation of the micro robot is determined.

[0062] As an optional embodiment, determining the correctness of the spontaneous deformation of the microrobot based on the posture state information after deformation and the optimal state estimation of the posture state information includes:

[0063] Determine whether the deformed pose state information is consistent with the optimal state estimate of the pose state information.

[0064] If so, the true or false result is that the deformation is correct.

[0065] If not, the true or false result is a deformation error.

[0066] Step 5: If the result is that the deformation is correct, the posture state information after deformation is determined as the final posture state information of the micro robot.

[0067] Step 6: If the error result shows that the deformation is correct, the control center sends a correction instruction to the microrobot. The microrobot uses the CART classification tree model to determine the final posture state information of the microrobot based on the correction instruction and the deformation amount generated by spontaneous deformation.

[0068] Furthermore, after determining the final posture state information of the microrobot, the microrobot performs corresponding deformation according to the final posture state information.

[0069] Specifically, the determination process of the CART classification tree model includes:

[0070] 1. Data preparation:

[0071] Collect the decision signal dataset D, which includes elements such as autonomous posture perception output, functional perception output, image feedback status and posture correction output.

[0072] Each element in the decision signal dataset D represents a specific decision signal, such as the probability of the microrobot being in a certain autonomous posture perception output, the probability of a certain functional perception output occurring, the probability of a certain state occurring based on image feedback, and the probability of the posture correction output being a specific posture.

[0073] The interpretation of each element and probability is:

[0074] (1) Autonomous posture perception output: This represents the posture changes (e.g., expansion or contraction) of the microrobot in its autonomous state in response to environmental conditions. For example, under certain pH conditions, the microrobot may automatically adjust its shape to cope with changes in the external environment. The decision signal dataset D contains probabilistic information about the autonomous posture state and is used to evaluate the postures adopted by the microrobot under different stimulus conditions.

[0075] (2) Functional perception output: This involves the functional performance of the microrobot in a specific scenario, such as the deformation that occurs when it comes into contact with tumor tissue or other specific physiological environments. Functional perception output refers to the microrobot's response behavior to the environment in this specific scenario and its probability of occurrence. For example, a microrobot may undergo a specific autonomous deformation upon detecting tumor tissue in order to better perform its task.

[0076] (3) Image feedback status: Visual feedback information obtained through an external image acquisition device is used to monitor the microrobot's posture status (stretching or contracting). The image feedback status is reflected in the decision signal dataset D as the state evaluation results based on the visual recognition algorithm and its occurrence probability. This data can help the operator monitor the microrobot's motion status in real time and make necessary adjustments.

[0077] (4) Posture Correction Output: When the operator discovers through monitoring equipment that the microrobot's deformation behavior is inconsistent with expectations, they may issue a posture correction instruction. The posture correction output is a control signal applied by the operator through manual intervention, which is used to adjust the microrobot's posture state. In the decision signal dataset D, this posture correction information from manual intervention is also quantified as the probability of different states occurring, so that it can be integrated with the autonomous decision information.

[0078] 2. Calculate the Gini coefficient:

[0079] The Gini coefficient of the decision signal dataset D is calculated using formula (3) to evaluate the purity or uncertainty of the decision signal dataset D.

[0080]

[0081] Where Gini(D) is the Gini coefficient of the decision signal data set D; p(x i ) is the i-th element x in the decision signal dataset D i The probability of occurrence reflects the possibility that the microrobot will perform a certain behavior (such as stretching or contracting) under specific conditions; n is the total number of elements in the decision signal dataset D.

[0082] When x i is the probability p(x i ): For example, when the microrobot is in a specific pH environment, it may automatically expand or contract. If we define x i is the category of the microrobot's stretched state in this environment, then p(x i ) is the probability that the microrobot is in the extended state under this pH environment. For example, if the pH value is 7, the microrobot has a 70% probability of being in the extended state, then p(x i )=0.7.

[0083] When x i The probability p(x i ): When the microrobot detects a specific physiological environment (such as tumor tissue), it may undergo a preset deformation to perform a specific function. In this case, x i represents the autonomous deformation behavior of the microrobot when detecting tumor tissue, and p(x i ) is the probability of such deformation. For example, if the microrobot has a 90% probability of deforming when it contacts a tumor, then p(x i )=0.9.

[0084] When x i The probability p(x i): When the operator finds that the micro robot does not work as expected, he can issue a posture correction command (i.e., error correction command) through the control system. In this case, x i It may represent the operator's instruction to adjust the microrobot to a specific posture (e.g., contraction), and p(x i ) is the probability of successful posture adjustment. For example, if the microrobot has a 95% probability of successfully adjusting to the retracted state after the operator's instruction, then p(x i )=0.95.

[0085] For each possible element A, the Gini coefficient is calculated using formula (4) when its value is a specific value a. This helps determine which element and its specific value can best reduce the uncertainty of the decision signal dataset D.

[0086]

[0087] Among them, GiniIndex(D|A=a) is the Gini coefficient of D when the value of element A is a specific value a; D1 and D2 are two data sets obtained by dividing data set D according to the value of element A, D1 is a data set consisting of all data points in the stretched state, and D2 is a data set consisting of all data points in the contracted state; |·| is the number of samples in the data set.

[0088] 3. Use the decision signal dataset to construct a CART classification tree:

[0089] In each step, the element A with the lowest Gini coefficient and its value are selected as the splitting criterion for the current node.

[0090] According to the value of attribute A, the dataset D is divided into two subsets D1 and D2, which are respectively used as the left and right child nodes of the current node.

[0091] The above process is repeated for each child node until the stopping condition is met. The stopping condition is that the number of samples in the child node is less than a predetermined threshold or the Gini coefficient is lower than a certain threshold.

[0092] Advantages of the method of this application:

[0093] 1. Significantly improve operational accuracy and real-time performance: By integrating non-real-time microrobot position state information (in the form of images) with dynamic physiological information, the system can more accurately identify the microrobot's operational scenario, enabling more precise decision-making. Furthermore, the application of the Kalman filter model can determine the optimal state estimate of the microrobot's position state information in real time, effectively compensating for image processing delays and improving the system's response speed and operational accuracy.

[0094] 2. Enhance the microrobot's environmental adaptability and flexibility: This allows the microrobot to spontaneously adjust its form based on environmental physiological information and error correction instructions to adapt to different operational requirements and environmental conditions. This flexibility not only improves the microrobot's operational efficiency but also enhances its survivability and operational safety in complex environments.

[0095] 3. Optimizing the stability and reliability of the decision-making system: The CART classification tree algorithm enables efficient processing of multimodal sensory information with low computational complexity and understandable decision rules, thereby optimizing the system's decision-making performance and stability. The application of this algorithm reduces the subjectivity and uncertainty of human decision-making, improving reliability and accuracy.

[0096] 4. Improve the robustness and scalability of the overall system: When faced with complex and changing operating environments and task requirements, it can maintain stable performance and good adaptability, while also providing more possibilities for future technology upgrades and functional expansion.

[0097] The CART classification tree model is used to screen the matching degree and priority of nonlinear differentiated command signals, thereby achieving the fusion output of multimodal signals and realizing the uncoupled output of decision signals (i.e., the final posture state information of the microrobot determined based on error correction instructions and the deformation amount generated by spontaneous deformation), thereby improving the high dynamics and fault tolerance of the microrobot during global motion and local posture adjustment.

[0098] In an exemplary embodiment, a micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception is provided, comprising:

[0099] The physiological information acquisition module is used to obtain the physiological information of the specified area where the micro robot is located.

[0100] The spontaneous deformation module is used to enable the microrobot to generate spontaneous deformation based on physiological information and obtain the posture state information after deformation.

[0101] The Kalman filter module is used to use the Kalman filter model to perform optimal state estimation on the deformed posture state information to obtain the optimal state estimation of the posture state information.

[0102] The judgment module is used by the control center to determine whether the spontaneous deformation of the micro robot is correct or incorrect; the correct or incorrect result is correct deformation or incorrect deformation.

[0103] The first final posture determination module is used to determine the posture state information after deformation as the final posture state information of the micro robot if the correction result shows that the deformation is correct.

[0104] The second final posture determination module is used to determine the final posture state information of the microrobot based on the error correction instruction and the deformation amount generated by the spontaneous deformation by using the CART classification tree model.

[0105] In an exemplary embodiment, a computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a micro-robot morphology human-machine collaborative decision-making method based on the fusion perception of image and physiological information.

[0106] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a micro-robot morphology human-machine collaborative decision-making method based on image and physiological information fusion perception is implemented.

[0107] In an exemplary embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements a micro-robot morphology human-machine collaborative decision-making method based on image and physiological information fusion perception.

[0108] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a micro-robot morphology human-computer collaborative decision-making method based on the fusion perception of image and physiological information is realized.

[0109] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0111] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0112] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0113] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception, characterized by: include: A physiological information acquisition module, used to acquire physiological information of a designated area where the microrobot is located; The physiological information of the designated area includes: physiological information of hydrogen ions, sodium ions and hydroxide ions in the designated area; A spontaneous deformation module, configured to cause the microrobot to spontaneously deform based on the physiological information and to obtain posture state information after deformation; A Kalman filter module is used to use a Kalman filter model to perform an optimal state estimation on the deformed posture state information to obtain the optimal state estimation of the posture state information, including: using the deformed posture state information as the measured value of the posture state information at the initial optimization time, performing an optimal state estimation on the posture state information at the initial optimization time according to a set optimization time step, and obtaining the optimal state estimation of the posture state information; A judgment module, configured to control the center to determine whether the spontaneous deformation of the microrobot is correct or incorrect; the correct or incorrect result is whether the deformation is correct or incorrect; a first final posture determination module, configured to determine the posture state information after deformation as the final posture state information of the microrobot if the correctness result indicates that the deformation is correct; The second final posture determination module is used to determine the final posture state information of the microrobot based on the error correction instruction and the deformation amount generated by the spontaneous deformation by the control center if the error result is a deformation error.

2. The micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception according to claim 1 is characterized in that: Before the physiological information acquisition module acquires the physiological information of the designated area where the microrobot is located, the microrobot acquires the position information of the designated area and moves to the designated area based on the position information.

3. The micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception according to claim 1 is characterized in that: The Kalman filter model includes: ； ； in, is the optimal state estimate of the pose state information at the k-th optimization moment; A predicted state estimate of the pose state information at the kth optimization moment determined based on the measured value of the pose state information at the k-1th optimization moment; is the Kalman gain at the kth optimization moment; is the measured value of the posture state information at the k-th optimization moment; is the observation matrix at the kth optimization moment; is the state transfer matrix at the kth optimization moment; It is the optimal state estimate of the pose state information at the k-1th optimization moment.

4. The micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception according to claim 1 is characterized in that: Determining whether the spontaneous deformation of the microrobot is correct or not, comprising: Based on the posture state information after deformation and the optimal state estimation of the posture state information, the correctness or incorrectness of the spontaneous deformation of the micro robot is determined.

5. The micro-robot-based human-machine collaborative decision-making system based on image and physiological information fusion perception according to claim 4 is characterized in that: Determining the correctness of the spontaneous deformation of the microrobot based on the posture state information after deformation and the optimal state estimation of the posture state information includes: Determine whether the deformed posture state information is consistent with the optimal state estimate of the posture state information; If so, the true or false result is that the deformation is correct; If not, the true or false result is a deformation error.

Images