Robot control method based on spiking neural network

Abstract

The application relates to the field of bionic control technology and discloses a robot control method based on a pulse neural network, which comprises the following steps: performing space-time feature coding on environment perception information and ontology state information of a robot to obtain a pulse feature sequence; performing pulse-dependent fusion on the pulse feature sequence and a feedback pulse signal to obtain a bionic information processing process, and performing membrane potential integration on the bionic information processing process to obtain an integrated output quantity; performing pulse firing processing on the integrated output quantity to obtain a driving pulse signal; performing path fitting on the driving pulse signal to obtain a continuous motion trajectory; performing joint space mapping on the continuous motion trajectory based on the continuous motion trajectory and the bionic information processing process to obtain an execution control instruction; and synchronously collecting a pulse response signal and performing real-time feedback on the execution control instruction to obtain an optimized control instruction of the robot; and the application can improve the robot control efficiency based on the pulse neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bionic control technology, and particularly relates to a robot control method based on a pulse neural network. BACKGROUND

[0002] The existing robot control method lacks an efficient spatiotemporal feature fusion and coding mechanism when processing environmental perception information and body state information, and is difficult to fully capture the dynamic association and deep law of the two types of information, resulting in insufficient precision of the generated basic feature data. At the same time, the information processing process fails to effectively simulate the pulse-dependent evolution and membrane potential integration characteristics of biological nerves, making the information processing process lack coherence and bionics, and thus affecting the generation quality of subsequent drive signals and control instructions.

[0003] The existing technology lacks a scientific quantitative evaluation and competition arbitration mechanism in the pulse firing decision-making link, and is difficult to accurately select the optimal firing point, resulting in insufficient pertinence and rationality of the drive pulse signal; in the path fitting and joint space mapping process, the dynamic adjustment combining task requirements and bionic characteristics is not fully considered, making it difficult to balance the smoothness of the motion trajectory and the joint synergy, and lacking a real-time and effective feedback adjustment mechanism, which cannot optimize the control strategy in a timely manner according to the actual motion state of the robot. Therefore, how to improve the precision of robot control and the adaptability of control strategy has become a problem to be solved. SUMMARY

[0004] The present application provides a robot control method based on a pulse neural network to solve the problems raised in the background technology.

[0005] To achieve the above-mentioned purpose, the robot control method based on a pulse neural network provided by the present application comprises:

[0006] S1, spatiotemporal feature coding is performed on the environmental perception information and body state information of a robot to obtain a pulse feature sequence of the robot;

[0007] S2, pulse-dependent fusion is performed on the pulse feature sequence and the feedback pulse signal of the robot to obtain a bionic information processing process of the robot, and membrane potential integration is performed on the bionic information processing process to obtain an integrated output quantity of the robot;

[0008] S3, pulse firing processing is performed on the integrated output quantity to obtain a drive pulse signal of the robot;

[0009] S4, path fitting is performed on the drive pulse signal to obtain a continuous motion trajectory of the robot;

[0010] S5, mapping the continuous motion trajectory in joint space based on the continuous motion trajectory and the bionic information processing process to obtain an execution control instruction of the robot;

[0011] S6, applying the execution control instruction, synchronously collecting a pulse response signal of the robot, and feeding back the pulse response signal to S2 to obtain an optimized control instruction of the robot.

[0012] In a preferred embodiment, the spatiotemporal feature encoding of the environmental perception information and the body state information of the robot obtains a pulse feature sequence of the robot, comprising:

[0013] Asynchronous event-driven sampling of the robot obtains environmental perception information of the robot;

[0014] Event-based sparse representation of the environmental perception information obtains a spatiotemporal event stream of the robot;

[0015] Pose collection of the robot obtains body state information of the robot;

[0016] Joint coordination analysis of the body state information obtains a body motion stream of the robot;

[0017] Pulse coding of the robot based on the spatiotemporal event stream and the body motion stream obtains a pulse feature sequence of the robot.

[0018] In a preferred embodiment, the pulse-dependent fusion of the pulse feature sequence and the feedback pulse signal of the robot obtains a bionic information processing process of the robot, and the membrane potential integration of the bionic information processing process obtains an integrated output quantity of the robot, comprising:

[0019] Bionic time sequence alignment of the pulse feature sequence and the feedback pulse signal of the robot obtains a synchronous pulse group of the robot;

[0020] Dependent relationship evolution of the pulse feature sequence based on the synchronous pulse group obtains a connection strength of the pulse feature sequence;

[0021] Based on the connection strength, the firing behavior of the synchronous pulse group is shaped to obtain a bionic information processing process of the robot;

[0022] Spatiotemporal accumulation of local potential changes in the bionic information processing process obtains a cumulative membrane potential field of the bionic information processing process;

[0023] Field domain filtering is performed on the accumulated membrane potential field to obtain the gated potential field of the accumulated membrane potential field;

[0024] The gated potential field is subjected to transmembrane phase convergence to obtain the integrated output of the robot.

[0025] In a preferred embodiment, the step of pulse-emitting processing the integrated output to obtain the robot's drive pulse signal includes:

[0026] A membrane potential field is constructed on the integrated output quantity to obtain the spatial membrane potential field of the integrated output quantity.

[0027] The spatial membrane potential field is biomimetically evolved to obtain the updated membrane potential field of the spatial membrane potential field;

[0028] Based on the updated membrane potential field, extreme region detection is performed on the robot to obtain the candidate release points of the robot;

[0029] Competitive arbitration is conducted on the candidate distribution points to obtain the priority distribution points for the candidate distribution points;

[0030] Based on the priority distribution point, the pulse distribution of the robot is driven and output to obtain the driving pulse signal of the robot.

[0031] In a preferred embodiment, the step of competitively arbitrating the candidate distribution points to obtain the priority distribution points includes:

[0032] Based on the updated membrane potential field, the potential amplitude of the candidate firing points is extracted to obtain the instantaneous membrane potential amplitude of the candidate firing points;

[0033] Based on the robot's biomimetic information processing process, the historical distribution time of the candidate distribution points is retrieved to obtain the historical distribution time of the candidate distribution points.

[0034] The instantaneous membrane potential amplitude and the historical emission time are weighted and evaluated to obtain the emission priority score of the candidate emission point;

[0035] Based on the priority score, the candidate distribution points are ranked globally to obtain the competitive dominance relationship of the candidate distribution points;

[0036] Based on the aforementioned competitive dominance relationship, conflict analysis is performed on the candidate distribution points to obtain the priority distribution points of the candidate distribution points.

[0037] In a preferred embodiment, the formula for calculating the priority score is as follows:

[0038] ;

[0039] wherein, is a release priority score of the candidate release site, is the i th instantaneous membrane potential amplitude, is the historical release time, is a current time, is a preset weight coefficient, is a preset weight coefficient, is a preset time decay coefficient. In a preferred embodiment, the path fitting of the driving pulse signal to obtain the continuous motion trajectory of the robot comprises:

[0040] spatiotemporal pose mapping of the release timing of the driving pulse signal to obtain a discrete space path of an end effector of the robot;

[0041] curvature continuous processing of the discrete space path of the end effector to obtain a smooth path segment of the robot;

[0042] manifold embedding of the robot based on the smooth path segment to obtain a smooth motion manifold of the robot;

[0043] point-by-point trajectory fitting of the smooth motion manifold to obtain the continuous motion trajectory of the robot.

[0044] In a preferred embodiment, the joint space mapping of the continuous motion trajectory based on the continuous motion trajectory and the bionic information processing process to obtain the execution control instruction of the robot comprises:

[0045] task-related feature decoding of the bionic information processing process to obtain bionic context information of the bionic information processing process;

[0046] task semantic segmentation of the continuous motion trajectory based on the bionic context information to obtain a trajectory segment set of the continuous motion trajectory;

[0047] importance marking of the trajectory segment set to obtain a hierarchical trajectory segment sequence of the continuous motion trajectory.

[0048] joint space projection of the trajectory segment sequence based on the hierarchical trajectory segment sequence to obtain a joint space path of the robot;

[0049]

[0050] ​​Based on the bionic context information, the joint space path is bionically adjusted to obtain the execution control instruction of the robot.

[0051] In a preferred embodiment, the bionic adjustment of the joint space path based on the bionic context information to obtain the execution control instruction of the robot comprises:

[0052] The bionic context information is spatiotemporally analyzed to obtain the motion rhythm synchronization signal of the robot;

[0053] The bionic context information is bionically compiled to obtain the inter-joint coordination pulse code of the robot;

[0054] Based on the motion rhythm synchronization signal, the time axis of the joint space path is dynamically scaled to obtain the pulse emission synchronization path of the robot;

[0055] Based on the inter-joint coordination pulse code, the relative amplitude of joint motion in the pulse emission synchronization path is coordinately gain-adjusted to obtain the bionic coupling path of the robot;

[0056] The bionic coupling path is mapped to the execution control instruction of the robot.

[0057] In a preferred embodiment, the application of the execution control instruction, the synchronous acquisition of the pulse response signal of the robot, and the feedback of the pulse response signal to S2 to obtain the optimized control instruction of the robot comprise:

[0058] The execution control instruction is servo-tracked to obtain the actual joint motion of the robot;

[0059] The execution control instruction is task space feedforward analyzed to obtain the end pose change of the robot;

[0060] Based on the actual joint motion and the end pose change, the motion state of the robot is micro-dynamically event-converted to obtain the original pulse response stream of the robot;

[0061] The original pulse response stream is amplitude-normalized to obtain the standardized pulse response signal of the robot;

[0062] The standardized pulse response signal is pulse-sparse to obtain the optimized feedback pulse signal of the robot;

[0063] The optimized feedback pulse signal is real-time fed back to obtain the optimized control instruction of the robot.

[0064] Compared with the prior art, the present application has the following beneficial effects:

[0065] 1. The control method fully mines the spatiotemporal association and internal law of the two types of information by spatiotemporal feature coding of the robot environmental perception information and ontology state information, realizes a bionic information processing process by combining the pulse-dependent fusion and membrane potential integration mechanism, and greatly improves the integrity and accuracy of information processing. At the same time, through the quantitative calculation of priority score and the competitive arbitration mechanism, the priority issuing point is accurately screened, so that the generation of the driving pulse signal is more targeted and reasonable, effectively improving the precision of robot motion control and the smoothness of motion trajectory, and ensuring that the robot can accurately respond to environmental and self-state changes.

[0066] 2. With the help of real-time feedback regulation mechanism, the method synchronously collects the pulse response signal of the robot and feeds it back to the information fusion link after standardization and sparsification processing, thereby constructing a closed-loop optimized control system, so that the control instruction can be dynamically adjusted according to the actual motion state of the robot, significantly improving the adaptability and robustness of the control. In addition, through joint space mapping and bionic adjustment, the bionic context information related to continuous motion trajectory and task is deeply combined, so that the execution of the control instruction is more consistent with the joint collaborative motion characteristics of the robot, further improving the consistency of the motion trajectory and the task demand, and improving the overall efficiency and reliability of the robot control. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 The flowchart of the robot control method based on the pulse neural network provided by an embodiment of the present application is shown.

[0068] The implementation, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0069] It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0070] Embodiments of the present application provide a robot control method based on a spiking neural network. The execution subject of the robot control method based on the spiking neural network includes but is not limited to at least one of electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiments of the present application. In other words, the robot control method based on the spiking neural network can be executed by software or hardware installed in a terminal device or a server device. The server includes but is not limited to a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be a stand-alone server, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0071] Referring to Figure 1 FIG. 1 shows a flowchart of a robot control method based on a spiking neural network provided by an embodiment of the present application. In this embodiment, the robot control method based on the spiking neural network includes:

[0072] S1, spatiotemporal feature encoding is performed on environment perception information and ontology state information of a robot to obtain a spiking feature sequence of the robot;

[0073] In the embodiment of the present application, the spatiotemporal feature encoding is performed on the environment perception information and the ontology state information of the robot to obtain the spiking feature sequence of the robot, including:

[0074] Asynchronous event-driven sampling is performed on the robot to obtain environment perception information of the robot;

[0075] Event-based sparse representation is performed on the environment perception information to obtain a spatiotemporal event stream of the robot;

[0076] Pose collection is performed on the robot to obtain ontology state information of the robot;

[0077] Joint coordination analysis is performed on the ontology state information to obtain an ontology motion stream of the robot;

[0078] Based on the spatiotemporal event stream and the ontology motion stream, spiking encoding is performed on the robot to obtain a spiking feature sequence of the robot.

[0079] The robot samples through various sensing devices such as vision sensors, tactile sensors, distance sensors, etc. mounted on the robot itself in an asynchronous event-driven manner, not following a fixed time interval, but when a perceivable event such as brightness change, object contact, distance change, etc. occurs in the environment, the sensor immediately responds and captures the specific information corresponding to the event, including the spatial position of the event occurrence, event type and related physical parameter data. All the captured information related to the environment collectively constitutes the environmental perception information of the robot.

[0080] The collected environmental perception information is screened and processed to remove repetitive, redundant and invalid information without practical significance, and only the core event data accurately reflecting the key changes in the environment are retained. Subsequently, the system is arranged and ordered according to the time sequence of each core event occurrence combined with the spatial position information of the event occurrence, forming a continuous data stream containing key information in time and space dimensions. This data stream is the spatio-temporal event stream.

[0081] With the help of devices such as gyroscopes, accelerometers, joint angle sensors, etc. configured on the robot itself, the position coordinate data of the robot in three-dimensional space is captured in real time to determine its specific orientation in space, and the attitude information of the robot is detected, including pitch angle, roll angle, yaw angle, etc. The current angle values of all joints of the robot are recorded, and all data about the position, attitude and joint angle of the robot are integrated and summarized to obtain the ontology state information that can fully reflect the current status of the robot.

[0082] The joint angle data contained in the ontology state information is analyzed in depth to explore the mutual coordination relationship of different joints in the movement process, and the logical sequence of joint movement, the degree of synchronization of angle change and the mutual influence law between joints are determined. According to these analysis results, the continuous change process of joint cooperative operation is systematically sorted out according to the time sequence, and this continuous change process is the ontology motion stream.

[0083] The environmental key change information contained in the spatio-temporal event stream and the joint cooperative motion information embodied in the ontology motion stream are correspondingly associated to deeply explore the internal logical relationship between the two types of information. According to the signal transmission characteristics of the pulse neural network, these associated information is converted into discrete pulse signals, each pulse signal accurately corresponds to a specific environmental event or joint motion state. Then, these discrete pulse signals are orderly arranged according to the time sequence and logical association to form an ordered pulse set, which is the pulse feature sequence.

[0084] The beneficial effects are: the key changes of the environment are accurately captured through asynchronous event-driven sampling, the spatio-temporal event stream is formed through event-based sparse representation, and the effectiveness and orderliness of the environmental information are ensured; the comprehensive body state information of the robot is obtained through pose acquisition, the body motion stream is obtained through joint coordination analysis, and the motion characteristics of the robot itself are accurately reflected; finally, the two types of information are converted into pulse feature sequences through pulse coding, the spatio-temporal correlation and internal law of the environment and the body information are fully mined, high-quality and high-precision basic data are provided for subsequent pulse-dependent fusion and membrane potential integration, and a solid foundation is laid for improving the accuracy of robot control.

[0085] S2, pulse-dependent fusion is performed on the pulse feature sequence and the feedback pulse signal of the robot to obtain a bionic information processing process of the robot, and membrane potential integration is performed on the bionic information processing process to obtain an integrated output quantity of the robot.

[0086] In the embodiment of the application, the pulse-dependent fusion of the pulse feature sequence and the feedback pulse signal of the robot is performed to obtain the bionic information processing process of the robot, and the membrane potential integration of the bionic information processing process is performed to obtain the integrated output quantity of the robot, comprising:

[0087] The pulse feature sequence and the feedback pulse signal of the robot are bionically time-aligned to obtain a synchronous pulse group of the robot.

[0088] Based on the synchronous pulse group, the dependence relationship evolution of the pulse feature sequence is performed to obtain the connection strength of the pulse feature sequence.

[0089] Based on the connection strength, the firing behavior of the synchronous pulse group is shaped to obtain the bionic information processing process of the robot.

[0090] The local potential change in the bionic information processing process is accumulated in space and time to obtain a cumulative membrane potential field of the bionic information processing process.

[0091] The cumulative membrane potential field is field-screened to obtain a gated potential field of the cumulative membrane potential field.

[0092] The gated potential field is transmembrane phase-converged to obtain the integrated output quantity of the robot.

[0093] The feedback pulse signal is a closed-loop feedback pulse signal generated after the robot executes the control instruction, and is a comprehensive pulse representation of the deviation between the actual joint motion state of the robot and the pose of the end effector. Referring to the time sequence coordination characteristics of signal transmission in the biological nervous system, the firing time nodes of the pulse feature sequence and the feedback pulse signal are compared, the pulse units in the two types of signals that time echo each other and logically exist in association are identified, the pulse units that match each other are grouped, the pulses in each group are ensured to maintain synchronization in the time dimension, and finally a synchronous pulse group with regular structure and consistent timing is formed.

[0094] The common occurrence frequency of each pulse in the synchronous pulse group at different time points is analyzed in depth, and the correlation between the pulses is analyzed. According to the closeness of the correlation, the correlation between each pair of pulses is evaluated, and a specific representation result of the correlation closeness between the pulses is obtained. The representation result that can reflect the correlation closeness between the pulses is the connection strength of the pulse feature sequence.

[0095] Taking the connection strength as the core basis, the firing order of the pulses in the synchronous pulse group is adjusted, the pulses with high connection strength are preferentially involved in the firing, the firing interval of the pulses is reasonably set, the firing rhythm of the pulses is adapted to the distribution of the connection strength, and the duration of the pulse firing is standardized. The differential firing mode presented by the biological nerve according to the difference in connection strength is simulated. The complete and bionic pulse processing process is the bionic information processing process.

[0096] The potential change of each local region in the bionic information processing process is tracked throughout the process. The potential change values generated by each local region at different time points are added one by one in the order of time. The spatial position information of each local region is recorded. The potential data containing the time accumulation result and the spatial position information are integrated to form a potential distribution whole covering the time and space dimensions, i.e., the cumulative membrane potential field.

[0097] A clear potential effectiveness determination standard is set. According to the standard, all potential regions in the cumulative membrane potential field are discriminated one by one. The regions whose potential values meet the determination standard and can have a positive effect on subsequent signal processing links are retained, and the invalid regions whose potential values do not meet the standard and do not have actual functions are excluded. The potential distribution region retained after screening is the gating potential field.

[0098] The phase properties of the potentials in different regions of the gating potential field are analyzed in detail. The potential signals with consistent phase characteristics and potential for synergistic action are selected. The phase-synergistic potential signals are concentrated, integrated and fused. The potential signals with conflicting phases are eliminated. The fused potential signal is converted into an output data that is unified, coherent and has a clear functional direction. The output data is the integrated output.

[0099] The beneficial effects are that the timing consistency of the pulse signal is ensured through the bionic timing alignment, the correlation characteristics between the pulses are accurately captured by the dependence evolution, the bionic information processing mode is realized by the shape shaping, the potential signal is gradually optimized by the space-time accumulation, field screening and transmembrane phase convergence, the final integrated output quantity has high precision and strong continuity through the layer-by-layer processing process, which provides high-quality data support for the generation of subsequent driving pulse signals, and effectively improves the bionics of robot control and the reliability of control decision.

[0100] S3, pulse firing processing is performed on the integrated output quantity to obtain a driving pulse signal of the robot;

[0101] In the embodiment of the application, the pulse firing processing on the integrated output quantity to obtain the driving pulse signal of the robot comprises:

[0102] The integrated output quantity is subjected to membrane potential field construction to obtain a spatial membrane potential field of the integrated output quantity;

[0103] The spatial membrane potential field is subjected to bionic evolution to obtain an updated membrane potential field of the spatial membrane potential field;

[0104] Based on the updated membrane potential field, extreme value region detection is performed on the robot to obtain a candidate firing point of the robot;

[0105] The candidate firing point is subjected to competitive arbitration to obtain a preferential firing point of the candidate firing point;

[0106] Based on the preferential firing point, driving output is performed on the pulse firing of the robot to obtain the driving pulse signal of the robot.

[0107] The competitive arbitration on the candidate firing point to obtain the preferential firing point of the candidate firing point comprises:

[0108] Based on the updated membrane potential field, potential amplitude extraction is performed on the candidate firing point to obtain an instantaneous membrane potential amplitude of the candidate firing point;

[0109] Based on the bionic information processing process of the robot, historical firing time retrieval is performed on the candidate firing point to obtain a historical firing time of the candidate firing point;

[0110] The instantaneous membrane potential amplitude and the historical firing time are subjected to weighted evaluation to obtain a firing priority score of the candidate firing point;

[0111] Based on the firing priority score, global competitive sorting is performed on the candidate firing point to obtain a competitive dominance relationship of the candidate firing point;

[0112] Based on the competition dominance relationship, the candidate issuing point is conflict resolved to obtain the priority issuing point of the candidate issuing point.

[0113] The calculation formula of the issuing priority score is as follows:

[0114] ;

[0115] Wherein, is the issuing priority score of the candidate issuing point, is the candidate issuing point, is the instantaneous membrane potential amplitude of the candidate issuing point, is the historical issuing time, is the current time, is the preset weight coefficient, is the preset weight coefficient, is the preset time decay coefficient.

[0116] The potential data contained in the integrated output quantity is systematically divided according to the spatial dimension of the robot motion control, the association between each spatial position and the corresponding potential value is determined, and the dispersed potential information is organized into a whole field that covers the spatial range related to the robot motion and can intuitively reflect the potential distribution. This field is the spatial membrane potential field of the integrated output quantity.

[0117] The dynamic characteristics of membrane potential changing with time and environment in biological nervous system are simulated, the conduction, diffusion and interaction rules of biological neural potential are referred to, and the potential values of each spatial position in the spatial membrane potential field are dynamically adjusted. The change of the potential field conforms to the response mode of biological nerve, and the new membrane potential field formed after dynamic adjustment is the updated membrane potential field.

[0118] All spatial positions in the updated membrane potential field are comprehensively traversed, and the potential values of each position and its adjacent region are compared one by one, and the region with a potential value significantly higher than the surrounding adjacent region and forming a local peak is identified. The region where these local peaks are located is the extreme region, and the core position of each extreme region is the candidate issuing point of the robot.

[0119] For each candidate issuing point, the specific potential value corresponding to it in the updated membrane potential field is directly read, which can reflect the current potential intensity of the candidate issuing point. This read potential value is the instantaneous membrane potential amplitude of the candidate issuing point.

[0120] ​The all pulse firing related data archives recorded in the robot bionic information processing process are consulted, the specific time point at which each candidate firing point has ever fired a pulse in the past information processing process is traced, and the time point is accurately extracted, which is the historical firing time of the candidate firing point.

[0121] According to the influence degree of the instantaneous membrane potential amplitude and the historical firing time on the pulse firing, fixed importance proportions are respectively given to the two, the greater the instantaneous membrane potential amplitude, the greater the contribution to the firing priority, the closer the historical firing time to the current time, the higher the addition to the firing priority, and the comprehensive score of each candidate firing point is calculated by comprehensively considering the influence of the two factors, which is the firing priority score.

[0122] The instantaneous membrane potential amplitude is obtained by performing a potential amplitude extraction operation on the candidate firing point based on the updated membrane potential field, and the extraction process is directly screening the potential value corresponding to the candidate firing point from the updated membrane potential field. The historical firing time is obtained by searching the past firing records of the candidate firing point based on the robot bionic information processing process, and the searching process strictly follows the firing time sequence data retained in the bionic information processing process. The current time is the time data collected in real time during the robot running process, and the preset weight coefficient and the preset time decay coefficient are fixed values set in advance before the robot control program starts according to the control requirements and the robot running characteristics, which are used to adjust the influence degree of different factors on the final result.

[0123] The firing priority score is the result obtained by weighting and evaluating the instantaneous membrane potential amplitude and the historical firing time, and the weighting and evaluating process is to multiply the instantaneous membrane potential amplitude by the corresponding preset weight coefficient, and then multiply the time difference between the historical firing time and the current time by the corresponding preset weight coefficient after exponential operation, and the sum of the two is the final score. The score is used for global competition sorting of all candidate firing points, and the sorting process arranges the candidate firing points in order from high to low according to the score, and then obtains the competition dominance relationship between the candidate firing points. According to the competition dominance relationship, the candidate firing points with firing conflicts are analyzed, and the priority firing point is finally determined.

[0124] The greater the instantaneous membrane potential amplitude, the greater the corresponding weighted result, and the higher the firing priority score. The greater the time interval between the historical firing time and the current time, the smaller the result after exponential operation, the smaller the corresponding weighted result, and the lower the firing priority score. The greater the preset weight coefficient, the greater the influence of the corresponding instantaneous membrane potential amplitude or exponential operation result on the firing priority score. The greater the preset time decay coefficient, the more obvious the weakening effect of the time difference between the historical firing time and the current time on the exponential operation result, and the more significant the influence on the firing priority score.

[0125] The distribution priority scores of all candidate distribution points are collected, all candidate distribution points are sorted in a whole according to the scores from high to low, and the priority of each candidate distribution point in all points is determined, the candidate distribution point with a high score occupies a dominant position in the competition, and can dominate the point with a low score, the domination of the candidate distribution point with a high score over the point with a low score is based on the distribution priority score, after global competition sorting, the point with a high score occupies a time window of pulse distribution or is reserved for distribution when a distribution conflict occurs in the same time window because of stronger necessity and rationality, and the point with a low score is inhibited from distribution because of insufficient priority, and the time resource occupation and the distribution priority in the conflict time are the competitive domination relationship of the candidate distribution points.

[0126] According to the priority order of the priority distribution points, the pulse distribution function of each priority distribution point is activated in sequence, the pulse signals of each point are arranged and transmitted in a format recognizable by the robot driving system, and a signal capable of directly driving the movement of each execution component of the robot is formed, and the signal is a driving pulse signal of the robot.

[0127] The beneficial effects are that the dynamic rationality and bionics of the potential information are ensured by constructing a spatial membrane potential field and bionic evolution, the candidate distribution points with distribution potential are accurately positioned by extreme value region detection, the optimal priority distribution point is screened out by competitive arbitration through multi-dimensional evaluation of the instantaneous membrane potential amplitude and historical distribution time, and the distribution conflict is effectively avoided, and the driving pulse signal generated based on the priority distribution point is targeted and logically rigorous, and can provide accurate and reliable driving support for the movement of the robot, and the precision and response efficiency of the robot control are significantly improved.

[0128] S4, path fitting is performed on the driving pulse signal to obtain a continuous motion trajectory of the robot;

[0129] In the embodiment of the application, the path fitting on the driving pulse signal to obtain the continuous motion trajectory of the robot comprises:

[0130] The distribution time sequence of the driving pulse signal is mapped in space-time pose to obtain a discrete space path of an end effector of the robot;

[0131] The discrete space path of the end effector is subjected to curvature continuous processing to obtain a smooth path segment of the robot;

[0132] based on the smooth path segment, performing manifold embedding on the robot to obtain a smooth motion manifold of the robot;

[0133] performing point-by-point fitting of a trajectory on the smooth motion manifold to obtain a continuous motion trajectory of the robot.

[0134] At each time of emission of the combinatorial driving pulse signal, a one-to-one correspondence is established between the pulse signal intensity corresponding to each time and the spatial position and attitude parameters of the robot end effector, and the specific coordinates of the end effector in the three-dimensional space and the attitude information such as pitch, roll and yaw at each time of emission are accurately determined. In order of emission timing, these discrete position and attitude data points are arranged in order to form a path composed of a series of discrete data points. The path is a discrete spatial path of the end effector.

[0135] The curvatures of the line segments between adjacent data points on the discrete spatial path of the end effector are analyzed one by one, and the line segments with sudden changes in curvature and turning sharp corners are identified. An appropriate number of transition data points are inserted between the adjacent data points of the sudden change line segments, and the spatial positions of the transition data points are adjusted so that the curvature changes between adjacent line segments gradually transition without abrupt turning. It is ensured that the curvatures of any adjacent line segments on the entire path can smoothly connect. The part of the path without sudden changes and sharp corners formed after adjustment is the smooth path segment.

[0136] Based on the smooth path segment, the joint activity range of the robot, the kinematic constraints, the spatial limitations of task execution and other conditions are combined to construct a low-dimensional space that meets the motion characteristics of the robot. All smooth path segments are embedded into the low-dimensional space according to their position relationship and motion logic in the three-dimensional space, so that the embedded path can naturally follow the motion constraints of the robot to form a continuous, smooth and conflict-free motion space representation. This representation is the smooth motion manifold of the robot.

[0137] Along the extension direction of the smooth motion manifold, a large number of continuous path points are uniformly selected according to the preset accuracy standard. The distribution density of these points is sufficient to ensure that the connected trajectory has no obvious breakpoints. Then, a continuous curve is used to connect these selected points in turn to ensure that the curve can closely fit the outline of the smooth motion manifold without deviating from its motion trend. The gap-free, smooth and coherent curve formed finally is the continuous motion trajectory of the robot.

[0138] The beneficial effects are as follows: by converting the driving pulse signal into a discrete spatial path through spatiotemporal pose mapping, the pulse delivery and the motion state of the end effector are accurately correlated; curvature continuity processing eliminates abrupt changes and sharp corners in the path, improving the smoothness of the path; manifold embedding makes the path fit the robot's motion constraints, ensuring the feasibility of the motion; point-by-point fitting finally forms a continuous motion trajectory. The entire process is optimized layer by layer, ensuring the coherence, smoothness and rationality of the motion trajectory, providing a high-quality trajectory foundation for the robot's subsequent joint space mapping and precise execution of control commands, effectively improving the stability of robot motion and the accuracy of task execution.

[0139] S5. Based on the continuous motion trajectory and the biomimetic information processing process, perform joint space mapping on the continuous motion trajectory to obtain the robot's execution control commands;

[0140] In this embodiment of the invention, the step of performing joint space mapping on the continuous motion trajectory based on the continuous motion trajectory and the biomimetic information processing process to obtain the robot's execution control commands includes:

[0141] Task-related feature decoding is performed on the biomimetic information processing process to obtain the biomimetic context information of the biomimetic information processing process;

[0142] Based on the biomimetic context information, the continuous motion trajectory is segmented into task semantics to obtain a set of trajectory segments of the continuous motion trajectory;

[0143] The importance of the trajectory segment set is marked to obtain a hierarchical trajectory segment sequence of the continuous motion trajectory.

[0144] Based on the hierarchical trajectory segment sequence, the trajectory segment sequence is projected into joint space to obtain the joint space path of the robot;

[0145] Based on the biomimetic context information, the joint space path is biomimetically adjusted to obtain the robot's execution control commands.

[0146] The step of biomimeticly adjusting the joint space path based on the biomimetic context information to obtain the robot's execution control commands includes:

[0147] The biomimetic context information is analyzed in a spatiotemporal manner to obtain the motion rhythm synchronization signal of the robot.

[0148] The biomimetic context information is subjected to joint co-compilation to obtain the inter-joint co-pulse code of the robot;

[0149] scaling the joint space path in time axis based on the motion rhythm synchronization signal to obtain a pulse firing synchronization path of the robot;

[0150] performing coordinated gain adjustment on the relative amplitude of joint motion in the pulse firing synchronization path based on the inter-joint coordinated pulse coding to obtain a bionic coupling path of the robot;

[0151] mapping the bionic coupling path to an execution control instruction of the robot.

[0152] In-depth analysis of the pulse firing characteristics, membrane potential change law and inter-pulse correlation mode contained in the bionic information processing process, screening out the key features directly related to the current execution task, including the action type corresponding to the task, the execution accuracy requirement, the motion rhythm and other core elements, systematically integrating and interpreting these key features to form comprehensive information that can clearly reflect the task background, execution requirements and motion correlation. This information is the bionic context information.

[0153] According to the task stage division and action function definition in the bionic context information, identify the trajectory segments corresponding to each task link in the continuous motion trajectory, and divide the entire continuous motion trajectory into multiple trajectory segments with independent functional significance according to the logical order of task execution and the action semantic boundary. The collection of these trajectory segments is the trajectory segment set of the continuous motion trajectory.

[0154] Combined with the core target and execution standard of the current task, evaluate the influence degree of each trajectory segment in the trajectory segment set on completing the task one by one, and determine the functional weight of each trajectory segment. The core trajectory segment that directly determines the success or failure of the task is marked as high importance, the trajectory segment that assists in completing the task is marked as medium importance, and the trajectory segment that has less influence is marked as low importance. All trajectory segments are sequentially arranged in order of importance from high to low to form a hierarchical trajectory segment sequence of the continuous motion trajectory.

[0155] Based on the hierarchical trajectory segment sequence, the position and attitude information of each trajectory segment in the Cartesian space is converted into motion parameters such as angles and angular velocities of the joints of the robot through coordinate conversion and kinematics inverse solution operation. At the same time, according to the importance level of the trajectory segment, the calculation resources are allocated to ensure that the joint parameter conversion accuracy of the high importance trajectory segment is higher. The joint motion parameters corresponding to all trajectory segments are concatenated in time sequence to form a joint motion parameter sequence covering the entire task process. This sequence is the joint space path of the robot.

[0156] Detailed analysis of the time interval, firing frequency, spatial distribution characteristics of the pulse firing in the bionic context information, extract the rhythm rule that can reflect the coordination of biological movement, clear the synchronization relationship between pulse firing and joint movement in time dimension, convert this rhythm rule into a signal with clear time reference, which can guide the joint movement and pulse firing to maintain consistent rhythm, that is, the movement rhythm synchronization signal of the robot.

[0157] Deeply interpret the coordination logic of each joint movement in the bionic context information, including the sequence of joint movement, the proportional relationship of action amplitude, the switching time of movement state and other coordination characteristics, convert these coordination characteristics into standardized pulse sequence coding, each coding corresponds to a set of joint coordination movement rules, through which the coordination constraints of each joint in the movement process can be clearly defined, and the coding is the inter-joint coordination pulse coding of the robot.

[0158] Take the movement rhythm synchronization signal as the time reference, dynamically adjust the time axis of the joint space path, when the synchronization signal indicates that the movement rhythm is accelerated, shorten the time length of the corresponding path segment in proportion, when the synchronization signal indicates that the movement rhythm is slowed down, lengthen the time length of the corresponding path segment in proportion, ensure that the time evolution of the joint space path completely matches the pulse firing rhythm, and the adjusted joint space path is the pulse firing synchronization path of the robot.

[0159] According to the clear joint coordination rules in the inter-joint coordination pulse coding, proportionally adjust the movement amplitude of each joint in the pulse firing synchronization path, enhance the specified key joint movement amplitude in the coordination coding, moderately adjust the movement amplitude of the auxiliary joint, make the relative amplitude of each joint movement meet the coordination constraint requirements, form a highly coordinated and mutually coordinated path, which is the bionic coupling path of the robot.

[0160] Convert the parameters such as joint angle, angular velocity and movement time contained in the bionic coupling path into the control protocol and signal format required by each execution joint of the robot, convert the path information into instruction signals that can directly drive the robot joint actuator to act, and the instruction signals can accurately control the movement state of each joint, which is the execution control instruction of the robot.

[0161] The beneficial effects are that the motion trajectory is accurately matched with the task demand through task-related feature decoding and semantic segmentation; the importance marking and joint space projection ensure the motion accuracy of the core trajectory segment; the bionic adjustment link encodes rhythm synchronization and joint coordination, so that the joint motion is in line with the biological motion characteristics, and the coordination gain adjustment improves the coordination of joint motion; finally, the bionic coupled path is mapped into an execution control instruction, and the whole process fully integrates bionic information and task demand, so that the control instruction has accuracy, coordination and bionics, effectively improves the coordination of robot joint motion and the reliability of task execution, and ensures that the robot action is more in line with the needs of the actual application scene.

[0162] S6, applying the execution control instruction, synchronously collecting the pulse response signal of the robot, and feeding back the pulse response signal to S2 to obtain the optimized control instruction of the robot.

[0163] In the embodiment of the application, the application of the execution control instruction, the synchronous collection of the pulse response signal of the robot, and the feedback of the pulse response signal to S2 to obtain the optimized control instruction of the robot, comprises:

[0164] servo tracking the execution control instruction to obtain the actual joint motion of the robot;

[0165] task space feedforward analysis of the execution control instruction to obtain the end pose change of the robot;

[0166] Based on the actual joint motion and the end pose change, the motion state of the robot is micro-dynamic event conversion to obtain the original pulse response stream of the robot;

[0167] amplitude normalization of the original pulse response stream to obtain the standardized pulse response signal of the robot;

[0168] pulse sparsification of the standardized pulse response signal to obtain the optimized feedback pulse signal of the robot;

[0169] real-time feedback of the optimized feedback pulse signal to obtain the optimized control instruction of the robot.

[0170] The servo system of the robot receives the execution control instruction, and real-time monitors the rotation angle, motion speed, acceleration and other motion parameters of each joint, continuously compares these monitoring data with the target motion parameters set in the instruction, and completely records the real motion trajectory, state change and motion details of each joint during the execution of the instruction. The data set which fully reflects the actual motion of the joint is the actual joint motion of the robot.

[0171] According to the joint movement target, task execution path and precision requirement specified in the execution control instruction, combined with the kinematics model of the robot, the expected position coordinates and attitude parameters of the end effector in the task space are predicted in advance. At the same time, through the positioning and attitude sensor carried by the end effector, the actual position and attitude information of the end effector is collected in real time. The deviation between the expected value and the actual value is calculated, and the dynamic change of the position and attitude of the end effector in the whole task execution process is recorded. These dynamic change information is the end pose change of the robot.

[0172] The detailed data in the actual joint movement, such as the fine adjustment of joint angle, the slight fluctuation of movement speed, and the instantaneous change of acceleration, and the information in the end pose change, such as the slight displacement and attitude deflection, are comprehensively integrated. Each of these independent fine dynamic changes is defined as a micro-dynamic event. The corresponding pulse signal characteristics are matched for each micro-dynamic event. According to the time sequence of the occurrence of the micro-dynamic event, all the corresponding pulse signals are sequentially connected in order to form a continuous and complete pulse data stream. This data stream is the original pulse response stream of the robot.

[0173] The amplitude data of all pulse signals in the original pulse response stream is counted to determine the maximum amplitude and the minimum amplitude. A fixed and unified standard amplitude interval is set. The conversion ratio of the original pulse amplitude to the standard interval is calculated. The amplitude of each pulse signal is adjusted in proportion according to the ratio, so that the amplitudes of all pulse signals are standardized to the set standard interval. The pulse signal set after adjustment, which maintains the original time sequence and logical association, is the standardized pulse response signal.

[0174] The function of each pulse in the standardized pulse response signal is analyzed to identify the core pulse signal that can key reflect the deviation of the robot movement state and has important reference value for the optimization of the subsequent control strategy. The invalid pulse signals that repeatedly appear, have redundant information, and have no substantial contribution to the control decision are removed. Under the premise of ensuring the complete retention of core feedback information, the number and complexity of pulse signals are simplified. The pulse signal set after screening and optimization is the optimized feedback pulse signal of the robot.

[0175] Through the signal transmission channel inside the robot, the optimized feedback pulse signal is transmitted to the pulse-dependent fusion link in real time. The feedback signal is re-coordinated and fused with the original pulse feature sequence. Based on the movement state deviation reflected in the feedback signal, the connection strength of pulse-dependent fusion and the screening standard of membrane potential integration are dynamically adjusted. The integrated output, driving pulse signal and other subsequent parameters can be corrected for movement error. Finally, a new control instruction that can accurately compensate for the deviation and improve the control precision is generated. This new control instruction is the optimized control instruction of the robot.

[0176] The beneficial effect is that the difference between the actual motion state and the target state of the robot is accurately captured through servo tracking and task space feedforward analysis, and a reliable basis is provided for feedback regulation; micro-dynamic event conversion converts motion details into processable pulse signals, amplitude normalization ensures the consistency of the pulse signals, pulse sparsification eliminates redundant information and improves feedback efficiency; the real-time feedback mechanism builds a closed-loop control system, so that the optimized control instruction can dynamically correct the motion deviation, and the adaptability, robustness and precision of the robot control are significantly improved, and the robot can also stably and efficiently complete the task in a complex scene.

[0177] It is apparent for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0178] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving the environment, acquiring knowledge and using the knowledge to obtain the best results.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A robot control method based on a spiking neural network, characterized by, The method comprises: S1, spatiotemporal feature coding of environment perception information and ontology state information of the robot to obtain a pulse feature sequence of the robot; S2, pulse-dependent fusion of the pulse feature sequence and the feedback pulse signal of the robot to obtain a bionic information processing process of the robot, and membrane potential integration of the bionic information processing process to obtain an integrated output quantity of the robot, comprising: bionic time sequence alignment of the pulse feature sequence and the feedback pulse signal of the robot to obtain a synchronous pulse group of the robot; based on the synchronous pulse group, dependent relationship evolution of the pulse feature sequence to obtain the connection strength of the pulse feature sequence; based on the connection strength, shaping of the firing behavior of the synchronous pulse group to obtain the bionic information processing process of the robot; spatiotemporal accumulation of local potential changes in the bionic information processing process to obtain a cumulative membrane potential field of the bionic information processing process; field domain screening of the cumulative membrane potential field to obtain a gating potential field of the cumulative membrane potential field; transmembrane phase convergence of the gating potential field to obtain the integrated output quantity of the robot; S3, pulse firing processing of the integrated output quantity to obtain a driving pulse signal of the robot, comprising: membrane potential field construction of the integrated output quantity to obtain a spatial membrane potential field of the integrated output quantity; bionic evolution of the spatial membrane potential field to obtain an updated membrane potential field of the spatial membrane potential field; based on the updated membrane potential field, extreme region detection of the robot to obtain a candidate firing point of the robot; competitive arbitration of the candidate firing point to obtain a priority firing point of the candidate firing point; based on the priority firing point, driving output of the pulse firing of the robot to obtain the driving pulse signal of the robot; S4, path fitting of the driving pulse signal to obtain a continuous motion trajectory of the robot; S5, based on the continuous motion trajectory and the bionic information processing process, joint space mapping of the continuous motion trajectory to obtain an execution control instruction of the robot; S6, application of the execution control instruction, synchronous acquisition of the pulse response signal of the robot, and feedback of the pulse response signal to S2 to obtain an optimized control instruction of the robot.

2. The pulse neural network-based robot control method of claim 1, wherein, The spatiotemporal feature coding of the environment perception information and the ontology state information of the robot to obtain the pulse feature sequence of the robot comprises: asynchronous event-driven sampling of the robot to obtain the environment perception information of the robot; event-based sparse representation of the environment perception information to obtain a spatiotemporal event stream of the robot; pose acquisition of the robot to obtain the ontology state information of the robot; joint cooperativity analysis of the ontology state information to obtain an ontology motion stream of the robot; based on the spatiotemporal event stream and the ontology motion stream, pulse coding of the robot to obtain the pulse feature sequence of the robot.

3. The pulse neural network based robot control method of claim 1, wherein, The competitive arbitration of the candidate issuing point position comprises: Based on the updated membrane potential field, the potential amplitude of the candidate issuing point position is extracted to obtain the instantaneous membrane potential amplitude of the candidate issuing point position; Based on the bionic information processing process of the robot, the historical issuing time of the candidate issuing point position is searched to obtain the historical issuing time of the candidate issuing point position; The instantaneous membrane potential amplitude and the historical issuing time are weighted and evaluated to obtain the issuing priority score of the candidate issuing point position; Based on the issuing priority score, the candidate issuing point position is globally competitively sorted to obtain the competitive dominance relationship of the candidate issuing point position; Based on the competitive dominance relationship, the conflict of the candidate issuing point position is analyzed to obtain the priority issuing point position of the candidate issuing point position.

4. The pulse neural network-based robot control method of claim 3, wherein, The calculation formula of the issuing priority score is as follows: ； wherein, is the release priority score of the candidate release point position, is the release priority score of the candidate release point position, is the release priority score of the candidate release point position, is the release priority score of the candidate release point position, is the preset weight coefficient, is the preset weight coefficient, is the preset time decay coefficient.​​ 5. The pulse neural network based robot control method of claim 1, wherein, The path fitting of the driving pulse signal comprises: The time and space pose mapping of the issuing time sequence of the driving pulse signal is performed to obtain the discrete space path of the end effector of the robot; The curvature continuous processing of the discrete space path of the end effector is performed to obtain the smooth path segment of the robot; Based on the smooth path segment, the robot is embedded into a flow form to obtain a smooth motion flow form of the robot; The trajectory point-by-point fitting of the smooth motion flow form is performed to obtain the continuous motion trajectory of the robot.

6. The pulse neural network based robot control method of claim 1, wherein, The joint space mapping of the continuous motion trajectory based on the continuous motion trajectory and the bionic information processing process is performed to obtain the execution control instruction of the robot, comprising: The task-related feature decoding of the bionic information processing process is performed to obtain the bionic context information of the bionic information processing process; Based on the bionic context information, the continuous motion trajectory is segmented in task semantics to obtain a trajectory segment set of the continuous motion trajectory; The importance marking of the trajectory segment set is performed to obtain a hierarchical trajectory segment sequence of the continuous motion trajectory; Based on the hierarchical trajectory segment sequence, the joint space projection of the trajectory segment sequence is performed to obtain the joint space path of the robot; Based on the bionic context information, the bionic adjustment of the joint space path is performed to obtain the execution control instruction of the robot.

7. The pulse neural network based robot control method of claim 6, wherein, The bionic adjustment of the joint space path based on the bionic context information to obtain the execution control instruction of the robot comprises: The spatiotemporal analysis of the pulse issuing of the bionic context information is performed to obtain the motion rhythm synchronization signal of the robot; The joint coordination compilation of the bionic context information is performed to obtain the inter-joint coordination pulse coding of the robot; Based on the motion rhythm synchronization signal, the time axis dynamic scaling of the joint space path is performed to obtain the pulse issuing synchronization path of the robot; Based on the inter-joint coordination pulse coding, the relative amplitude of the joint motion in the pulse issuing synchronization path is adjusted by a coordination gain to obtain the bionic coupling path of the robot; Map the bionic coupling path to the execution control instruction of the robot.

8. The pulse neural network based robot control method of claim 1, wherein, The application of the execution control instruction, synchronous acquisition of the pulse response signal of the robot, and the feedback of the pulse response signal to S2 to obtain the optimization control instruction of the robot, comprising: Servo tracking of the execution control instruction to obtain the actual joint motion of the robot; Task space feedforward analysis of the execution control instruction to obtain the end pose change of the robot; Based on the actual joint motion and the end pose change, the motion state of the robot is micro-dynamic event conversion to obtain the original pulse response flow of the robot; Amplitude normalization of the original pulse response flow to obtain the standardized pulse response signal of the robot; Pulse sparsification of the standardized pulse response signal to obtain the optimized feedback pulse signal of the robot; Real-time feedback of the optimized feedback pulse signal to obtain the optimization control instruction of the robot.

Images