A method and system for risk prediction and safety control of an industrial humanoid robot for complex industrial environments

Abstract

This application belongs to the field of robot control technology and discloses a method and system for risk prediction and safety control of industrial humanoid robots in complex industrial environments. The method includes: dividing the robot's hand perception area and setting area perception confidence parameters; real-time acquisition and preprocessing of multimodal tactile time-series data (normal force, shear force, micro-vibration, and temperature) output by the electronic skin during operation; identifying contact events and contact areas based on this data and extracting a set of contact state time-series features; subsequently modulating this feature set using the area perception confidence parameters; predicting contact instability risks based on the modulated feature set; and finally combining the prediction results with the area perception confidence parameters to generate a safety control strategy. This invention achieves differentiated early identification and safety control of potential hazards in different contact areas, improving the safety, stability, and operational efficiency of robots in complex industrial scenarios.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a method and system for risk prediction and safety control of industrial humanoid robots in complex industrial environments. Background Technology

[0002] In complex industrial environments, industrial humanoid robots are widely used for tasks such as grasping, assembly, handling, and human-robot collaboration. The frequent contact between their hands and workpieces, equipment, or human bodies makes safety control a core technological requirement. To achieve tactile perception, robot hands are typically equipped with flexible electronic skin; however, current technologies still present many unresolved issues.

[0003] On the one hand, the electronic skin's sensing capabilities are naturally uneven across different areas of a robot's hand. Areas like the fingertips and pads have high spatial resolution and low signal-to-noise ratio, resulting in reliable sensing. However, the joints and transition areas are prone to deformation and noise due to their high structural flexibility, leading to unstable tactile signal responses. Existing risk assessment methods often treat tactile information from different areas as equivalent, failing to consider the differences in sensing capabilities, thus distorting risk assessments.

[0004] On the other hand, the risk of contact instability does not occur instantaneously, but rather evolves through a process of stable contact, approaching the friction boundary, micro-slippage, and finally obvious instability. Existing technologies mostly rely on static threshold judgments and only take control measures after instability occurs, resulting in a delayed safety response. At the same time, instability precursors in low-sensitivity areas are easily masked by noise, posing a risk of missed detection, while in high-sensitivity areas, the algorithm is overly conservative and frequently causes false stops, seriously affecting operational efficiency. Summary of the Invention

[0005] In response, this application provides a method and system for risk prediction and safety control of industrial humanoid robots in complex industrial environments, so as to at least partially solve the above-mentioned technical problems.

[0006] This application provides a risk prediction and safety control method for industrial humanoid robots in complex industrial environments. The robot hand is equipped with flexible electronic skin, comprising: dividing the robot hand into sensing regions and setting a region sensing confidence parameter for each sensing region; during robot operation, real-time acquisition and preprocessing of multimodal tactile time-series data output by the electronic skin; the multimodal tactile time-series data including at least: normal force signal, shear force signal, micro-vibration signal, and temperature signal; and based on the multimodal tactile time-series data, identifying contact events and confirming the occurrence of contact. The system identifies the contact area; extracts a set of contact state temporal features from the corresponding multimodal tactile temporal data based on the confirmed contact area; modulates the contact state temporal feature set based on the area perception confidence parameter of the confirmed contact area to obtain a contact state temporal feature set under area confidence constraints; predicts the instability risk of the contact event based on the contact state temporal feature set under area confidence constraints; and generates a safety control strategy based on the predicted instability risk and the area perception confidence parameter of the confirmed contact area.

[0007] In one possible embodiment, setting the region perception confidence parameter for each perception region includes:

[0008] An initial area perception confidence parameter is set for the sensing area based on at least one of the following factors: spatial resolution of the electronic skin in the corresponding sensing area, statistical results of historical noise level, and structural compliance.

[0009] In one possible embodiment, the contact state timing feature set includes at least: normal force change rate, shear force direction stability, energy change of micro-vibration signal in a predetermined frequency band, and temperature change slope.

[0010] In one possible embodiment, modulating the contact state timing feature set specifically includes:

[0011] Based on the comparison results between the area perception confidence parameter of the confirmed contact area and the preset confidence threshold, modulation weights are determined for each feature component in the contact state temporal feature set.

[0012] The contact state time series feature set is obtained by multiplying each feature component in the contact state time series feature set with its corresponding modulation weight.

[0013] In one possible embodiment, modulation weights are determined for each feature component, specifically as follows:

[0014] When the area-aware confidence parameter is less than the preset confidence threshold, the modulation weight determined for each feature component is greater than 1; when the area-aware confidence parameter is greater than or equal to the preset confidence threshold, the modulation weight determined for each feature component is less than or equal to 1.

[0015] In one possible embodiment, predicting the instability risk of the contact event specifically includes:

[0016] The set of contact state time-series features under the region confidence constraint is arranged in time sequence and input into a pre-trained time-series prediction model; the instability evolution probability and prediction confidence output by the time-series prediction model are obtained; based on the instability evolution probability, the prediction confidence and the preset judgment threshold, the prediction result of the instability risk is obtained.

[0017] In one possible embodiment, generating a security control policy specifically includes:

[0018] When the prediction result indicates that there is a risk of instability, and the area perception confidence parameter of the area where the contact is confirmed is less than the preset confidence threshold, a safety control strategy is generated that includes reducing contact stiffness, limiting the speed of movement in the shear direction, and adjusting the hand posture.

[0019] When the prediction result indicates an instability risk, and the area perception confidence parameter is greater than or equal to the preset confidence threshold, a safety control strategy is generated that includes moderately reducing contact stiffness and / or fine-tuning hand posture.

[0020] In another aspect, this application also provides a risk prediction and safety control system for industrial humanoid robots in complex industrial environments, comprising:

[0021] The perception region division and confidence setting module is used to divide the robot hand into perception regions and set the region perception confidence parameters for each perception region.

[0022] The multimodal tactile data acquisition and preprocessing module is used to acquire and preprocess multimodal tactile time-series data output by the electronic skin in real time during robot operation; the multimodal tactile time-series data includes at least: normal force signal, shear force signal, micro-vibration signal and temperature signal;

[0023] The contact event recognition and area confirmation module is used to identify contact events and confirm the sensing area where contact occurs based on the multimodal tactile timing data.

[0024] The contact state feature extraction module is used to extract a set of contact state temporal features from the corresponding multimodal tactile temporal data based on the sensory area where contact has been confirmed to have occurred.

[0025] The confidence modulation module is used to modulate the contact state temporal feature set based on the regional perception confidence parameter of the confirmed contact area to obtain the contact state temporal feature set under regional confidence constraints.

[0026] The instability risk prediction module is used to predict the instability risk of the contact event based on the contact state time series feature set under the region confidence constraint.

[0027] The safety control strategy generation module is used to generate a safety control strategy based on the prediction results of the instability risk and the area perception confidence parameters of the perceived area where the contact has been confirmed.

[0028] In another aspect, this application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the risk prediction and safety control method for industrial humanoid robots in complex industrial environments as described above.

[0029] Another aspect of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the risk prediction and safety control method for industrial humanoid robots in complex industrial environments as described above.

[0030] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the risk prediction and safety control method for industrial humanoid robots in complex industrial environments as described above.

[0031] This application achieves early identification and differentiated safety control of robot contact instability risks in complex industrial environments through a series of steps, including perception region division and confidence parameter setting, multimodal tactile data acquisition and preprocessing, contact event identification and region confirmation, contact state feature extraction, feature modulation based on region confidence, instability risk time-series prediction, and generation of region-risk coupled control strategies. This method deeply integrates region perception confidence into the risk formation process, effectively solving problems such as missed detection of instability precursors in low-perception regions, excessively conservative erroneous stops in high-perception regions, and delayed safety response. It can amplify potential risk signals in low-perception regions and suppress excessively conservative control in high-perception regions, elevating risk judgment from a static threshold to a dynamic evolution judgment under region constraints. This significantly improves the safety, stability, and operational efficiency of industrial humanoid robots in complex contact scenarios. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0034] Figure 1 This is a schematic diagram illustrating a risk prediction and safety control method for industrial humanoid robots in complex industrial environments, provided as an embodiment of this application.

[0035] Figure 2 This is a schematic diagram illustrating the process of region division and confidence setting provided in the embodiments of this application.

[0036] Figure 3 This is a schematic diagram of the feature extraction process provided in an embodiment of this application.

[0037] Figure 4 This is a schematic diagram of the contact state modulation process provided in an embodiment of this application.

[0038] Figure 5 This is a schematic diagram of the security control strategy generation process provided in the embodiments of this application.

[0039] Figure 6 This is a schematic diagram of the structure of a risk prediction and safety control system for industrial humanoid robots in complex industrial environments, provided in an embodiment of this application.

[0040] Figure 7 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] It should be noted that all user information (including but not limited to user device information, user personal information, object information corresponding to device usage data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, device usage data, etc.) involved in all embodiments of this application are information and data authorized by the user or fully authorized by all parties.

[0043] The risk prediction and safety control method for industrial humanoid robots disclosed in this application is applicable to industrial humanoid robots equipped with flexible electronic skin. The electronic skin is applied to areas such as the fingertips, fingertips, knuckles, and palm of the robot's hand. The execution entity is the industrial humanoid robot's body control system, which includes a tactile information processing module, a risk prediction module, and a safety control decision-making module. These modules can be integrated into the same control unit or implemented in a distributed manner. This method is suitable for complex industrial environments where robots perform tasks such as grasping, assembly, handling, and human-robot collaboration, which involve frequent contact and potential risks of pinching, scratching, and dragging. It can operate without relying on additional detection equipment and can achieve differentiated early identification and safety control of potential hazards in different contact areas.

[0044] The following detailed description, in conjunction with specific embodiments, illustrates the implementation process of the risk prediction and safety control method for industrial humanoid robots in complex industrial environments described in this application. It should be noted that these embodiments are merely for explaining this application and not for limiting its scope of protection. Any conventional adjustments or substitutions made by those skilled in the art to the steps without departing from the concept of this application should be included within the scope of protection of this application.

[0045] like Figure 1 As shown in the figure, this application discloses a schematic diagram 100 of a risk prediction and safety control method for industrial humanoid robots in complex industrial environments. The robot's hand is equipped with flexible electronic skin, and the method includes the following steps:

[0046] S1: Divide the robot hand into perception regions and set a region perception confidence parameter for each perception region;

[0047] S2: During the robot's operation, the multimodal tactile timing data output by the electronic skin is collected in real time and preprocessed; the multimodal tactile timing data includes at least: normal force signal, shear force signal, micro-vibration signal and temperature signal;

[0048] S3: Based on the multimodal tactile timing data, identify contact events and confirm the sensing area where contact occurs;

[0049] S4: Based on the confirmed sensory area where contact has occurred, extract the contact state temporal feature set from the corresponding multimodal tactile temporal data;

[0050] S5: Based on the region perception confidence parameter of the confirmed contact area, the contact state temporal feature set is modulated to obtain the contact state temporal feature set under region confidence constraint.

[0051] S6: Based on the set of temporal features of the contact state under the aforementioned regional confidence constraint, predict the instability risk of the contact event;

[0052] S7: Generate a safety control strategy based on the prediction results of the instability risk and the area perception confidence parameter of the perceived area where the contact has been confirmed.

[0053] In some embodiments, the purpose of step S1, which involves dividing the perception area and setting the perception confidence parameter, is to clarify the perception attributes of different areas of the robot hand. By scientifically dividing the perception area and setting dynamically adjustable perception confidence parameters, a structured constraint basis is provided for subsequent contact state modulation, risk prediction, and safety control strategy generation, thus solving the problem of risk assessment distortion caused by ignoring the differences in perception capabilities of different areas of the hand in the prior art.

[0054] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the region division and confidence setting process provided in this application embodiment. In S201, the sensing region is divided. Specifically, based on the actual deployment structure of the electronic skin of the robot hand and following the principle of physical position continuity, the robot hand is divided into multiple independent sensing regions. During the division process, it is necessary to ensure that each sensing region corresponds to a set of spatially continuous sensing units with similar functional attributes within the electronic skin, avoiding situations where sensing units cross-regional affiliation or blurred region boundaries.

[0055] For example, the divided sensing areas include at least the fingertip area, fingertip area, knuckle area, and palm area. The fingertip area corresponds to the set of sensing units on the tip of the robot's finger, where sensing units are densely distributed and directly participate in fine contact actions. The fingertip area is the set of sensing units on the fingertip, which is the main contact area for actions such as grasping and pressing. The knuckle area covers the sensing units at the finger joints and joint transitions, and this area has structural deformation characteristics due to joint movement. The palm area corresponds to the entire sensing units on the inner side of the palm, mainly undertaking the tactile sensing function during large-area contact. In some embodiments, secondary sensing areas such as the finger side area and palm heel area can be further subdivided according to the density of the electronic skin and actual operational needs. Each secondary sensing area also follows the division rule of physical position continuity.

[0056] In S202, the area identification and mapping relationship are constructed. Specifically, (1) the area identification data is generated. A unique area identification is assigned to each divided area. The area identification adopts a standardized coding form, which can be a numerical code, an alphanumeric code, or a combination of numbers and letters. The coding rules must ensure the uniqueness and recognizability of the identification. For each electronic skin sensing unit, its sensing area is determined by reading its factory-preset physical address or location coordinate information, and the unique identification of the area is bound to the physical address or location coordinate of the sensing unit to form the sensing area identification data. This identification data will serve as the core basis for quickly locating the contact area when a subsequent contact event occurs.

[0057] (2) Creation of the Region Mapping Table. A region mapping table is constructed in the form of a table or array. The core function of this mapping table is to store the correspondence between sensing units and region identifiers. The mapping table must include at least a sensing unit identifier field and a region identifier field. The sensing unit identifier field records the unique identification information of each sensing unit, such as physical address and location coordinate code. The region identifier field records the unique identifier of the sensing area to which the sensing unit belongs. In terms of data structure design, a hash table can be used, with the sensing unit identifier as the key and the region identifier as the corresponding value. This ensures that when a contact event occurs, the corresponding region identifier can be quickly retrieved through the sensing unit identifier, improving region identification efficiency. After the mapping table is created, it is stored in the robot controller's local storage module or cache for quick retrieval in subsequent steps.

[0058] In S203, the confidence parameter setting for area perception is as follows: (1) Initial parameter setting: The confidence parameter for area perception is the core parameter characterizing the reliability of the tactile signal in the corresponding perception area. Its initial setting can take into account the inherent properties and long-term operating characteristics of the electronic skin in the area, specifically including the spatial resolution of the electronic skin in the area, the statistical results of the noise level in the area during long-term operation, and the influence of the structural compliance of the area on the stability of the tactile signal. The higher the spatial resolution, the higher the accuracy of the sensing unit in capturing the contact signal, the stronger the signal reliability, and the higher the corresponding confidence parameter value; the lower the noise level, the less the signal is interfered with, and the higher the confidence parameter value; the stronger the structural compliance, the greater the signal fluctuation caused by structural deformation during contact, the worse the signal stability, and the lower the confidence parameter value.

[0059] The area perception confidence parameter is initialized to a preset numerical value, configurable within the range of [0,1]. A value closer to 1 indicates higher reliability of the tactile signal in that area, while a value closer to 0 indicates lower reliability. For example, the initial confidence parameter for the fingertip area, due to its high spatial resolution and low noise level, can be set to 0.85-0.95; for the fingertip area, 0.8-0.9; for the knuckle area, 0.4-0.6; and for the palm area, 0.6-0.75. During parameter initialization, the parameters are entered through the robot control system's parameter configuration interface, stored in the system parameter library, and associated with the corresponding area identifier.

[0060] Optionally, the area perception confidence parameter is not fixed but dynamically updated based on long-term operational data to reflect the true trend of changes in area tactile performance. The update cycle can be configured as a preset time period, such as once a week, or a work cycle, such as after every 1000 touch operations. During the update process, the system automatically extracts the tactile signal data of the corresponding area within that cycle, re-calculates indicators such as spatial resolution effectiveness, noise level, and signal fluctuation amplitude, and adjusts the initial confidence parameter using a weighted average algorithm. The adjustment formula is: in, For the updated region-aware confidence parameters, The parameter values ​​before the update. This is the confidence level value calculated based on the data from the current period. This is a weighting coefficient, configurable from 0.7 to 0.9, used to balance the influence of historical parameter values ​​and current calculated values, ensuring the smoothness and stability of parameter updates. For example, if the initial confidence parameter for a certain knuckle region is 0.5, and the confidence value calculated in the current period is 0.45, the weighting coefficient... If we take 0.8, the updated parameter value will be 0.8×0.5+0.2×0.45=0.49. This slow adjustment method avoids the impact of sudden parameter changes on risk assessment.

[0061] After this step is completed, the robot control system will generate and store three sets of core data: the hand perception area division results, the initial perception confidence parameters for each perception area, and the area mapping table. This data will serve as the foundation for subsequent steps such as multimodal tactile information processing, contact area recognition, and contact state modulation.

[0062] In some embodiments, step S2 involves multimodal tactile timing data acquisition and preprocessing. The aim is to acquire complete tactile signal data during the robot hand's contact process and improve data quality through preprocessing, providing reliable raw data support for subsequent contact state feature extraction and risk prediction.

[0063] Once the robot initiates its task, the control system automatically triggers a multimodal tactile signal acquisition process. This acquisition is synchronized with the robot's hand movements, ensuring real-time capture of tactile signal changes when a contact event occurs. The acquisition frequency is set according to the dynamic characteristics of the task. For example, for highly dynamic tasks such as high-speed grasping and precision assembly, a higher acquisition frequency, such as 500Hz-1000Hz, is used to avoid signal distortion. For tasks such as low-speed handling and static pressing, a lower acquisition frequency, such as 100Hz-300Hz, can be used to reduce the system's computational load.

[0064] The acquired multimodal tactile time-series data includes at least four types: normal force signal, shear force signal, micro-vibration signal, and temperature signal. The normal force signal is the force perpendicular to the contact surface, reflecting the pressure between the robot hand and the contact object. The shear force signal is the force parallel to the contact surface, including information such as shear force amplitude and direction, and is a key signal for determining whether there is a slippage tendency. The micro-vibration signal is a high-frequency, minute vibration signal generated during the contact process, reflecting the friction state and contact stability of the contact surface. The temperature signal is used to distinguish whether the contact object is a living organism, such as a human body, or a non-living organism, such as a workpiece or equipment, avoiding the risk of misjudging contact due to environmental temperature disturbances. All signals are output by the sensing units of the flexible electronic skin deployed on the robot hand. Each sensing unit simultaneously acquires the above four types of signals to ensure the spatiotemporal consistency of the signals.

[0065] The data source is the various sensing units of the robot's electronic skin. Each sensing unit has an independent data output interface, transmitting signal data in real time to the tactile information processing module of the control system via a bus transmission method. Data storage adopts a timestamp-aligned multidimensional time-series data structure. Each time point corresponds to a set of five-dimensional data including normal force, shear force amplitude, shear force direction, micro-vibration signal, and temperature signal. The timestamp accuracy can be configured to milliseconds, such as 1ms, ensuring strict alignment of different modal signals in the time dimension. For example, at timestamp t=100ms, the corresponding dataset is ( , , , , ),in Let be the value of the normal force signal at time t. Let be the amplitude of the shear force at time t. Let be the direction of the shear force at time t. Let be the value of the micro-vibration signal at time t. Let t be the temperature signal value at time t.

[0066] Optionally, statistical filtering can be used to remove obvious outliers in the acquired signals. The specific operation is as follows: First, calculate the mean value of each modal signal within a preset sliding time window, such as 100ms. and standard deviation Then, within the window that exceeds [ Signal values ​​within a certain range are identified as outliers. For outliers, a linear interpolation method between two adjacent normal signal values ​​is used for replacement.

[0067] Although signal synchronization was ensured as much as possible during the acquisition process, slight time deviations may exist between different modal signals due to factors such as transmission delay and differences in the response speed of sensing units. Therefore, further time synchronization calibration is required in the preprocessing stage. Using the timestamp of the normal force signal as a reference, linear interpolation is used to align and adjust the timestamps of the shear force signal, micro-vibration signal, and temperature signal. For a specific target timestamp... If the shear force signal is in The two most recent sampling points before and after are and ,in The corresponding signal value is and ,but The shear force signal value at time t is The other modal signals are synchronized in the same way to ensure that each modal signal accurately corresponds to the same contact state at the same timestamp.

[0068] The synchronized multimodal tactile signals are segmented according to a fixed time window. The length of the time window can be set according to the speed of contact state evolution, for example, it can be configured to 50ms-200ms, preferably 100ms. During the segmentation process, continuous signal data is extracted in units of time windows to form multiple independent signal segments. Each signal segment contains all modal signal data within that time window. For the last incomplete signal segment, i.e., signal data that is less than the length of one time window, if its length is greater than 50% of the time window length (e.g., if the time window is 100ms and the remaining signal segment length is 60ms), the signal segment is retained and padded with zeros to the full time window length; if the remaining length is less than or equal to 50% of the time window length, the signal segment is discarded to avoid short signal segments affecting the accuracy of feature extraction.

[0069] After this step is completed, a multimodal tactile timing data window is generated for subsequent processing. Each data window is a set of multimodal signals that are time-synchronized, have no obvious outliers, and have a fixed length. The control system stores these data windows in a cache module and marks them with additional information such as the data acquisition time and the corresponding sensing unit.

[0070] In some embodiments, for step S3, by analyzing the preprocessed tactile timing data, it is determined whether a real contact event exists, and the hand-sensing area where the contact occurs is located, providing a clear regional basis for subsequent contact state modulation based on region perception confidence. The key to this step lies in the accuracy of contact event judgment and the speed of region positioning, avoiding distortion of subsequent risk assessment due to misjudgment of contact events or incorrect region positioning.

[0071] Specifically, contact event recognition uses the normal force signal as the core criterion for judging contact events, and presets a contact judgment threshold. The threshold value can be set by comprehensively considering the sensing sensitivity of the electronic skin, the level of environmental interference, and the contact pressure range of the robot's operation. For example, it can be configured to be 10%-20% of the minimum range of the electronic skin's sensing unit to ensure effective differentiation between real contact and non-operational contact such as environmental noise and slight touches. For instance, if the electronic skin's normal force sensing range is 0-10N and the minimum range is 0.1N, then the contact determination threshold... It can be set to 0.01N-0.02N.

[0072] The control system monitors the normal force timing data of each sensing unit in real time. If the normal force signal value of a certain sensing unit exceeds the contact determination threshold within a consecutive preset number of sampling points, such as 3 sampling points, the system will determine the contact determination threshold. If the signal shows a continuous upward trend or remains stable above the threshold, a contact event is initially determined to have occurred at the sensing unit. If only one or two sampling points exceed the threshold in terms of normal force signal, and the subsequent signal quickly falls back below the threshold, it is determined to be an interference signal and not considered a contact event. Furthermore, when the normal force signals of multiple adjacent sensing units, such as two or more, simultaneously meet the above-mentioned continuous threshold exceeding condition, it is determined to be a large-area contact event and is also considered valid contact.

[0073] In this process, once a contact event is preliminarily determined, the unique identifier of the sensing unit that has made contact, such as its physical address or location coordinate code, is extracted. The area identifier corresponding to the sensing unit is then queried by calling the area mapping table created in step S1.

[0074] To avoid area location errors caused by the failure of a single sensing unit, the queried contact area can be verified. The verification method is as follows: check if the normal force signal of other adjacent sensing units in the area is close to or exceeds the contact determination threshold. If it reaches more than 80% of the threshold, the area is confirmed as a real contact area. If only the signal of the single sensing unit in contact exceeds the threshold, and the signals of other sensing units in the area are all at normal noise levels, then the historical fault records of the sensing unit are further checked. If there are frequent abnormal records, it is determined to be a sensing unit failure, and the contact area is not confirmed. If the sensing unit's historical status is normal, the contact area is temporarily confirmed and marked as a single sensing unit contact, and further verification is carried out in conjunction with other modal signals.

[0075] After this step is completed, the system will record the current contact state, including the contact time, contact duration, peak normal force, and corresponding sensing area identifier. If no valid contact event is identified, the system will maintain the current state and continue to monitor the tactile signal. If a contact area is confirmed, the contact area identifier and the corresponding contact state data will be transmitted to the subsequent contact state timing feature extraction step.

[0076] In some embodiments, for step S4, for the confirmed contact area, core features that can describe the evolution trend of the contact state are extracted from the preprocessed multimodal tactile time-series data to form a contact state time-series feature set, providing feature input for subsequent contact state modulation and instability risk prediction. The extracted features need to be able to comprehensively reflect the evolution process of contact from stability to instability to ensure the accuracy of subsequent risk prediction.

[0077] Please see Figure 3 , Figure 3 This is a schematic diagram of the feature extraction process provided in an embodiment of this application. Based on the confirmed contact area, signal data of all sensing units within the area are filtered out from the multimodal tactile time-series data window. Taking the moment of contact as the starting point, signal data from one or more subsequent consecutive time windows, such as 1-3 time windows, are selected as the data source for feature extraction, ensuring that the extracted features can cover the state changes from the initial contact stage to the stable stage.

[0078] The specific feature extraction process includes, in step S301, the extraction of the normal force change rate. The normal force change rate describes the rate of change of the normal force signal over time, reflecting the dynamic trend of contact pressure. The calculation method involves, within the feature extraction time window, analyzing the normal force signal sequence... The rate of change between two adjacent sampling points is calculated using the first-order difference method, i.e. Where i = 1, 2, ..., n-1, Let be the timestamp of the i-th sampling point. Then, calculate the mean and standard deviation of all rates of change, which serve as the eigenvalues ​​of the normal force rate of change, i.e., the eigenvector. ,in The mean rate of change This represents the standard deviation of the rate of change. An increase in the rate of change of normal force and a large standard deviation usually indicates unstable contact pressure and a potential tendency towards instability.

[0079] In S302, shear force direction stability is extracted. Shear force direction stability characterizes the degree of fluctuation in the shear force direction and is a key feature for determining whether there is a slippage tendency in the contact. First, the shear force direction is converted into an angle value. The range is 0°-360°, and then the fluctuation amplitude and average change frequency of the angle values ​​within the feature extraction time window are calculated. Fluctuation amplitude ,in This represents the maximum value of the shear force in the direction of the window. Minimum value; average frequency of change , where k is the number of times the shear force direction changes significantly within the window, such as when the angle of change exceeds 10°, and T is the length of the time window. As an eigenvector of shear force direction stability The bigger, The higher the value, the more unstable the shear force direction, and the higher the risk of contact slippage.

[0080] In S303, the energy change of the micro-vibration signal is extracted. First, frequency domain analysis is performed on the micro-vibration signal, and then the time domain signal is converted using Fast Fourier Transform. Convert to frequency domain signal Then, a predetermined frequency band related to contact instability is selected, such as 100Hz-1000Hz. The vibration energy change in this frequency band is usually related to the change in the friction state of the contact surface. The energy value E within this frequency band is calculated. The energy calculation method is as follows: ,in , , Subsequently, the rate of change of energy in this frequency band within adjacent time windows was calculated. ,in The energy value for the k-th time window. Let E be the energy value for the (k+1)th time window. As the characteristic vector of the micro-vibration signal increases, E also becomes more prominent. When the value is positive and large, it indicates that the friction state of the contact surface has changed drastically and may be close to instability.

[0081] In S304, the temperature change slope is extracted. The temperature change slope is used to distinguish between biological contact and environmental disturbance. It is calculated by analyzing the temperature signal within the feature extraction time window. Perform linear fitting to obtain the fitted line. Where k is the slope of the temperature change. The fitting process uses the least squares method to minimize the sum of squared errors between the fitted line and the actual temperature signal. The slope of the temperature change, k, is used as the unique feature value of this characteristic. When the absolute value of k is large, such as... When k approaches 0, it is more likely to be contact with living organisms, such as human skin contact, which can cause a rapid rise or fall in temperature. When k approaches 0, it is more likely to be environmental disturbances or contact with non-living organisms.

[0082] In S305, an unweighted set of contact state time-series features is formed. The extracted features of normal force change rate, shear force direction stability, micro-vibration signal energy change, and temperature change slope are integrated to form an unweighted set of contact state time-series features. Each feature corresponds to a key dimension of the contact state, and together they form a multidimensional feature vector describing the evolution of the contact state over time. This set of features will serve as the input data for subsequent contact state modulation.

[0083] In some embodiments, the core of contact state modulation based on region perception confidence in step S5 lies in overcoming the limitation of existing technologies that merely use region information as an additional label for the final decision. Instead, it uses region perception confidence as a structural constraint for contact state modeling. By differentially modulating the temporal characteristics of the contact state, the same physical contact signal can have different risk interpretation weights in different perception confidence regions. This specifically addresses the technical problems in existing technologies where neglecting the differences in perception capabilities across different hand regions leads to noise masking instability precursors in low-perception-capability regions (i.e., missed risk assessment) and overly conservative algorithms in high-perception-capability regions (i.e., frequent false stops). Through this modulation mechanism, potential risk signals can be amplified in low-perception-confidence regions, while the complete instability feature judgment space can be preserved in high-perception-confidence regions. This achieves more accurate and differentiated risk assessment, providing more realistic feature inputs for subsequent instability risk prediction, ultimately improving the reliability and operational efficiency of robot safety control.

[0084] Please see Figure 4 , Figure 4 This is a schematic diagram of the contact state modulation process provided in an embodiment of this application. In S401, the area perception confidence parameter is obtained. Specifically, parameter query and matching involves retrieving the current area perception confidence parameter corresponding to the contact area identified in step S3 from the system parameter database. The query process uses region identifiers to perform precise matching with region-confidence relationships stored in the parameter database, ensuring that the obtained parameters are the latest dynamically updated values ​​for the contact region, rather than the initial values.

[0085] In S402, the modulation weights are determined. A confidence threshold is preset. It is used to distinguish between low-perceived confidence regions and high-perceived confidence regions. The value can be determined based on the actual operating scenario and the overall perception performance of the electronic skin; for example, it can be configured to 0.7. When the area perception confidence parameter... Less than When, the region is determined to be a low-confidence region; when Greater than or equal to When the time is right, the area is determined to be a high-confidence area.

[0086] Based on region-aware confidence parameters confidence threshold The comparison results are used to determine the modulation weights for each feature component in the contact state temporal feature set. The calculation of the modulation weights follows these rules: when In the low-confidence region, the modulation weights determined for each feature component are greater than 1 to amplify the impact of potential risk signals; when In other words, in the high-confidence region, the modulation weight determined for each feature component is less than or equal to 1, so as to suppress overly sensitive risk judgment and preserve the formation space of complete unstable features.

[0087] Specifically, a modulation weight matrix W is constructed for the temporal features of the contact states. The dimension of the weight matrix is ​​consistent with the dimension of the contact state temporal feature set F, i.e. ,in The modulation weights are the characteristics of the rate of change of normal force. Modulation weights for shear force direction stability characteristics. The modulation weights are used to represent the energy variation characteristics of the micro-vibration signal. The modulation weights are for the slope characteristics of temperature changes.

[0088] The specific calculation method for the weight values ​​is as follows: For regions with low perceived confidence, : , , , For regions with high perceived confidence : , , , in, This represents the weighting factor for the low-confidence region. These are the weighting suppression coefficients for high-confidence regions. All coefficients are positive numbers greater than 0 and can be set according to the degree of influence of different features on risk assessment. The design logic of this weight matrix is ​​as follows: in low-perceived confidence regions, the influence of potential risk signals is amplified by increasing feature weights; in high-perceived confidence regions, overly sensitive risk judgments are suppressed by decreasing feature weights, preserving the formation space of complete instability features.

[0089] Optionally, for regions with low perceived confidence, In this region, tactile signals have low reliability and high noise interference, and early signs of instability are easily masked by noise. Therefore, the modulation logic focuses on amplifying small but persistent risk-related signals to improve the system's sensitivity to potential instability. Specifically, for the normal force rate of change feature, the weights of the mean and standard deviation of the rate of change are amplified, so that even small fluctuations in normal force can be significantly reflected in the feature. For example, when... , hour, The normal force change rate characteristic is amplified by 1.16 times, enabling the system to more sensitively detect the instability risk caused by pressure fluctuations. For the shear force direction stability characteristic, the weights of fluctuation amplitude and average change frequency are significantly increased, because small changes in the shear force direction may be early precursors of slippage and require close attention in low-sensitivity regions, such as... This significantly amplifies slight instabilities in the shear force direction, preventing missed detections due to signal noise. For the energy change characteristics of micro-vibration signals, the weights of energy value and rate of change are amplified. Enhanced micro-vibrations on the contact surface may indicate changes in frictional state; therefore, the influence of this feature needs to be strengthened in low-sensitivity regions, for example... For the slope characteristic of temperature change, the weight should be appropriately increased to ensure effective differentiation between biological contact and environmental disturbance, avoiding misjudgment due to low signal reliability, for example... .

[0090] Optionally, for regions with high perceived confidence, In this region, tactile signals exhibit high reliability and low noise interference, accurately capturing the complete evolution of the contact state. Therefore, the modulation logic focuses on suppressing overly conservative risk assessments, allowing for risk determination only after the complete instability characteristics have formed, thereby improving operational efficiency. Specifically, for the normal force change rate characteristic, the weight is appropriately reduced to avoid triggering unnecessary risk warnings due to minor pressure fluctuations. For example, when... , hour, The influence of the normal force change rate characteristic is slightly suppressed, and the system can tolerate a certain degree of pressure fluctuation; for the shear force direction stability characteristic, the weight is reduced so that only when the shear force direction becomes significantly and persistently unstable will it be judged as a risk signal, for example... To avoid accidental stopping due to brief directional fluctuations; for the energy change characteristics of micro-vibration signals, the weight should be appropriately reduced to ensure that risk assessment is only triggered when vibration energy experiences a significant and sustained increase, for example... For the temperature change slope feature, the weight is slightly reduced because the temperature signal in the high-sensitivity area can accurately distinguish the contact type, and there is no need to over-amplify the influence of this feature. For example... .

[0091] In S403, contact state feature modulation is performed. Specifically, feature-by-feature modulation calculation is performed, multiplying each feature in the contact state temporal feature set F with the corresponding weight in the modulation weight matrix W to obtain the modulated contact state temporal feature. The specific calculation process is as follows: Modulated normal force rate of change feature Modulated shear force directional stability characteristics Energy variation characteristics of modulated micro-vibration signal Modulated temperature change slope characteristics .

[0092] Feature-by-feature modulation The data are integrated to form a set of temporal features of contact states under regional confidence constraints. This feature set has fully incorporated the perceived confidence information of the contact area. The same physical contact state will exhibit different characteristics in different areas. The potential risk signal in the low-perception area is amplified, while the features in the high-perception area remain relatively stable, providing differentiated feature inputs for subsequent instability risk prediction.

[0093] In some embodiments, for step S6, based on the modulated temporal feature set of the contact state, a temporal modeling method is used to predict the short-term evolution trend of the contact state. This overcomes the limitation of existing technologies that rely solely on static thresholds to determine risk, enabling early identification of potential hazardous states that have not yet experienced significant slippage or impact. It solves the technical problem of delayed safety response in existing technologies, which often only take control measures after instability occurs, failing to proactively avoid risks in the irreversible stage before slippage. By analyzing the dynamic evolution trend of the contact state through temporal modeling and combining the modulated features of the regional confidence level, it is possible to accurately determine whether the contact state is developing towards instability and whether this trend constitutes an actual risk under the current regional perception conditions. This provides advance notice for the generation of safety control strategies, ensuring the timeliness and accuracy of safety responses and significantly improving the safety of robots in complex contact scenarios.

[0094] Specifically, for the time-series prediction model, Long Short-Term Memory (LSTM) network is adopted as the core method for time-series modeling. LSTM has the ability to capture long-term dependencies in time-series data and can effectively analyze the evolution trend of contact state characteristics over time, making it suitable for processing the multimodal, long-sequence contact state data in this application. Optionally, the LSTM model has a multi-layer structure, including an input layer, hidden layers, and an output layer. The number of hidden layers can be configured to 2-4 layers, and the number of neurons in each layer can be configured to 64-256. The specific number of layers and neurons can be adjusted according to the complexity of the actual operation scenario and the amount of data. For example, for precision assembly scenarios with strong dynamics and complex contact state changes, 4 hidden layers with 256 neurons per layer can be configured; for handling scenarios with weaker dynamics, 2 hidden layers with 64 neurons per layer can be configured.

[0095] The model input is a set of temporal features of contact states under regional confidence constraints. The input sequence length is the number of feature sets within a preset prediction time window. For example, if the prediction time window is 300ms and the feature extraction time window is 100ms, then the input sequence length is 3, which means 3 consecutive feature sets. The model output is a two-dimensional vector (P, C), where P is the probability of evolving from the contact state to the unstable state, P∈[0,1], and C is the confidence level of the prediction result, C∈[0,1]. The closer P is to 1, the higher the risk of instability; the closer C is to 1, the more reliable the prediction result.

[0096] Model hyperparameters include learning rate, number of iterations, and batch size. The learning rate can be configured from 0.001 to 0.01, preferably 0.005, to ensure the stability and convergence speed of the model training process. The number of iterations can be configured from 100 to 500, preferably 300, to avoid overfitting or underfitting. The batch size can be configured from 16 to 64, preferably 32, to balance training efficiency and model generalization ability. Furthermore, the model uses the ReLU activation function, the Sigmoid activation function for the output layer, the binary cross-entropy loss function, and the Adam optimizer.

[0097] The training process of the model includes: (1) Construction of training dataset: collecting historical contact data of the robot in different work scenarios and different contact areas, including stable contact data, unstable evolution process data, and obvious unstable data. For each group of historical data, preprocessing, feature extraction and state modulation are performed according to the process of steps S2-S5 to form the input feature sequence of the training dataset; the output label is marked according to the actual contact result. If the contact eventually becomes unstable, the label is (1, 1); if the contact is always stable, the label is (0, 1); if the contact state is ambiguous and it is impossible to clearly determine whether it is unstable, the label is (0.5, 0.5). The training dataset is divided into training set and validation set in a ratio of 7:3 for model training and performance verification.

[0098] The training set is input into the LSTM model, and iterative training is performed according to the preset hyperparameters. After each iteration, the model's loss value and prediction accuracy on the validation set are calculated. Training stops when the loss value no longer decreases after 10 consecutive iterations or reaches the preset minimum loss value. Early stopping and dropout techniques are employed during training, with a dropout rate of 0.2, to prevent overfitting. After training, the model parameters are saved, forming the final contact instability risk prediction model. This model can be directly called during robot operation to predict real-time modulated feature sequences.

[0099] The real-time risk prediction execution includes the construction of input feature sequences, which involves extracting the latest N modulation feature sets corresponding to the current contact event from the modulation feature library of the control system, where N is the length of the model input sequence, and arranging them in chronological order to form the input feature sequence.

[0100] The input feature sequence is fed into the trained LSTM risk prediction model, which calculates the output vector (P, C) through forward propagation. During inference, the system records the model's inference time to ensure it does not exceed a preset time, such as 10ms, to meet real-time requirements. If the inference time exceeds 10ms, the model complexity is appropriately reduced, such as by decreasing the number of hidden layer neurons, and the model is retrained before inference.

[0101] Based on the P-value output by the model and the preset risk assessment threshold , It can be configured to 0.6-0.8, preferably 0.7, for preliminary risk assessment: when At that time, it was initially determined that there was a potential risk of instability; when At that time, it was initially determined that there was no potential risk of instability; when P is in [ When the range is within a certain range, it is considered a state of risk ambiguity, and further judgment is needed based on the confidence level C of the prediction result.

[0102] Optionally, the prediction results can be revised based on the prediction confidence level. For cases initially identified as having potential instability risk, if the prediction confidence level... , To predict the confidence threshold, it can be configured to 0.7-0.9, preferably 0.8, which ultimately determines that there is a potential risk of instability; if If the result is positive, the situation is considered suspicious and requires continued monitoring of subsequent feature sequences, with a comprehensive assessment based on the next prediction results.

[0103] If the initial assessment is that there is no potential risk of instability, If so, it is ultimately determined that there is no potential risk of instability; If so, it is also judged as a risky and suspicious state, and its characteristics are continuously monitored for changes.

[0104] In the case of a state of risk ambiguity, if Then according to the P value and The final determination is made based on the size relationship. It was determined that there was a risk. It is determined that there is no risk; if If so, continue monitoring until a clear prediction is obtained or the contact event ends.

[0105] In one embodiment, considering the differences in the instability evolution patterns of different contact areas, the final judgment result is revised a second time: for areas with low perception confidence, If the initial assessment is that there is no risk but If the risk level is low, the status will be revised to "suspected risk," and the monitoring period will be extended. If a risk is determined to exist, even if C is slightly lower than the threshold, the monitoring period will be extended. ,like This also maintains the assessment that risks exist, avoiding missed assessments.

[0106] For regions with high perceived confidence If the system is deemed to be in a state of suspected risk, the monitoring frequency can be appropriately reduced to avoid excessive consumption of system resources; if the system is deemed to be in a state of risk, it is necessary to ensure... Otherwise, the status will be changed to "risky but suspected" to avoid misjudgment.

[0107] After this step is completed, the final result of the contact instability risk prediction is generated, including information such as whether there is a potential instability risk and the confidence level of the prediction result. It is then associated with the contact area identifier and modulation feature set and stored in the risk prediction result library of the control system.

[0108] In some embodiments, for step S7, a region-risk coupled decision-making mechanism is constructed based on the perception confidence level of the contact area, the instability risk prediction result, and the prediction confidence level, generating differentiated safety control strategies. This solves the problem of insufficient flexibility in safety control strategies in existing technologies. Existing methods either employ conservative strategies in all areas, leading to inefficiency, or use uniform strategies that cannot adapt to the differences in perception capabilities across different areas. By coupling region characteristics with risk status, a conservative strategy is adopted in areas with low perception confidence and instability risk to ensure safety; a flexible adjustment strategy is adopted in areas with high perception confidence and instability risk to balance safety and efficiency; and normal operation is maintained in the absence of risk to ensure operational continuity. This strategy generation mechanism achieves precise and differentiated safety control, avoiding the risk of missed detection in low perception areas and improving operational efficiency in high perception areas, fully leveraging the robot's operational flexibility.

[0109] Please see Figure 5 , Figure 5 This is a schematic diagram of the security control strategy generation process provided in an embodiment of this application. In S501, core decision parameters are extracted. Three sets of core decision parameters are extracted from various modules of the system: one is the perception confidence parameter of the contact area. The parameters are: 1) from step S5; 2) the instability risk prediction result R, R∈{risk exists, risk does not exist, risk is questionable}; and 3) the confidence level C of the prediction result, from step S6. These three sets of parameters are then integrated into a decision parameter vector. ), which serves as the input for generating security control strategies.

[0110] Optionally, the parameter priority can be set as follows: instability risk prediction result R > prediction confidence level C > area perception confidence level That is, first determine whether safety control measures are needed based on R, then determine the strength of the control measures based on C, and finally combine... Adjust the specific type of control strategy to ensure the rationality and relevance of the decision-making logic. For example, when R represents a risk, regardless of... The specific values ​​of C and must be determined with safety control measures in place; only C and are considered. Adjust the intensity and type of measures; when R is no risk, only apply measures when C is low and Weak monitoring and control measures are only implemented in cases of low risk; otherwise, normal operations are maintained.

[0111] In S502, safety control strategies are classified and decision rules are designed. Safety control strategies are divided into three categories: conservative safety control strategies, flexible adjustment safety control strategies, and maintenance control strategies. The specific content of each strategy is as follows: For conservative safety control strategies: applicable to scenarios with low perceived confidence and a clear risk of instability, the core objective is to maximize safety and avoid danger. Specific measures include: actively reducing contact stiffness to a preset minimum value, such as 30%-50% of normal contact stiffness; limiting the shear direction movement speed to a preset safe speed, such as 20%-40% of normal speed; adjusting hand posture to make the contact surface perpendicular to the force direction, reducing the risk of frictional boundaries; and considering the risk prediction confidence level. If this happens, a safety deceleration will be triggered directly until the robot's hand stops moving.

[0112] For flexible adjustment-type safety control strategies: applicable to scenarios with high perception confidence and instability risks, the core objective is to maintain operational continuity and flexibility as much as possible while controlling risks. Specific measures include: appropriately reducing contact stiffness, such as to 70%-80% of normal contact stiffness; limiting the shear direction movement speed to 60%-80% of normal speed; fine-tuning hand posture to optimize contact force distribution; and continuously monitoring contact state characteristics. If the risk prediction results continue to deteriorate, such as the P-value continuously rising above 0.8, the strategy is upgraded to a conservative approach.

[0113] For maintenance-type control strategies: These are suitable for scenarios where there is no risk of instability or the risk is questionable. The core objective is to maintain the current operational state and ensure operational efficiency. Specific measures include: maintaining the current contact stiffness, movement speed, and hand posture unchanged; continuously collecting and analyzing tactile signals; if the questionable risk state persists for more than a preset time, such as 500ms, then initiating weak intensity monitoring, such as increasing the feature extraction frequency, until the contact event ends or the risk state is clarified.

[0114] In one embodiment, based on the decision parameter vector ( Based on the parameters (R, C) and their priorities, the following specific decision rules are designed, for example:

[0115] Rule 1: When R = risk exists

[0116] like Low-confidence areas:

[0117] like Implement a complete conservative safety control strategy, including reducing stiffness, limiting speed, and adjusting attitude. If so, a safety stop is triggered; Implement conservative strategies such as stiffness reduction, speed limiting, and attitude adjustment to avoid triggering a safety stop and continuously monitor changes in risk; if Implement conservative strategies such as stiffness reduction and attitude adjustment, moderately limit the shear direction velocity to 50% of the normal velocity, and increase the subsequent prediction frequency, such as shortening the prediction time window to 200ms.

[0118] like High-confidence perception region:

[0119] like Implement a comprehensive, flexible, and adaptable safety control strategy, and continuously monitor risks; if : Implement stiffness reduction and fine-tuning attitude measures in the flexible adjustment strategy, without limiting motion speed, to increase feature extraction frequency; if Implement a maintenance control strategy, while doubling the prediction frequency. If the subsequent prediction result R still indicates a risk, and If so, it will be upgraded to a flexible adjustment strategy.

[0120] Rule 2: When R = no risk

[0121] like Implement a maintenance control strategy and continue according to normal operating procedures;

[0122] like Implement a maintenance control strategy, increase the feature extraction frequency, such as 1.5 times the original frequency, and continuously monitor;

[0123] like :

[0124] like Implement a maintenance control strategy while appropriately reducing contact stiffness to 80% of normal stiffness and strengthening monitoring; if : Implement a maintenance control strategy, only increase the feature extraction frequency, without adjusting the contact parameters.

[0125] Rule 3: When R = Questionable Risk

[0126] like Implement a maintenance control strategy, increasing the forecast frequency to twice the original frequency. If three consecutive forecasts still indicate a questionable risk, then adjust the strategy based on R = no risk and Rule processing;

[0127] like :

[0128] like Implement a maintenance control strategy to reduce contact stiffness to 70% of normal stiffness, thereby increasing feature extraction and prediction frequency; if : Implement a maintenance control strategy, only increase the frequency of feature extraction and prediction, and continue monitoring until the contact ends.

[0129] In S503, control strategy parameter calculation and command generation specifically involve quantifying the parameters of various control strategies based on the aforementioned decision rules, including contact stiffness adjustment parameters: assuming the normal contact stiffness is... The target stiffness of a conservative strategy ,in The specific values ​​are adjusted based on C and P; the higher C is, the higher P is. The smaller the target stiffness of the flexible adjustment strategy, the better. ,in Similarly, the stiffness of the maintenance strategy is adjusted according to C. .

[0130] Shear direction velocity adjustment parameters: Let the normal shear direction velocity be v0, and the target velocity of the conservative strategy be v0. The target speed of the flexible adjustment strategy Speed ​​of maintenance strategy .

[0131] Hand posture adjustment parameters: The target posture angle is calculated based on the normal vector of the current contact surface and the shear force direction vector. For a conservative strategy, the target posture angle is such that the contact surface normal is perpendicular to the shear force direction, i.e., the posture adjustment angle. , This is the angle between the current normal and the shear force direction; for flexible adjustment strategies, the target attitude angle ensures that the angle between the contact surface normal and the shear force direction is not less than 60°, i.e. .

[0132] This process involves converting quantified control parameters, such as stiffness, velocity, and attitude angles, into control commands that the robot actuator can recognize. These commands employ a standardized protocol format and include information such as command type, target parameter value, and execution time. For example, the control command set for a conservative strategy is: {Command type: reduce stiffness, target stiffness: 0.4} Execution time: 100ms; {Instruction type: speed limit, target speed: 0.3} Execution time: instantaneous; {Instruction type: attitude adjustment, target attitude angle:} Execution time: 200ms. If a safe stop is required, a separate safe stop instruction is generated, with higher priority than other control instructions.

[0133] After this step is completed, the set of safety control instructions to be executed is determined and stored in the control instruction library of the control system. At the same time, the parameters, rule matching results and control strategy types in the decision-making process are recorded. Subsequently, the control instructions can be sent to the robot actuator to adjust the movement state of the robot hand.

[0134] Therefore, the industrial humanoid robot risk prediction and safety control method, through a series of steps including perception area division and confidence parameter setting, multimodal tactile data acquisition and preprocessing, contact event identification and area confirmation, contact state feature extraction, feature modulation based on area confidence, instability risk time-series prediction, and area-risk coupled control strategy generation, achieves early identification and differentiated safety control of robot contact instability risks in complex industrial environments. This method deeply integrates area perception confidence into the risk formation process, effectively solving problems such as missed detection of instability precursors in low-perception areas, excessively conservative erroneous stops in high-perception areas, and delayed safety response. It can amplify potential risk signals in low-perception areas and suppress excessively conservative control in high-perception areas, elevating risk judgment from a static threshold to a dynamic evolution judgment under area constraints. This significantly improves the safety, stability, and operational efficiency of industrial humanoid robots in complex contact scenarios.

[0135] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a risk prediction and safety control system for industrial humanoid robots in complex industrial environments, provided in an embodiment of this application. Figure 6 As shown, the robot is equipped with flexible electronic skin for its hand, and the system 600 includes:

[0136] The perception region division and confidence setting module 601 is used to divide the robot hand into perception regions and set the region perception confidence parameters for each perception region.

[0137] The multimodal tactile data acquisition and preprocessing module 602 is used to acquire and preprocess multimodal tactile time-series data output by the electronic skin in real time during robot operation; the multimodal tactile time-series data includes at least: normal force signal, shear force signal, micro-vibration signal and temperature signal;

[0138] The contact event recognition and area confirmation module 603 is used to recognize contact events and confirm the sensing area where contact occurs based on the multimodal tactile timing data.

[0139] The contact state feature extraction module 604 is used to extract a set of contact state temporal features from the corresponding multimodal tactile temporal data based on the sensing area where contact has been confirmed to have occurred.

[0140] The confidence modulation module 605 is used to modulate the contact state temporal feature set based on the regional perception confidence parameter of the perceived area where the contact has been confirmed to have occurred, so as to obtain the contact state temporal feature set under the regional confidence constraint.

[0141] The instability risk prediction module 606 is used to predict the instability risk of the contact event based on the contact state time series feature set under the region confidence constraint.

[0142] The safety control strategy generation module 607 is used to generate a safety control strategy based on the prediction results of the instability risk and the area perception confidence parameters of the perceived area where the contact has been confirmed.

[0143] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0144] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0145] Please see Figure 7 It shows a schematic diagram of the structure of an electronic device according to an embodiment of this application, which can be used to implement... Figure 1 The method in the illustrated embodiment. (As shown) Figure 7 As shown, the electronic device 700 may include:

[0146] The system includes at least one processor 701, at least one network interface 704, a user interface 703, a memory 705, and at least one communication bus 702. The communication bus 702 is used to enable connection and communication between the components. The user interface 703 may include buttons, and optionally include a standard wired or wireless interface. The network interface 704 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0147] The processor 701 may include one or more processing cores and connect to various parts within the device 700 via various interfaces and lines. It implements the various functions and data processing of the device 700 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 705, and by accessing data in the memory 705. Optionally, the processor 701 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 701 may also integrate one or more combinations of CPU, GPU, and modem. The CPU is mainly used to handle the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; and the modem is used for wireless communication. It is understood that the modem may not be integrated into the processor 701, but may be implemented through a separate chip.

[0148] The memory 705 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 705 includes a non-transitory computer-readable medium for storing instructions, programs, code, code sets, or instruction sets. The memory 705 may be divided into a program storage area and a data storage area, wherein the program storage area may be used to store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, audio playback functionality, image playback functionality, etc.), and instructions for implementing the foregoing method embodiments; the data storage area may be used to store data involved in the relevant method embodiments. The memory 705 may also be at least one storage device located remotely from the processor 701. Figure 7 As shown, the memory 705, which serves as a computer storage medium, may contain an operating system, a network communication module, a user interface module, and program instructions.

[0149] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by processor 701, the functions defined in the methods of this application are performed.

[0150] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0151] The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

Claims

1. A method for risk prediction and safety control of industrial humanoid robots in complex industrial environments, characterized in that, The robot's hand is equipped with flexible electronic skin, including: The robot hand is divided into perception regions, and a region perception confidence parameter is set for each perception region; During the robot's operation, the multimodal tactile time-series data output by the electronic skin is collected in real time and preprocessed; the multimodal tactile time-series data includes at least: normal force signal, shear force signal, micro-vibration signal and temperature signal; Based on the multimodal tactile timing data, contact events are identified and the sensing area where contact occurs is confirmed; Based on the confirmed sensory area where contact has occurred, extract the contact state temporal feature set from the corresponding multimodal tactile temporal data; Based on the region perception confidence parameter of the confirmed contact area, the contact state temporal feature set is modulated to obtain the contact state temporal feature set under region confidence constraint. Based on the set of temporal features of the contact state under the aforementioned regional confidence constraint, the instability risk of the contact event is predicted. Based on the predicted results of the instability risk and the area perception confidence parameters of the sensed area where contact has been confirmed, a safety control strategy is generated.

2. The method for risk prediction and safety control of industrial humanoid robots in complex industrial environments according to claim 1, characterized in that, The step of setting a region perception confidence parameter for each perception region includes: An initial area perception confidence parameter is set for the sensing area based on at least one of the following factors: spatial resolution of the electronic skin in the corresponding sensing area, statistical results of historical noise level, and structural compliance.

3. The method for risk prediction and safety control of industrial humanoid robots in complex industrial environments according to claim 1, characterized in that, The contact state temporal feature set includes at least: normal force change rate, shear force direction stability, energy change of micro-vibration signal in a predetermined frequency band, and temperature change slope.

4. The method for risk prediction and safety control of industrial humanoid robots in complex industrial environments according to claim 3, characterized in that, Modulating the contact state timing feature set specifically includes: Based on the comparison results between the area perception confidence parameter of the confirmed contact area and the preset confidence threshold, modulation weights are determined for each feature component in the contact state temporal feature set. The contact state time series feature set is obtained by multiplying each feature component in the contact state time series feature set with its corresponding modulation weight.

5. The method for risk prediction and safety control of industrial humanoid robots in complex industrial environments according to claim 4, characterized in that, The modulation weights are determined for each feature component separately, specifically as follows: When the area-aware confidence parameter is less than the preset confidence threshold, the modulation weight determined for each feature component is greater than 1; when the area-aware confidence parameter is greater than or equal to the preset confidence threshold, the modulation weight determined for each feature component is less than or equal to 1.

6. The method for risk prediction and safety control of industrial humanoid robots in complex industrial environments according to claim 1, characterized in that, Predicting the instability risk of the contact event specifically includes: The set of contact state time-series features under the region confidence constraint is arranged in time sequence and input into a pre-trained time-series prediction model; the instability evolution probability and prediction confidence output by the time-series prediction model are obtained; based on the instability evolution probability, the prediction confidence and the preset judgment threshold, the prediction result of the instability risk is obtained.

7. The method for risk prediction and safety control of industrial humanoid robots in complex industrial environments according to claim 1, characterized in that, Generate security control policies, specifically including: When the prediction result indicates that there is a risk of instability, and the area perception confidence parameter of the area where the contact is confirmed is less than the preset confidence threshold, a safety control strategy is generated that includes reducing contact stiffness, limiting the speed of movement in the shear direction, and adjusting the hand posture. When the prediction result indicates an instability risk, and the area perception confidence parameter is greater than or equal to the preset confidence threshold, a safety control strategy is generated that includes moderately reducing contact stiffness and / or fine-tuning hand posture.

8. A risk prediction and safety control system for industrial humanoid robots in complex industrial environments, characterized in that, include: The perception region division and confidence setting module is used to divide the robot hand into perception regions and set the region perception confidence parameters for each perception region. The multimodal tactile data acquisition and preprocessing module is used to acquire and preprocess the multimodal tactile time-series data output by the electronic skin in real time during robot operation. The multimodal tactile time-series data includes at least: normal force signal, shear force signal, micro-vibration signal, and temperature signal; The contact event recognition and area confirmation module is used to identify contact events and confirm the sensing area where contact occurs based on the multimodal tactile timing data. The contact state feature extraction module is used to extract a set of contact state temporal features from the corresponding multimodal tactile temporal data based on the sensory area where contact has been confirmed to have occurred. The confidence modulation module is used to modulate the contact state temporal feature set based on the regional perception confidence parameter of the confirmed contact area to obtain the contact state temporal feature set under regional confidence constraints. The instability risk prediction module is used to predict the instability risk of the contact event based on the contact state time series feature set under the region confidence constraint. The safety control strategy generation module is used to generate a safety control strategy based on the prediction results of the instability risk and the area perception confidence parameters of the perceived area where the contact has been confirmed.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, The computer program instructions can be executed by a processor to implement the method as described in any one of claims 1 to 7.

Images