Abnormal collision detection and control method of robot and related device

Abstract

This application discloses an abnormal collision detection and control method and related apparatus for a robot, applied to the robot's control module. The method includes: acquiring first multi-channel sound data through a multi-channel sound acquisition module within a first preset time period; acquiring first state data through a robot state perception module; preprocessing the first multi-channel sound data and the first state data to obtain second multi-channel sound data and second state data; extracting features from the second multi-channel sound data to obtain first feature data; performing abnormal collision detection based on the first feature data and the second state data to obtain a target collision detection result; and optimizing the robot's control parameters based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result. Using this application, abnormal collision events during robot operation can be accurately detected, and control optimization can be performed accordingly.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a method and device for abnormal collision detection and control of a robot. Background Technology

[0002] With the increasing prevalence of ultra-long-range robot operation scenarios, robots need to perform tasks in unstructured environments. Data acquisition systems simultaneously collect operational data for model training to improve operational accuracy. However, ultra-long-range environments are complex, and robot components are prone to accidental collisions with objects, resulting in abnormal collision events.

[0003] Currently, traditional data acquisition systems typically focus only on the state records related to the robot's main task, lacking the ability to perceive abnormal events during the robot's interaction with the environment. When the robot accidentally collides with objects in the environment, the system cannot effectively identify such abnormal collisions, resulting in the collected data being mixed with information about unexpected external forces. This makes it difficult to accurately reflect the true state of the robot under normal interaction, significantly reducing data usability and quality. Robots trained on such low-quality data often fail to achieve the expected results in practical applications.

[0004] Therefore, how to accurately detect abnormal collision events during robot operation and optimize control accordingly has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a method and related apparatus for abnormal collision detection and control of a robot, which can accurately detect abnormal collision events during the operation of the robot and optimize control accordingly.

[0006] In a first aspect, embodiments of this application provide an abnormal collision detection and control method for a robot, applied to the control module of a robot, wherein the robot further includes: a multi-channel sound acquisition module and a robot state perception module, and the method includes: Within a first preset time period, the multi-channel sound acquisition module acquires sound data within a preset distance range of the robot to obtain first multi-channel sound data; The robot state perception module collects the robot's state data within the first preset time period to obtain first state data. The first multi-channel audio data and the first state data are preprocessed to obtain the second multi-channel audio data and the second state data. Feature extraction is performed on the second multi-channel audio data to obtain the first feature data; Anomaly collision detection is performed based on the first feature data and the second state data to obtain target collision detection results; the target collision detection results include any of the following: abnormal collision, non-abnormal collision. The control parameters of the robot are optimized based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection results.

[0007] Secondly, embodiments of this application provide an abnormal collision detection and control device for a robot, applied to the control module of a robot. The robot further includes: a multi-channel sound acquisition module and a robot state perception module. The device includes: an acquisition unit, a processing unit, a collision detection unit, and an optimization unit, wherein: The acquisition unit is used to acquire sound data of the robot within a preset distance range through the multi-channel sound acquisition module within a first preset time period to obtain first multi-channel sound data; and to acquire the robot's state data within the first preset time period through the robot state perception module to obtain first state data. The processing unit is configured to preprocess the first multi-channel audio data and the first state data to obtain second multi-channel audio data and second state data; and to extract features from the second multi-channel audio data to obtain first feature data. The collision detection unit is used to perform abnormal collision detection based on the first feature data and the second state data to obtain a target collision detection result; the target collision detection result includes any one of the following: abnormal collision, non-abnormal collision. The optimization unit is used to optimize the control parameters of the robot based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result.

[0008] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the steps in the first aspect of embodiments of this application.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of embodiments of this application.

[0010] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0011] Implementing this application will have the following beneficial effects: As can be seen, the abnormal collision detection and control method and related apparatus for robots described in this application firstly captures collision information from both acoustic and motion dimensions by synchronously acquiring multi-channel sound and robot state data. Combining acoustic features with cross-validation of state data, abnormal collisions can be accurately identified, effectively reducing false positives and false negatives. Secondly, after detecting an abnormal collision, control parameters are optimized based on collision-related data to avoid secondary collisions and ensure safe task execution. Using this method, abnormal collision events during robot operation can be accurately detected, and control optimization can be performed accordingly. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0013] Figure 1 This is a block diagram of the functional modules of a robot provided in an embodiment of this application; Figure 2 This is an application scenario diagram of an abnormal collision detection and control method for a robot provided in an embodiment of this application; Figure 3 This is a flowchart of an abnormal collision detection and control method for a robot provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a robot state perception module provided in an embodiment of this application; Figure 5 This is a flowchart of a method for optimizing control parameters provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a control module provided in an embodiment of this application; Figure 7 This is a functional unit block diagram of an abnormal collision detection and control device for a robot provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0015] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0016] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0017] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0018] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] The electronic device described in the embodiments of this application may include a robot, or a robot control module.

[0021] The control module may include smartphones (such as Android phones, iOS phones, Windows Phones, etc.), tablets, PDAs, laptops, video matrices, monitoring platforms, mobile internet devices (MIDs) or wearable devices, etc. The above are just examples and not an exhaustive list, including but not limited to the above devices.

[0022] The following describes the relevant content, concepts, meanings, technical issues, technical solutions, and beneficial effects involved in the embodiments of this application.

[0023] First, let me explain some of the technical terms or phrases used in this application: Abnormal collision: refers to an unexpected or abnormal contact or collision event between the robot and external objects, the environment, or its own structure during the execution of a task, which may result in equipment damage, task interruption, or safety risks.

[0024] Short-time energy: refers to the energy value corresponding to each frame of a sound signal after it has been processed into frames according to a fixed time window. It is used to reflect the intensity change of the sound signal in a short period of time and is an important feature for judging sudden acoustic events such as collisions.

[0025] Kurtosis: Used to describe the steepness of the probability distribution of a signal. In collision detection, it is used to characterize the strength of the signal's impact characteristics. The greater the kurtosis, the more obvious the impact of the signal, and the more effectively it can reflect the transient changes caused by the collision.

[0026] Spectral centroid: refers to the central position of energy distribution in the signal spectrum, reflecting the frequency concentration trend of the sound signal. It can be used to distinguish the spectral differences between collision sounds and normal noise and environmental sounds, and improve the accuracy of abnormal event identification.

[0027] PTP: Precision Time Protocol, also known as the IEEE 1588 protocol, is a network protocol used to achieve sub-microsecond clock synchronization in distributed systems, ensuring a high degree of consistency in time bases among multiple modules.

[0028] GPS: Global Positioning System, provides high-precision UTC timestamps and is often combined with PTP for cross-device time synchronization.

[0029] EtherCAT: Ethernet for Control Automation Technology, is a real-time industrial Ethernet protocol characterized by low communication latency, flexible topology, and high synchronization accuracy. It is widely used in robotics, motion control, and other fields.

[0030] PCIe 4.0: PCI Express 4.0 is a high-speed serial computer expansion bus standard with a data transfer rate of up to 16GT / s, enabling high-speed data exchange between chips.

[0031] TOPS: Tera Operations Per Second, is a commonly used unit for measuring the computing power of AI chips, representing the number of trillions of computational operations that can be completed per second.

[0032] Please see Figure 1 , Figure 1 This is a functional module block diagram of a robot provided in an embodiment of this application. It can be seen that the robot may include: a multi-channel sound acquisition module, a robot state perception module, and a control module, wherein: A multi-channel sound acquisition module is used to synchronously acquire multi-channel sound signals generated by the robot's working environment and its own structure, and obtain acoustic data including events such as collisions, friction, and impacts, providing raw data support for the extraction and identification of acoustic features of abnormal collisions. In specific implementation, the multi-channel sound acquisition module may include an 8-channel MEMS microphone array and a synchronous sampling chip with a sampling rate ≥48kHz, a bit depth of 16bit, and a frequency response of 20Hz~20kHz. The microphones are respectively installed on the robot's manipulator joints (e.g., 3), the middle of the body (e.g., 2), and the end effector (e.g., 3), covering all possible collision areas.

[0033] The robot state perception module is used to acquire the robot's motion state information in real time, including but not limited to joint angles, joint torques, motion speed, acceleration, and position information, reflecting whether the robot has experienced abnormal collisions or external impacts from a motion perspective. In specific implementations, the robot state perception module may include: joint encoders, IMU sensors, and force sensors; the joint encoders have a sampling rate of 1kHz, the IMU has a sampling rate of 1kHz, and the force sensor has a range of -50N to +50N; it is installed in spatial alignment with the microphone array to ensure that the state data and sound data are synchronized in time and space.

[0034] The control module is connected to the multi-channel sound acquisition module and the robot state perception module respectively. It is used to receive and process sound data and robot state data, realize the detection and judgment of abnormal collisions based on the two-dimensional information, and output corresponding control commands after detecting abnormal collisions to adjust or protect the robot's motion state, avoid secondary collisions and ensure operational safety.

[0035] In some embodiments, in addition to a multi-channel sound acquisition module, a robot state perception module, and a control module, the robot may also include: a power supply and synchronization module; the control module may include a real-time data processing module and a communication module; wherein: The power supply and synchronization module may include a high-precision synchronous clock (GPS / PTP) and an isolated power supply module; the clock synchronization error is ≤10μs and the power supply ripple is ≤50mV; the power supply and synchronization module is used to provide a unified power supply and time reference for all modules; it can ensure that multi-channel sound data and robot status data are strictly aligned in the time dimension, eliminating the impact of synchronization deviation on collision detection accuracy.

[0036] The communication module may include an EtherCAT bus controller with a communication latency of ≤1ms, supporting real-time control command issuance. The communication module is used to receive collision detection results output by the real-time data processing module, transmit optimized control commands, realize the instant adjustment of the robot's motion strategy, and avoid secondary collisions.

[0037] The real-time data processing module may include an FPGA and a GPU. The FPGA has a logic latency of ≤50μs and a GPU computing power of ≥200TOPS. It is connected at high speed to the multi-channel sound acquisition module and the robot state perception module via PCIe 4.0. The FPGA is responsible for sound signal preprocessing (e.g., framing, filtering, and noise reduction) and basic feature extraction to achieve low-latency computing. The GPU accelerates collision detection and quantization inference to accurately identify abnormal collision events and obtain collision detection results.

[0038] Please see Figure 2 , Figure 2 This is an application scenario diagram of an abnormal collision detection and control method for a robot provided in an embodiment of this application. The application scenario includes: a task area, a robot, a refrigerator, and multiple objects (object A, object B, object C, and object D); wherein: Task area: The defined spatial range within which the robot performs its tasks; Robot: The main entity responsible for performing the task of retrieving items from the refrigerator within the designated area; Refrigerator: This is the target location for the robot's current task. Objects A, B, C, and D: These are obstacles within the task area, located around the path the robot takes to the refrigerator. They may interfere with the robot's normal movement or pose a collision risk.

[0039] Specifically, the robot can move towards the refrigerator along a planned path to perform the task of retrieving items. During the movement, the robot's control module can execute the abnormal collision detection and control method for the robot provided in this application embodiment, identify unexpected collisions between the robot and obstacles or the refrigerator in real time, and quickly optimize control parameters to avoid secondary collisions and ensure that the robot safely arrives at the refrigerator to complete the task of retrieving items.

[0040] Please see Figure 3 , Figure 3 This is a flowchart of an abnormal collision detection and control method for a robot provided in an embodiment of this application. It is applied to the control module of a robot, which further includes: a multi-channel sound acquisition module and a robot state perception module. The method includes, but is not limited to, the following steps: S301. Within a first preset time period, the robot acquires sound data within a preset distance range through the multi-channel sound acquisition module to obtain first multi-channel sound data.

[0041] In this embodiment, the first preset time period and the preset distance range can be preset in advance or defaulted.

[0042] In a specific embodiment, the sampling rate (e.g., 48kHz), sampling precision (e.g., 16bit), and number of channels (e.g., 4-8 channels, distributed in different positions on the robot body) of the multi-channel sound acquisition module can be configured first. The control module can send acquisition commands to the multi-channel sound acquisition module, and the multi-channel sound acquisition module can synchronously start the sound acquisition of all channels within a first preset time period to ensure that the timestamps of the data of each channel are aligned. Distance threshold filtering is performed on the acquired raw sound data: based on the time delay difference and amplitude difference of the multi-channel sound, the relative distance from the sound source to the robot is calculated, and only the sound signal within the preset distance range is retained, while environmental noise outside the range (e.g., the sound of distant equipment running, the sound of people talking) is filtered out. Then, the filtered sound data of each channel can be spliced ​​together in time sequence to form the first multi-channel sound data, and the timestamp, channel number, and sampling parameters are added to the data. The data is then output to the control module for acoustic feature analysis of collision events.

[0043] S302. The robot state perception module collects the state data of the robot within the first preset time period to obtain the first state data.

[0044] In the embodiments of this application, please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of a robot state perception module provided in an embodiment of this application. As can be seen, the robot state perception module may include: a joint encoder, an IMU sensor, and a force sensor.

[0045] In a specific embodiment, within a first preset time period, the control module synchronously triggers the robot state perception module to collect data. Specifically, the joint encoder collects the angle, angular velocity, and angular acceleration information of each joint of the robot in real time, and outputs digital signals according to a preset sampling frequency (e.g., 1kHz); the IMU sensor collects the linear acceleration, angular velocity, and attitude information of the robot body or end effector in real time to reflect the changes in the overall motion of the robot and the sudden changes in attitude caused by external impacts; the force sensor collects the contact force and torque information when the robot comes into contact with the external environment in real time to determine whether a collision or abnormal force has occurred; this information is transmitted to the control module, which processes this information and combines the joint angle, angular velocity, acceleration, attitude, contact force, and other information at the same moment into a single frame of state data. After the first preset time period ends, the continuous multiple frames of state data are integrated into the first state data for subsequent abnormal collision detection.

[0046] S303. Preprocess the first multi-channel audio data and the first state data to obtain the second multi-channel audio data and the second state data.

[0047] In some embodiments, the preprocessing of the first multi-channel audio data and the first state data to obtain second multi-channel audio data and second state data includes: S11. The first multi-channel audio data and the first state data are processed according to a preset interpolation algorithm to obtain third multi-channel audio data and third state data; the sampling rate of the third multi-channel audio data is equal to the sampling rate of the third state data; S12. Determine the target operating environment corresponding to the robot; S13. Determine the reference noise type corresponding to the target operating environment; S14. Determine the reference denoising algorithm corresponding to the reference noise type; S15. Process the third multi-channel audio data according to the reference denoising algorithm to obtain the fourth multi-channel audio data; S16. Enhance the fourth multi-channel audio data to obtain the second multi-channel audio data; S17. Filter the third state data to obtain the second state data.

[0048] In this application embodiment, the preset interpolation algorithm may include any of the following: linear interpolation algorithm, nearest neighbor interpolation algorithm, polynomial interpolation algorithm, spline interpolation algorithm, etc., and is not limited here.

[0049] In a specific embodiment, the first multi-channel audio data and the first state data can be processed according to a preset interpolation algorithm to obtain the third multi-channel audio data and the third state data. Specifically, the first sampling rate corresponding to the first multi-channel audio data and the second sampling rate corresponding to the first state data can be obtained first. The higher of the first sampling rate and the second sampling rate is taken as the target sampling rate. The preset interpolation algorithm is used to interpolate the data with the lower sampling rate (usually the first state data) to obtain the third multi-channel audio data and the third state data with the target sampling rate. Alternatively, the preset interpolation algorithm can be used to interpolate both types of data to the preset sampling rate so that the final third multi-channel audio data and the third state data have the same sampling rate.

[0050] In some embodiments, the multi-channel sound acquisition module can be an 8-channel MEMS microphone array, and the original multi-channel sound signal (i.e., the first multi-channel sound data) can be denoted as: ; in, , Microphone channel index (8 acquisition channels in total); For timestamps, the unit is seconds (s); Indicates the first One microphone The sound signal collected at all times is measured in Pascals (Pa).

[0051] The first state data may include: joint angles Joint angular velocity IMU acceleration ; in, For the robot's joints in The angle value at any given time, in radians (rad). The joint angular velocity is expressed in radians per second (rad / s). The three-dimensional acceleration of the robot body is collected by the IMU, and the unit is meters per second squared (m / s²).

[0052] Robot body parameters: DH parameters Microphone array coordinates ; Among them, DH parameters are the standard geometric parameters of the robot links. The length of the connecting rod (m) The link twist angle (rad) Linkage offset (m) Joint angle (rad); For the first The three-dimensional coordinates of each microphone in the robot's base coordinate system are expressed in meters (m).

[0053] The preset interpolation algorithm can be a linear interpolation algorithm. Linear interpolation is used to unify the data sampling rate to 48kHz, and time deviation is corrected based on the PTP clock stamp to obtain synchronized data. ; in, : indicates the first The time deviation between the channel's microphone and the robot's reference clock, in seconds (s), can be obtained through GPS / PTP synchronization calibration. Indicates the first The raw sound signal collected by the microphone of the channel at time t; : indicates the first The audio signal after channel time calibration; : Represents a linear interpolation function; : Indicates the raw sampling rate of the joint encoder (1000 times / second); : Unified preset sampling rate (48,000 times / second); : Original joint angle data before interpolation; : Joint angle data that is interpolated and aligned with the audio data sampling rate.

[0054] In this way, third-channel audio data and third-state data can be obtained.

[0055] Next, the target operating environment for the robot can be determined. Specifically, various typical operating environments can be pre-configured and stored in the control module, including but not limited to: home indoor environment, laboratory environment, industrial operation environment, warehouse handling environment, etc., without limitation. The current scene information of the robot is obtained through user configuration, scene selection instructions, or the robot's environmental sensors. The target operating environment corresponding to the scene information is selected from the above-mentioned typical operating environments. For example, if the scene information is a home indoor environment with fixed facilities such as refrigerators, air conditioners, and furniture, then the target operating environment is determined to be a home indoor environment. Or, if the scene information is an industrial production line with mechanical equipment, conveyor belts, and metal workbenches, then the target operating environment is determined to be an industrial operation environment.

[0056] Furthermore, the reference noise type corresponding to the target operating environment can be determined. Specifically, a pre-stored mapping relationship between the operating environment and noise type can be used to determine the reference noise type corresponding to the target operating environment. Then, the reference denoising algorithm corresponding to the reference noise type can be determined. Similarly, a pre-stored mapping relationship between the noise type and denoising algorithm can be used to determine the reference denoising algorithm corresponding to the reference noise type. Next, the third multi-channel sound data can be processed according to the reference denoising algorithm to obtain the fourth multi-channel sound data. For example, assuming the reference noise type is steady-state background noise, its corresponding reference denoising algorithm is spectral subtraction. Spectral subtraction is used to estimate and filter the noise spectrum of the third multi-channel sound data to suppress environmental steady-state noise interference, thereby obtaining the fourth multi-channel sound data.

[0057] In some embodiments, the reference denoising algorithm can be an adaptive Wiener filtering algorithm, which is used to estimate the noise power spectral density of the third multi-channel audio data. With signal power spectral density The filter transfer function is: ; Output noise reduction signal: ; Angular frequency, measured in radians per second (rad / s). ( (frequency, unit: Hz). The power spectral density of noise, expressed in Pa² / Hz; Power spectral density of the original sound signal (i.e., the third multi-channel sound data), in Pa² / Hz; Wiener filter frequency domain transfer function (dimensionless); The Fast Fourier Transform (FFT) function converts a time-domain signal into a frequency-domain signal. The inverse fast Fourier transform function converts a frequency domain signal back to the time domain. : No. The audio signal after noise reduction of the channel, measured in Pa.

[0058] In this way, the denoised sound data (i.e., the fourth multi-channel sound data) can be obtained.

[0059] Finally, the fourth multi-channel audio data can be enhanced to obtain the second multi-channel audio data. For example, a delay-and-sum (DAS) beamforming algorithm can be used to enhance the signal, highlighting the acoustic energy in the collision direction and suppressing noise interference in non-target directions. The specific implementation is as follows: Based on the robot's current motion state, environmental structure information, and historical collision characteristics, the control module estimates the approximate direction of the collision in advance, represented as a unit direction vector: Next, calculate the delay of each channel: ; Enhanced fused signal: ; in, : Represents the prior estimate of the collision direction. For azimuth, The pitch angle; : indicates the first The sound propagation delay of the microphone in a channel relative to a reference point is measured in seconds (s). The reference point is the "baseline position" used to calculate the time delay. It is usually the geometric center of the eight microphones, or the position of a specific microphone (e.g., the first microphone). Microphone coordinates With direction vector The dot product of , in meters (m); The speed of sound in air is 343 m / s (at standard atmospheric pressure at 20°C). The fused audio signal (i.e., the second multi-channel audio data) after beamforming at time t is expressed in Pa.

[0060] Finally, a preset filtering method can be used to filter the third-state data to obtain the second-state data. The preset filtering method can include at least one of the following: low-pass filtering, moving average filtering, Kalman filtering, etc., which are not limited here.

[0061] Thus, by interpolating to unify the sampling rate to achieve temporal alignment of multimodal data, and by combining the operating environment with adaptive matching of noise type and denoising algorithm, the sound data is denoised and enhanced, and the state data is filtered and smoothed. This can effectively improve the synchronization and signal-to-noise ratio of multimodal data and enhance the accuracy of abnormal collision detection in complex environments.

[0062] S304. Perform feature extraction on the second multi-channel audio data to obtain the first feature data.

[0063] In this embodiment of the application, acoustic features of the second multi-channel sound data can be extracted, as follows: Short-time energy is calculated using a sliding window method, and the calculation formula is as follows: ; in, express The short-time energy value at time t, in Pa²; L represents the window length (number of sampling points) for short-time energy calculation, with a value of 256, corresponding to a time window of t. ( ); Indicates the index of the sampling points within the window (dimensionless); express The fused sound signal after beamforming at a given moment; according to this formula, multiple short-time energy values ​​can be calculated, which is also the first short-time energy data.

[0064] Next, kurtosis can be calculated using the following formula: ; in, The kurtosis value (dimensionless) represents the steepness of the signal distribution; Operator representing mathematical expectation (mean); This represents the mean value of the fused audio signal after beamforming, expressed in Pa. The standard deviation of the fused audio signal is expressed in Pa; the first kurtosis data can be calculated using this formula.

[0065] Then, the spectral centroid can be calculated (assuming an STFT window size of 512 and an overlap rate of 50%), using the following formula: ; in, express The centroid of the spectrum at time t, in rad / s; in, , representing the short-time Fourier transform result of the fused audio signal (complex number representation in the frequency domain); This represents the energy distribution (or short-time energy spectrum) of the frequency domain signal; the first spectral centroid data can be calculated based on this formula.

[0066] In this way, we can obtain the first short-time energy data, the first kurtosis data, and the first spectral centroid data, which together form the first feature data.

[0067] Thus, by extracting short-time energy, kurtosis, and spectral centroid to form multi-dimensional first feature data, the acoustic characteristics of the collision can be comprehensively characterized from the perspectives of signal strength, impact characteristics, and frequency distribution, effectively improving the sensitivity, accuracy, and environmental adaptability of abnormal collision detection.

[0068] S305. Perform abnormal collision detection based on the first feature data and the second state data to obtain target collision detection results; the target collision detection results include any one of the following: abnormal collision, non-abnormal collision.

[0069] In this embodiment, the robot state is analyzed based on the first feature data and the second state data to determine whether there is an abnormal collision, thereby obtaining the target collision detection result.

[0070] In some embodiments, the step of performing abnormal collision detection based on the first feature data and the second state data to obtain a target collision detection result includes: S21. Obtain the reference parameter set of the robot in a collision-free state; S22. Determine the collision parameter threshold set based on the reference parameter set and the preset calibration coefficient set; S23. Determine the reference collision detection result based on the collision parameter threshold set and the first feature data; S24. Verify the reference collision detection result based on the second state data to obtain the target verification result; the target verification result includes any of the following: verification successful, verification failed; S25. When the target verification result includes the verification success, the target collision detection result is determined based on the reference collision detection result; S26. When the target verification result includes the verification failure, new sound data and new status data within a second preset time period are obtained, and abnormal collision detection is performed based on the new sound data and the new status data to obtain the target collision detection result; the start time of the second preset time period is later than the end time of the first preset time period.

[0071] In this embodiment, the preset calibration coefficient set and the second preset time period can be preset in advance or defaulted.

[0072] In a specific embodiment, a set of baseline parameters for the robot in a collision-free state can be obtained first. Specifically, within the normal operation and collision-free state interval of the robot, short-time energy, kurtosis, and spectral centroid can be statistically analyzed and adaptively updated online to obtain and maintain a set of baseline parameters in a collision-free state. The baseline parameter set can include: short-time energy baseline value, kurtosis baseline value, and spectral centroid baseline value. Then, a collision parameter threshold set can be determined based on the baseline parameter set and a preset calibration coefficient set, which includes energy coefficients, kurtosis coefficients, and spectral coefficients. The details are as follows: ; ; ; in, Indicates the short-time energy threshold; Indicates the kurtosis threshold; Indicates the spectral centroid threshold; Indicates the energy coefficient; Indicates the kurtosis coefficient; Represents the spectral coefficients; Indicates the short-time energy reference value; Indicates the kurtosis baseline value; This represents the reference value for the spectral centroid; the collision parameter threshold set includes: short-time energy threshold, kurtosis threshold, and spectral centroid threshold.

[0073] In some embodiments, The value range is 1.2 to 2.0, for example, it can be 1.5; The value range is 1.1 to 1.8, for example, it can be 1.3; The value range is 1.1 to 1.5, for example, it can be 1.2.

[0074] Next, the first feature data can be analyzed based on the collision parameter threshold set to determine the reference collision detection result. Then, the reference collision detection result can be verified based on the second state data to obtain the target verification result. Specifically, when the reference collision detection result is an abnormal collision, it is determined whether there is a sudden change in the joint acceleration, joint angular velocity, or contact force at the corresponding moment in the second state data that exceeds the normal range. If there is a sudden change, the target verification result is a successful verification; if there is no sudden change, the target verification result is a failed verification. Alternatively, when the reference collision detection result is a non-abnormal collision, it is determined whether the joint acceleration, joint angular velocity, and contact force at the corresponding moment in the second state data are all within the normal range. If all are normal, the target verification result is a successful verification; if there is an abnormal sudden change, the target verification result is a failed verification.

[0075] If the target verification result includes a successful verification, the reference collision detection result is directly determined as the target collision detection result.

[0076] When the target verification result includes verification failure, the multi-channel sound acquisition module and the robot state perception module can be controlled to collect new sound data and new state data within a second preset time period. Based on the new sound data and new state data, abnormal collision detection is re-performed to obtain the target collision detection result.

[0077] Thus, by adaptively determining the collision parameter threshold set using the baseline parameter set under collision-free conditions, it is possible to adapt to noise changes in different environments and improve the stability of detection. By combining the first feature data for preliminary collision judgment and using the second state data for verification, multimodal cross-validation of acoustic features and robot motion state can be achieved, which can effectively eliminate environmental interference and significantly reduce the false and false judgment rates. When the verification fails, new data from subsequent time periods is automatically used for re-detection to ensure reliable detection results and a closed-loop process.

[0078] In some embodiments, the first feature data includes: first short-time energy data, first kurtosis data, and first spectral centroid data; the collision parameter threshold set includes: short-time energy threshold, kurtosis threshold, and spectral centroid threshold; determining the reference collision detection result based on the collision parameter threshold set and the first feature data includes: S31. Determine the maximum short-time energy value corresponding to the short-time energy data; S32. Determine the maximum kurtosis value corresponding to the first kurtosis data; S33. Determine the maximum spectral centroid value corresponding to the first spectral centroid data; S34. When the maximum short-time energy value is greater than the short-time energy threshold, the maximum kurtosis value is greater than the kurtosis threshold, and the maximum spectral centroid value and the spectral centroid threshold meet preset conditions, the reference collision detection result is determined to include the abnormal collision. S35. When the maximum short-time energy value is not greater than the short-time energy threshold, or the maximum kurtosis value is not greater than the kurtosis threshold, or the maximum spectral centroid value and the spectral centroid threshold do not meet the preset conditions, the reference collision detection result is determined to include the non-abnormal collision.

[0079] In this embodiment of the application, the preset conditions can be preset in advance or defaulted. Specifically, the preset conditions can be: the target absolute value is greater than the spectrum centroid threshold, wherein the target absolute value is the absolute value of the difference between the maximum spectrum centroid value and the spectrum centroid reference value.

[0080] In a specific embodiment, the short-time energy data can be traversed to find the maximum value, which is the maximum short-time energy value; similarly, the maximum kurtosis value corresponding to the first kurtosis data and the maximum spectral centroid value corresponding to the first spectral centroid data can be determined.

[0081] When the maximum short-time energy value is greater than the short-time energy threshold, the maximum kurtosis value is greater than the kurtosis threshold, and the maximum spectral centroid value and the spectral centroid threshold meet the preset conditions, that is, when all three conditions are met simultaneously, it indicates that the current sound signal has the typical collision signal characteristics of high energy, non-stationarity, and prominent high-frequency components, indicating that the robot is very likely to have an abnormal collision. At this time, the reference collision detection result can be directly determined as an abnormal collision. If the maximum short-time energy value is not greater than the short-time energy threshold, or the maximum kurtosis value is not greater than the kurtosis threshold, or the maximum spectral centroid value and the spectral centroid threshold do not meet the preset conditions, it indicates that the current sound signal does not simultaneously possess the high energy, non-stationary, and high-frequency prominent characteristics that a collision event should have, and the signal is more in line with environmental noise or normal operating conditions. In this case, the reference collision detection result can be directly determined as a non-abnormal collision.

[0082] Thus, by extracting the maximum values ​​of three features—short-time energy, kurtosis, and spectral centroid—and employing a method where all three features must be simultaneously satisfied to classify a collision as abnormal, and any non-satisfied feature is considered a normal collision, this approach fully leverages the typical characteristics of collision signals in terms of energy, abrupt changes, and spectral distribution. This effectively distinguishes real collisions from environmental noise and normal operational sounds. This method improves the accuracy and reliability of collision detection, avoids misjudgments and missed detections caused by the susceptibility of single features to interference, and features clear logic and low computational cost, meeting the real-time detection needs of robots.

[0083] S306. Optimize the control parameters of the robot based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result.

[0084] In some embodiments, the second multi-channel audio data includes *a* channels of audio data; *a* is an integer greater than 1; see [link to documentation]. Figure 5 , Figure 5 This is a flowchart of a control parameter optimization method provided in an embodiment of this application. The optimization of the robot's control parameters based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result includes: S41. Determine the time when a collision signal is detected in each of the a channels of audio data, thus obtaining a time points; S42. Determine the time difference between every two times in the a time intervals to obtain b time differences; b is a positive integer. S43. Determine the target positioning equation based on the b time differences; S44. Determine the collision parameter set based on the first feature data, the target localization equation, the second multi-channel sound data, and the target collision detection result; S45. Optimize the control parameters of the robot based on the collision parameter set and the second state data.

[0085] In this embodiment, the time when a collision signal is detected in each of the a channels of audio data is first determined, resulting in a time points. Specifically, for each channel of audio data, abnormal collision detection is performed. When an abnormal collision is detected, the time corresponding to the abnormal collision is recorded, i.e., the time when the collision signal is detected. Thus, a time points are obtained. Then, the a time points are combined pairwise to calculate the time difference between each pair of different time points, resulting in b time differences, as follows: ; in, Indicates the first The arrival time of the passage and the first The time difference between the arrival times of channel collision signals; Indicates the first The moment when the channel detects a collision signal; Indicates the first The moment when the channel detects a collision signal; and All are integers greater than or equal to 1 and less than or equal to a; according to the above formula, b time differences can be obtained.

[0086] Then, the target localization equation can be determined based on b time differences, where the target localization equation is as follows: ; in, This indicates the position coordinates of the collision point in the base coordinate system (i.e., the target collision position coordinates); Indicates the first Similarly, the Euclidean distance from the microphone in the channel to the collision point P is... Indicates the first The Euclidean distance from the microphone in the channel to the collision point P; It indicates the speed of sound.

[0087] Then, the first feature data, the target localization equation, the second multi-channel sound data, and the target collision detection results can be analyzed to obtain a collision parameter set; finally, the robot's control parameters can be optimized based on the collision parameter set and the second state data.

[0088] In this way, by detecting collision moments through multi-channel sound data, constructing a localization equation based on the difference between different collision moments, and combining acoustic features with robot state data to determine collision parameters and optimize control parameters, the anti-interference capability and localization accuracy of collision detection can be effectively improved, achieving closed-loop fusion of perception and control, and avoiding misjudgments and missed judgments.

[0089] In some embodiments, the collision parameter set includes: a collision event marker, the time of occurrence of the collision event, the target collision location coordinates, the target collision direction, a first feature vector, and the magnitude of the collision force; determining the collision parameter set based on the first feature data, the target localization equation, the second multi-channel sound data, and the target collision detection result includes: S51. Determine the collision event marker and the time of occurrence of the collision event based on the target collision detection results; S52. Solve the target positioning equation to obtain the target collision position coordinates; S53. Determine the joint center coordinates corresponding to the target collision position coordinates in the robot; S54. Determine the target collision direction based on the joint center position coordinates and the target collision position coordinates; S55. Determine the first feature vector based on the first feature data; S56. Determine the magnitude of the collision force based on the time of occurrence of the collision event and the second multi-channel sound data.

[0090] In this embodiment of the application, a collision event marker and the occurrence time of the collision event can be determined based on the target collision detection result. Specifically, the collision event marker can be a Boolean marker. If the target collision detection result is an abnormal collision, the collision event marker is set to 1. At the same time, the earliest time among a times can be determined and the earliest time is taken as the occurrence time of the collision event. Conversely, if the target collision detection result is a non-abnormal collision, the collision event marker is set to 0. At the same time, it is determined that the occurrence time of the collision event does not exist and the collision event has not occurred.

[0091] Next, the least squares method can be used to find the optimal solution for the target localization equation, as follows: ; in, Indicates the coordinates of the target collision location; The constraint space (robot body and operating range) represents the collision location; In constrained space The inner solution finds the P value that minimizes the objective function; thus, the target collision position coordinates can be obtained.

[0092] Furthermore, the target collision location coordinates are determined to correspond to the joint center coordinates within the robot. Specifically, the position coordinates of all joint centers in the robot in the base coordinate system can be obtained, resulting in multiple position coordinates. Then, based on the target collision location coordinates and these multiple position coordinates, the Euclidean distances between the collision location and each joint center are calculated, resulting in multiple Euclidean distances. The minimum distance among these multiple Euclidean distances is determined, and the position coordinates corresponding to this minimum distance among these multiple position coordinates are determined as the joint center position coordinates. Finally, the target collision direction can be determined based on the joint center position coordinates and the target collision location coordinates, as follows: ; in, A unit vector representing the direction of the target collision; The coordinates of the joint center position are represented; according to the above formula, the unit vector of the target collision direction can be calculated, and then the target collision direction can be determined; then, the first feature vector can be determined according to the first feature data. Specifically, the first feature data package contains multiple acoustic features, and these multiple acoustic features are arranged in a preset order to construct a multi-dimensional feature vector, that is, the first feature vector.

[0093] In some embodiments, the preset order can be "short-time energy, kurtosis, spectral centroid", then the first feature vector ;in, The peak energy of a sound signal is the maximum value of its time-domain energy, used to characterize the intensity of a collision or impact. The kurtosis of a sound signal is used to characterize the impact characteristics of the signal. The kurtosis of a collision signal is significantly higher than that of normal environmental noise. The centroid of the sound spectrum represents the frequency distribution characteristics of the signal. The centroid of the sound spectrum of a collision sound signal is significantly shifted.

[0094] Next, the magnitude of the collision force can be determined based on the time of the collision event and the second multi-channel sound data. Specifically, based on a differentiable physics model, the collision sound energy and impulse satisfy the following: ; in, This represents the preset energy-impulse calibration coefficient (dimensionless, which can be obtained through offline calibration of a force sensor). , representing the impact impulse; , representing the linear velocity at the point of collision. The Jacobian matrix of the robot (dimensionless) maps joint velocities to end effector linear velocities. express The angular velocity vectors of each joint of the robot at each instant; solving the above formula yields: ; ; in, This indicates the magnitude of the collision force, measured in Newtons (N). The duration of the collision is expressed in seconds (s). , (This represents the full width at half maximum (FWHM) of the collision spectrum, in rad / s). The magnitude of the collision force can be calculated using the formula above.

[0095] In this way, by determining the event and time through collision detection, solving the localization equation to obtain the collision position, matching the joint center and quantifying the collision direction, constructing the feature vector and inverting the collision force, the robot can achieve full parameter quantitative perception of collision, improve detection accuracy and anti-interference ability, provide a comprehensive basis for control optimization, and enhance the safety and intelligence level of robot operation.

[0096] In some embodiments, optimizing the robot's control parameters based on the collision parameter set and the second state data includes: S61. The collision parameter set is scored according to a preset data quality scoring function to obtain a target score; S62. When the target score is greater than the preset score, a training dataset is determined based on the second multi-channel sound data, the collision parameter set, and the second state data; the control parameters of the robot are optimized based on the training dataset and the preset update algorithm to obtain new control parameters.

[0097] In this embodiment, the preset data quality scoring function, preset scoring, and preset update algorithm can all be preset in advance or defaulted.

[0098] In a specific embodiment, the collision parameter set can be scored according to a preset data quality scoring function, which is as follows: ; in, This represents the data quality score (dimensionless, with a value range of 0 to 1). This represents the preset collision force weighting coefficient; Indicates the magnitude of the collision force; This represents the natural exponential function; the magnitude of the collision force in the collision parameter set is substituted into the preset data quality scoring function for calculation to obtain the target score.

[0099] In some embodiments, It can be equal to 0.02.

[0100] When the target score is greater than the preset score, the second multi-channel sound data, collision parameter set and second state data are packaged into a training dataset. The robot is trained using the training dataset. During the training process, the robot's control parameters are optimized and updated using a preset update algorithm to obtain new control parameters. The new control parameters are then written into the robot's control module to complete the online update of the control parameters.

[0101] The preset update algorithm may include any of the following: gradient descent algorithm, adaptive control algorithm, reinforcement learning algorithm, etc., without limitation.

[0102] When the target score is less than or equal to the preset score, the second multi-channel audio data is marked as low-quality data and removed from the time window. The data within is used to obtain the fifth multi-channel audio data; among which, Indicates the moment of the collision event; stores the fifth multi-channel sound data into the preset robot data resource pool.

[0103] In some embodiments, the preset score can be equal to 0.6.

[0104] In some embodiments, the preset update algorithm can be a gradient descent algorithm; model predictive control (MPC) is used to optimize the robot's operation strategy after a collision; the optimization of the robot's control parameters based on the training dataset and the preset update algorithm to obtain new control parameters is as follows: First, define the optimization objective function (trajectory tracking + torque smoothing): ; in, : Indicates the optimization objective function value (dimensionless); : Indicates the prediction time domain (number of steps) of MPC; : Indicates the prediction step index (dimensionless). :express The predicted position of the robot's end effector at any given time, in meters (m). : Represents the regularization coefficient (dimensionless) of the torque smoothing term; :express The joint torque command at any given time, in Newton-meter (N·m).

[0105] The constraints are: ; in, : Represents the L2 norm of joint torque (dimensionless); : Indicates the upper limit of joint torque; :express The L2 norm of the velocity vector of the robot's end effector at any given moment, in meters per second (m / s). : Indicates the safe speed threshold after a collision, in meters per second (m / s). Dangerous area: This refers to the area in the robot's operating space where secondary collisions are likely to occur (defined by the robot's geometric model).

[0106] Predicting states using differentiable dynamics models: ; : Represents a differentiable dynamics model function for a robot; :express The actual position of the robot's end effector at any given time, in meters (m). : Represents the collision force vector (including magnitude and direction), with the unit being Newtons (N).

[0107] Solving for the optimal control sequence using the gradient descent algorithm: ; : Optimal joint torque sequence, in Newton-meter (N·m); Solve for the objective function Minimum torque sequence .

[0108] Output control commands that are adjusted in real time: .

[0109] : Represents the optimized robot joint torque control command, in Newton-meter (N·m).

[0110] Online updates of control parameters: The control parameters (including the energy-impulse calibration coefficients) are updated using the gradient descent algorithm. Short-time energy threshold To improve the accuracy of subsequent detection: ; ; in, Updated energy-impulse calibration coefficient; : Energy-impulse calibration coefficient before update; The learning rate for gradient descent; Collision energy measured by force sensor; Updated energy threshold; : Short-time energy baseline value under collision-free conditions; : Environmental noise energy at any given moment; The output is as follows: Optimized robot control instructions: ; Control parameters updated online: , .

[0111] In summary, the abnormal collision detection and control method for robots described in this application firstly captures collision information from both acoustic and motion dimensions by synchronously acquiring multi-channel sound and robot state data. Combining acoustic features with cross-validation of state data accurately identifies abnormal collisions, effectively reducing false positives and false negatives. Secondly, after detecting an abnormal collision, control parameters are optimized based on collision-related data to avoid secondary collisions and ensure safe task execution. This method can accurately detect abnormal collision events during robot operation and optimize control accordingly.

[0112] In some embodiments, please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a control module provided in an embodiment of this application. As can be seen, the control module includes a six-layer software architecture, as detailed below: Data acquisition layer: Multi-channel sound data and robot status data are acquired synchronously through a multi-channel sound acquisition module and a robot status perception module.

[0113] Preprocessing layer: Performs preprocessing operations such as noise reduction, alignment, and outlier removal on multi-channel audio data and robot status data.

[0114] Feature extraction layer: Extracts acoustic features from multi-channel sound data and combines them with the robot's state to obtain feature vectors.

[0115] Collision Detection and Quantization Layer: Based on feature vectors and robot state data, it identifies collisions and calculates the collision location, collision force magnitude, and collision direction.

[0116] Strategy optimization layer: Optimizes and updates the robot's control parameters based on MPC and a preset update algorithm.

[0117] Data filtering layer: Filters low-quality data using data quality scores and updates the training dataset.

[0118] By processing each layer sequentially, the robot achieves real-time, accurate collision detection and adaptive control.

[0119] Please see Figure 7 , Figure 7 This is a functional unit block diagram of an abnormal collision detection and control device for a robot provided in an embodiment of this application. It is applied to the control module of a robot, and the robot further includes: a multi-channel sound acquisition module and a robot state perception module. The abnormal collision detection and control device 700 for the robot includes: an acquisition unit 701, a processing unit 702, a collision detection unit 703, and an optimization unit 704, wherein: The acquisition unit 701 is used to acquire sound data of the robot within a preset distance range through the multi-channel sound acquisition module within a first preset time period to obtain first multi-channel sound data; and to acquire the robot's state data within the first preset time period through the robot state perception module to obtain first state data. The processing unit 702 is used to preprocess the first multi-channel sound data and the first state data to obtain second multi-channel sound data and second state data; and to extract features from the second multi-channel sound data to obtain first feature data. The collision detection unit 703 is used to perform abnormal collision detection based on the first feature data and the second state data to obtain a target collision detection result; the target collision detection result includes any one of the following: abnormal collision, non-abnormal collision. The optimization unit 704 is used to optimize the control parameters of the robot based on the first feature data, the second multi-channel sound data, the second state data and the target collision detection result.

[0120] In specific implementations, the abnormal collision detection and control device 700 for robots described in the embodiments of the present invention can also execute other implementations described in the abnormal collision detection and control method for robots provided in the embodiments of the present invention, which will not be repeated here.

[0121] Please see Figure 8 , Figure 8This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor, a memory, a communication interface, and one or more programs. The processor, memory, and communication interface can be interconnected via a bus. The one or more programs are stored in the memory and configured to be executed by the processor. In this embodiment, the programs include instructions for performing some or all of the steps described in the above method embodiments.

[0122] The processor can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, cells, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication unit can be a communication interface, transceiver, transceiver circuit, etc., and the storage unit can be a memory.

[0123] The memory can be volatile or non-volatile, or a combination of both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0124] It is understood that electronic devices may include more or fewer structural elements than those shown in the above block diagram, such as power modules, physical buttons, Wi-Fi modules, speakers, Bluetooth modules, sensors, display modules, etc., without limitation.

[0125] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0126] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0127] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0128] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0131] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Furthermore, the ASIC can reside in a terminal device or management device. Alternatively, the processor and storage medium can exist as discrete components in the terminal device or management device.

[0132] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated.

[0133] The aforementioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.

[0134] The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0135] The modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on the processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented through a software program that runs on the processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0136] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. An abnormal collision detection and control method of a robot, characterized by, A control module for a robot, the robot further comprising: a multi-channel sound acquisition module and a robot state perception module, the method comprising: Within a first preset time period, the multi-channel sound acquisition module acquires sound data within a preset distance range of the robot to obtain first multi-channel sound data; The robot state perception module collects the robot's state data within the first preset time period to obtain first state data. The first multi-channel audio data and the first state data are preprocessed to obtain the second multi-channel audio data and the second state data. Feature extraction is performed on the second multi-channel audio data to obtain the first feature data; Anomaly collision detection is performed based on the first feature data and the second state data to obtain target collision detection results; the target collision detection results include any of the following: abnormal collision, non-abnormal collision. The control parameters of the robot are optimized based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result; Wherein, the second multi-channel audio data includes a channels of audio data; a is an integer greater than 1; The optimization of the robot's control parameters based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result includes: Determine the time when a collision signal is detected in each of the a channels of audio data to obtain a time points; Determine the time difference between every two times in the given a times to obtain b time differences; b is a positive integer. The target positioning equation is determined based on the b time differences; Based on the first feature data, the target localization equation, the second multi-channel sound data, and the target collision detection result, a collision parameter set is determined; The control parameters of the robot are optimized based on the collision parameter set and the second state data.

2. The method as described in claim 1, characterized in that, The preprocessing of the first multi-channel audio data and the first state data to obtain second multi-channel audio data and second state data includes: The first multi-channel audio data and the first state data are processed according to a preset interpolation algorithm to obtain third multi-channel audio data and third state data; the sampling rate of the third multi-channel audio data is equal to the sampling rate of the third state data; Determine the target operating environment corresponding to the robot; Determine the reference noise type corresponding to the target operating environment; Determine the reference denoising algorithm corresponding to the reference noise type; The third multi-channel audio data is processed according to the reference denoising algorithm to obtain the fourth multi-channel audio data; The fourth multi-channel audio data is enhanced to obtain the second multi-channel audio data; The third state data is filtered to obtain the second state data.

3. The method as described in claim 1, characterized in that, The step of performing abnormal collision detection based on the first feature data and the second state data to obtain the target collision detection result includes: Obtain the reference parameter set of the robot in a collision-free state; Based on the reference parameter set and the preset calibration coefficient set, determine the collision parameter threshold set; Based on the collision parameter threshold set and the first feature data, a reference collision detection result is determined; The reference collision detection result is verified based on the second state data to obtain the target verification result; the target verification result includes any of the following: verification successful, verification failed; When the target verification result includes the verification success, the target collision detection result is determined based on the reference collision detection result; When the target verification result includes the verification failure, new sound data and new status data within a second preset time period are obtained, and abnormal collision detection is performed based on the new sound data and the new status data to obtain the target collision detection result; the start time of the second preset time period is later than the end time of the first preset time period.

4. The method as described in claim 3, characterized in that, The first feature data includes: first short-time energy data, first kurtosis data, and first spectral centroid data; the collision parameter threshold set includes: short-time energy threshold, kurtosis threshold, and spectral centroid threshold. The step of determining the reference collision detection result based on the collision parameter threshold set and the first feature data includes: Determine the maximum short-time energy value corresponding to the short-time energy data; Determine the maximum kurtosis value corresponding to the first kurtosis data; Determine the maximum spectral centroid value corresponding to the first spectral centroid data; When the maximum short-time energy value is greater than the short-time energy threshold, the maximum kurtosis value is greater than the kurtosis threshold, and the maximum spectral centroid value and the spectral centroid threshold meet preset conditions, the reference collision detection result is determined to include the abnormal collision. When the maximum short-time energy value is not greater than the short-time energy threshold, or the maximum kurtosis value is not greater than the kurtosis threshold, or the maximum spectral centroid value and the spectral centroid threshold do not meet the preset conditions, the reference collision detection result is determined to include the non-abnormal collision.

5. The method as described in claim 3, characterized in that, The collision parameter set includes: collision event marker, collision event occurrence time, target collision position coordinates, target collision direction, first feature vector, and collision force magnitude; The step of determining the collision parameter set based on the first feature data, the target localization equation, the second multi-channel sound data, and the target collision detection result includes: The collision event marker and the time of occurrence of the collision event are determined based on the target collision detection results; The target location equation is solved to obtain the target collision position coordinates; Determine the joint center coordinates corresponding to the target collision position coordinates in the robot; The target collision direction is determined based on the joint center position coordinates and the target collision position coordinates; The first feature vector is determined based on the first feature data; The magnitude of the collision force is determined based on the time of occurrence of the collision event and the second multi-channel sound data.

6. The method as described in claim 5, characterized in that, The step of optimizing the robot's control parameters based on the collision parameter set and the second state data includes: The collision parameter set is scored according to a preset data quality scoring function to obtain the target score; When the target score is greater than the preset score, a training dataset is determined based on the second multi-channel sound data, the collision parameter set, and the second state data; the control parameters of the robot are optimized based on the training dataset and the preset update algorithm to obtain new control parameters.

7. An abnormal collision detection and control device for a robot, used to perform the method as described in any one of claims 1-6, characterized in that, The control module is applied to a robot, which further includes: a multi-channel sound acquisition module and a robot state perception module. The device comprises: an acquisition unit, a processing unit, a collision detection unit, and an optimization unit, wherein: The acquisition unit is used to acquire sound data of the robot within a preset distance range through the multi-channel sound acquisition module within a first preset time period to obtain first multi-channel sound data; and to acquire the robot's state data within the first preset time period through the robot state perception module to obtain first state data. The processing unit is configured to preprocess the first multi-channel audio data and the first state data to obtain second multi-channel audio data and second state data; and to extract features from the second multi-channel audio data to obtain first feature data. The collision detection unit is used to perform abnormal collision detection based on the first feature data and the second state data to obtain a target collision detection result; the target collision detection result includes any one of the following: abnormal collision, non-abnormal collision. The optimization unit is used to optimize the control parameters of the robot based on the first feature data, the second multi-channel sound data, the second state data, and the target collision detection result.

8. An electronic device, characterized in that, include: Processor, memory, communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange is provided, wherein the computer program causes a computer to perform the method as described in any one of claims 1-6.

Images