A robot arm calibration method based on deep learning and ultrasonic microarray

Abstract

The application discloses a kind of based on deep learning and ultrasonic microarray's mechanical arm calibration method, the method includes: step S1, set square wave signal, through audio power amplifier module emits square wave signal, and through microphone array receives;Step S2, the ultrasonic signal received by microphone array is AD sampled, and digital signal is obtained;Step S3, constructs the azimuth angle and pitch angle acquisition model of mechanical arm, input to the azimuth angle and pitch angle acquisition model of mechanical arm in digital signal, and the azimuth angle and pitch angle of sound source are obtained;Step S4, based on the azimuth angle and pitch angle of sound source obtained in step S3, the coordinate system of microphone array is established to obtain the position of sound source in three-dimensional space, and the difference between the target position of mechanical arm and the spatial coordinates of sound source is adjusted to target position.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of direction of arrival estimation, and particularly relates to a mechanical arm calibration method based on deep learning and an ultrasonic microarray. BACKGROUND

[0002] At present, in the assembly of aerospace devices and the industrialized production of precision instruments, although automatic production and assembly can be realized, for the production devices such as mechanical arms and robots, due to the relatively large weight of the mechanical arm itself, the mutual friction between parts in the mechanical arm, and the temperature change of the environment, the mechanical arm will generate certain loss, which will inevitably cause errors in the production and processing assembly process, and the quality of the finished products produced will also decrease.

[0003] The robot precision mainly includes repeat positioning precision and absolute positioning precision, which is one of important indexes for evaluating the comprehensive performance of the robot. At present, the repeat positioning precision of the industrial robot developed at home and abroad is relatively high, which can reach 0.01 mm, but due to the combined action of the machining error, assembly error, wear of parts, change of end load and temperature influence of the robot, the absolute positioning precision is relatively low. This has an adverse effect on the application of the measuring robot, the high-precision assembly of the mechanical arm satellite and the integrated assembly of the aircraft. Especially, the motion of the mechanical arm in the narrow space is not accurate, which causes the collision between the end assembly parts and the surrounding high-precision parts, resulting in the damage of the high-precision parts and unnecessary loss. With the continuous application and development of the industrial robot, higher and higher requirements are put forward for the motion precision.

[0004] Nowadays, a series of mathematical models are researched at home and abroad to optimize the debugging of the mechanical arm, such as SVD decomposition and projection constraint, which can improve the precision to a certain extent, but still cannot eliminate the error of artificial measurement. There is also a way of calibrating the mechanical arm based on binocular vision and camera, but the cost of the equipment is too high, which is not suitable for popularization.

[0005] Therefore, it is of great significance to deeply study the absolute positioning precision of the mechanical arm for the development of the industrial robot. SUMMARY

[0006] In view of the deficiencies in the prior art, the application provides a mechanical arm calibration method based on deep learning and an ultrasonic microarray.

[0007] According to a first aspect of the embodiment of the application, a mechanical arm calibration method based on deep learning and an ultrasonic microarray is provided, and the method comprises the following steps:

[0008] Step S1, a square wave signal is set, the square wave signal is emitted through an audio power amplifier module, and the square wave signal is received through a microphone array;

[0009] Step S2, AD sampling is performed on the ultrasonic signal received by the microphone array to obtain a digital signal;

[0010] Step S3, a model for obtaining the azimuth angle and the pitch angle of the mechanical arm is constructed, the digital signal is input into the model for obtaining the azimuth angle and the pitch angle of the mechanical arm, and the azimuth angle and the pitch angle of the sound source are obtained.

[0011] Step S4, based on the azimuth angle and the pitch angle of the sound source obtained in step S3, a coordinate system of the microphone array is established to obtain the position of the sound source in a three-dimensional space, and the mechanical arm is adjusted to the target position according to the difference between the target position of the mechanical arm and the spatial coordinates of the sound source.

[0012] According to a second aspect of the embodiment of the present application, an electronic device is provided, comprising a memory and a processor, the memory being coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the above-mentioned mechanical arm calibration method based on deep learning and ultrasonic microarray.

[0013] According to a third aspect of the embodiment of the present application, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to realize the above-mentioned mechanical arm calibration method based on deep learning and ultrasonic microarray.

[0014] Compared with the prior art, the present application has the following beneficial effects: the present application provides a mechanical arm calibration method based on deep learning and ultrasonic microarray, which only needs to install a transmitter on the mechanical arm, and can collect and position the sound source through the microphone array, and adjust the mechanical arm to the target position according to the difference between the target position of the mechanical arm and the spatial coordinates of the sound source. The present application is portable, reduces the cost, and improves the calibration accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0016] Figure 1 The flow chart of the mechanical arm calibration method based on deep learning and ultrasonic microarray provided by the embodiment of the present application is shown in the figure.

[0017] Figure 2 The schematic diagram of the mechanical arm calibration device provided by the embodiment of the present application is shown in the figure.

[0018] Figure 3 The structure diagram of the model for obtaining the azimuth angle and the pitch angle of the mechanical arm provided by the embodiment of the present application is shown in the figure.

[0019] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0022] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a robotic arm calibration method based on deep learning and ultrasonic microarrays, the method comprising:

[0023] Step S1: Set the square wave signal, transmit the square wave signal through the audio power amplifier module, and receive it through the microphone array.

[0024] Specifically, a 40kHz sine wave with an amplitude of 3V and a duty cycle of 50% is generated by a signal generator and connected to the LM386 audio amplifier module via an SW-M221 MOSFET module. In this example, the SW-M221 MOSFET module and the LM386 are both connected to a Raspberry Pi. The Raspberry Pi provides a PWM signal with an adjustable duty cycle; the SW-M221 MOSFET is on when the level is high and off when the level is low, allowing the sine wave signal from the LM386 audio amplifier module to be switched on and off within a certain period, achieving a square wave effect. In this example, a data frame is 0.1s, therefore the square wave frequency is 10Hz, and the PWM signal generated by the Raspberry Pi is also 10Hz with a 50% duty cycle.

[0025] It should be noted that the square wave form is used to make the reverberation more obvious. Since the reverberation is manifested by the time offset and amplitude attenuation of the main path signal, when the signal transmitted by the LM386 audio power amplifier module is shielded, the signal received by the microphone will definitely be reverberation and noise. This data can help the neural network learn the characteristics of the data better, which is convenient for the neural network to fit.

[0026] The LM386 chip emits a generated square wave signal, which is received by a microphone array. The microphone array consists of two 4-element cross-shaped ultrasonic arrays, for a total of eight microphones. The radius of each cross-shaped ultrasonic array is 2mm. Figure 2As shown, the ultrasonic signal received by the microphone array is considered as a far-field signal in the current scenario.

[0027] Step S2, AD sampling is performed on the ultrasonic signal received by the microphone array to obtain a digital signal.

[0028] Specifically, in the present example, the AD7606 chip is driven by the Raspberry Pi, and eight channels of the AD7606 chip are connected with the microphone array respectively, that is, the ultrasonic signals received by the eight microphones are AD sampled, and eight-channel parallel analog-to-digital conversion is achieved. The converted digital signal is recorded and saved in the Raspberry Pi for easy reading by the subsequent deep learning.

[0029] Step S3, a mechanical arm azimuth and pitch angle acquisition model is constructed, and the digital signal is input into the mechanical arm azimuth and pitch angle acquisition model to obtain the azimuth and pitch angle of the sound source.

[0030] It should be noted that the deep learning method has excellent robustness to noise and reverberation carried by the digital signal. In theory, under the condition that the data set is large enough, the deep learning method can realize high-precision positioning in a reverberation and noise environment, but in practice, it is limited by GPU resources and time, so such a method cannot be used here. However, the high robustness of deep learning to noise and reverberation can still be utilized, that is, a mechanical arm azimuth and pitch angle acquisition model is trained to extract the main path from data carrying noise and reverberation, and then the MUSIC algorithm is used to estimate the DOA of the extracted signal to realize 0.1 ° precision positioning. Since the Raspberry Pi has limited computing resources and the device is required to have certain real-time performance, the Intel Neural Compute Stick is used to help the model inference in the present example.

[0031] Specifically, the step S3 specifically includes the following sub-steps:

[0032] Step S301, simulation data including a main path signal, reverberation and noise are generated.

[0033] The simulation data is 0.1s per frame of data, a 4-element cross ultrasonic array includes 4 microphones, and the sampling frequency is 3kHz, so the size of a group of data is 4*300. A frame of data is composed of a main path signal, reverberation and Gaussian noise.

[0034] The main path signal S1 is a combination of a sine signal and a square wave, and the signal formula is as follows:

[0035] S1=u(sin(ω2t+φ2))*sin(ω1t+φ1)

[0036] In the formula, u is a step function combined with a sine function to generate a square wave, φ1 is the phase of the sound source signal, φ2 is the phase of the square wave signal, ω1 is the frequency of the sound source signal, ω2 is the frequency of the square wave signal, and t is time.

[0037] The reverberation signal is in the form of a virtual sound source in space that is much farther away from the microphone than the sound source. The reverberation is derived from the reflection of the main path signal in space, so the virtual sound source signal is essentially the main path signal that is attenuated in amplitude and delayed in time, but its initial phase is consistent with the main path signal. The formula for reverberation is as follows:

[0038] S2 = u (sin (ω2 (t-Δt) + φ2) * A * sin (ω1 (t-Δt) + φ1)

[0039] where A is the amplitude of the signal after attenuation, and Δt is the time offset of the reverberation relative to the main path signal. In spherical propagation of sound, the signal energy is inversely proportional to the square of the distance, and then this is the signal energy proportional to the square of the amplitude, so the final signal amplitude is inversely proportional to the distance. Here it is assumed that the time difference between the reverberation and the main path cannot be greater than 0.05s, since 0.05*340 = 17m, i.e. the main path signal travels 17m after 0.05s, the amplitude will be attenuated to 1 / 17 of the original, which can be ignored.

[0040] In the entire signal generation process, Gaussian noise needs to be added to the signal according to a certain signal-to-noise ratio. Let the Gaussian noise be N, then the function of the final simulation data is:

[0041] Signal = S1 + S2 + N

[0042] The number of virtual sound sources in a frame of data is randomly specified, and the positions of the virtual sound sources are different. The signal-to-noise ratio of the signal between each frame is generated in the form of a random number. A certain amount of data is generated for each combination of azimuth angle and elevation angle.

[0043] In step S302, a mechanical arm azimuth angle and elevation angle acquisition model is constructed, and the simulation data is input into the mechanical arm azimuth angle and elevation angle acquisition model for training.

[0044] As shown in Figure 3 In this example, the mechanical arm azimuth angle and elevation angle acquisition model adopts a self-encoding structure and is constructed by referring to U-Net and ResNet networks.

[0045] The simulation data generated in step S301 is input into the mechanical arm azimuth and elevation angle acquisition model, wherein the simulation data is input in groups of 3 frames of data, one frame of data being a 4*300 matrix, and three groups of simulation data with the same azimuth and elevation angle but different initial phase and reverberation are input as the input of the mechanical arm azimuth and elevation angle acquisition model. In this way, on the one hand, 2D convolution can be used for training, and on the other hand, the accuracy of three groups of data is often higher than that of one group of data. Therefore, the format of the input data of the mechanical arm azimuth and elevation angle acquisition model is 3*4*300. The label of each group of data, i.e., the output of the mechanical arm azimuth and elevation angle acquisition model, is the main path signal under each group of data.

[0046] In step S303, the digital signal is input into the trained mechanical arm azimuth and elevation angle acquisition model to obtain the azimuth and elevation angle of the sound source.

[0047] In step S4, based on the azimuth and elevation angle of the sound source obtained in step S3, a coordinate system of the microphone array is established to obtain the position of the sound source in the three-dimensional space, and the mechanical arm is adjusted to the target position according to the difference between the target position of the mechanical arm and the spatial coordinates of the sound source.

[0048] Specifically, establishing a coordinate system of the microphone array to obtain the position of the sound source in the three-dimensional space includes: according to the relative coordinate relationship in the space, two groups of four-element cross ultrasonic receiving arrays are respectively established as coordinate system A and coordinate system B. The origin of each coordinate system is the center point of the four microphones, the distance between each microphone and the microphone center is 2mm, the distance between the two origins is 2cm, and the line connecting the other two origins is set as the x axis. The midpoint of the two coordinate system origins is set as the origin of the final coordinate system, so that the problem can be converted into finding the position of this point in the final coordinate system.

[0049] Let the azimuth of the sound source in the coordinate system A be θ1, and the elevation angle be The azimuth in the coordinate system B is θ2, and the elevation angle is The coordinates of the sound source in the final coordinate system are (x, y, z), and the following calculation formula is obtained:

[0050]

[0051] In the formula, r is the distance of the sound source from the center point of the microphone array.

[0052] The three-dimensional space position of the sound source is obtained from the above formula, the target pose of the mechanical arm is preset, and the mechanical arm is adjusted to the target position by using the ROS system according to the difference between the preset target position of the mechanical arm and the spatial coordinates of the sound source.

[0053] In conclusion, the application provides a mechanical arm calibration method based on deep learning and ultrasonic microarray, which only needs a transmitter mounted on the mechanical arm to collect and locate the sound source through the ultrasonic array, and adjusts the mechanical arm to the target position according to the difference between the target position of the mechanical arm and the spatial coordinates of the sound source. The method is portable, reduces the cost, and improves the calibration accuracy.

[0054] Correspondingly, the application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the mechanical arm calibration method based on deep learning and ultrasonic microarray as described above. Figure 4 As shown in the figure, the mechanical arm calibration method based on deep learning and ultrasonic microarray provided by the embodiment of the application is a hardware structure diagram of any data processing capable device. In addition to the processor, the memory and the network interface shown in the figure, any data processing capable device in which the device in the embodiment is usually according to the actual function of the data processing capable device, and can also include other hardware, and this will not be described again. Figure 4

[0055] Correspondingly, the application also provides a computer readable storage medium having computer instructions stored thereon, which are executed by a processor to implement the mechanical arm calibration method based on deep learning and ultrasonic microarray as described above. The computer readable storage medium can be an internal storage unit of any data processing capable device, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit of any data processing capable device and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0056] The above-described embodiments only express several implementation manners of the application, which are described in detail and specifically, but should not be understood as limitations on the patent scope of the application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the application, several modifications and improvements can be made, which are all within the protection scope of the application. Therefore, the patent protection scope of the application should be subject to the appended claims.​

Claims

1. A robotic arm calibration method based on deep learning and ultrasonic microarray, characterized in that, A transmitter is mounted on a robotic arm, and a microphone array is used to collect and locate the sound source. The method includes the following steps: Step S1: Set the square wave signal, transmit the square wave signal through the audio power amplifier module, and receive it through the microphone array; Step S2: Perform AD sampling on the ultrasonic signal received by the microphone array to obtain a digital signal; Step S3: Construct a model for obtaining the azimuth and pitch angles of the robotic arm. Input the digital signal into the model to obtain the azimuth and pitch angles of the sound source. The model can extract the main path of the data carrying noise and reverberation, and then use the MUSIC algorithm to estimate the DOA of the extracted signal. Step S3 specifically includes the following sub-steps: Step S301: Generate analog data including the main path signal, reverberation signal, and noise; including: Main path signal The expression is: ； In the formula, This is a step function used to generate the envelope of a square wave signal. The phase of the sound source signal. The phase of the square wave signal. The frequency of the sound source signal, The frequency of the square wave signal, For time; Reverberation signal The expression is: ； In the formula, A is the amplitude of the signal after attenuation. The time offset of the reverberation signal relative to the main path signal; The expression for the simulated data is: ； In the formula, It is Gaussian noise; Step S302: Construct a model for obtaining the azimuth and pitch angles of the robotic arm, and input the simulation data into the model for training. Step S303: Input the digital signal into the trained robotic arm azimuth and pitch angle acquisition model to obtain the azimuth and pitch angle of the sound source. Step S4: Based on the azimuth and elevation angles of the sound source obtained in step S3, establish a coordinate system for the microphone array to obtain the position of the sound source in three-dimensional space, and adjust the robotic arm to the target position according to the difference between the target position of the robotic arm and the spatial coordinates of the sound source.

2. The robotic arm calibration method based on deep learning and ultrasonic microarray according to claim 1, characterized in that, The square wave signal is defined as follows: A 40kHz, 3V amplitude, and 50% duty cycle sine wave signal is generated by a signal generator and connected to the audio amplifier module via one end of a field-effect transistor (FET). The other end of the FET is connected to a controller. The controller provides an adjustable PWM signal, which turns the FET on when the level is high and turns it off when the level is low. This allows the sine wave signal of the audio amplifier module to be turned on and off within a cycle, resulting in a square wave signal.

3. The robotic arm calibration method based on deep learning and ultrasonic microarray according to claim 1, characterized in that, The microphone array consists of two quadruple cross-shaped ultrasonic arrays, for a total of eight microphones.

4. The robotic arm calibration method based on deep learning and ultrasonic microarray according to claim 3, characterized in that, The radius of the four-element cross-shaped ultrasound array is 2 mm.

5. The robotic arm calibration method based on deep learning and ultrasonic microarray according to claim 1, characterized in that, Step S302 includes: The model for acquiring the azimuth and pitch angles of the robotic arm adopts an autoencoder structure and includes U-Net and ResNet networks; The model for obtaining the azimuth and pitch angles of the robotic arm is trained using 2D convolution.

6. The robotic arm calibration method based on deep learning and ultrasonic microarray according to claim 3, characterized in that, Step S4 includes: Based on the relationship of relative coordinates in three-dimensional space, coordinate systems are established for the two four-element cross-shaped ultrasonic arrays, denoted as coordinate system A and coordinate system B, respectively. The origin of each coordinate system is the center point of the four microphones. Let the line connecting the origins of the two coordinate systems be denoted as The midpoint of the line connecting the origins of the two coordinate systems is set as the origin of the final coordinate system. Let the azimuth angle of the sound source in coordinate system A be . The pitch angle is The azimuth angle in coordinate system B is The pitch angle is The coordinates of the sound source in the final coordinate system are The expression for the position of the sound source in three-dimensional space is as follows: 。 7. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the robotic arm calibration method based on deep learning and ultrasonic microarray as described in any one of claims 1-6.

8. A computationally readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the robotic arm calibration method based on deep learning and ultrasonic microarray as described in any one of claims 1-6.

Images