Neural Network-Based Electromagnetic Signal Image Data Processing Method

Abstract

The present application provides a method for processing electromagnetic signal image data based on a neural network. A detection radar is used to scan the to-be-assembled component according to a preset scanning path, and the echo electromagnetic signal returned by the detection radar when the scanned electromagnetic signal hits the to-be-assembled component is obtained. The echo electromagnetic signal and the scanned electromagnetic signal are subjected to mixing processing to obtain an intermediate frequency signal. The intermediate frequency signal is subjected to wavelet transform to obtain a time-frequency signal. The time-frequency signal is converted into a time-frequency image. The time-frequency image is segmented to obtain a plurality of time-frequency blocks and the positions of each time-frequency block. Input features are generated according to the plurality of time-frequency blocks and the positions of each time-frequency block. An encoder is used to encode the input features to obtain low-dimensional point cloud features. A decoder is used to decode the low-dimensional point cloud features to obtain the point cloud data of the to-be-assembled component. The encoder is controlled to output the scale of the point cloud data, so as to output point cloud data with a higher density.

Description

Technical Field

[0001] This application relates to the field of assembly robots, and particularly to a method for processing electromagnetic signal image data based on a neural network. Background Art

[0002] With the development of technology, assembly robots are widely used in industrial production. A detection radar is mounted on the assembly robot to detect the pose of the component to be assembled, and then assembly is performed based on the pose of the component.

[0003] The detection radar scans the component to be assembled to obtain the point cloud of the assembly component. More specifically, the detection radar continuously emits signals externally and receives the echo electromagnetic signals reflected by the component to be assembled. Based on the intensity and phase of the echo electromagnetic signals, a point position of the component to be assembled is obtained. By emitting signals at different angles, scanning of one surface of the component to be assembled is achieved. After a scanning cycle, the obtained multiple echo electromagnetic signals are filtered to obtain each echo electromagnetic signal, and then each echo electromagnetic signal is processed to obtain a coordinate data. By traversing each echo electromagnetic signal, the point cloud of one surface is obtained.

[0004] However, the point cloud obtained by the above processing method is relatively sparse, which in turn leads to insufficient accuracy of the obtained point cloud. Summary of the Invention

[0005] The embodiments of this application provide a method for processing electromagnetic signal image data based on a neural network, which is used to convert the intermediate frequency signal obtained by mixing the echo electromagnetic signal and the scanning electromagnetic signal into a time-frequency spectrogram, and process the time-frequency spectrum based on the model constructed by the neural network to obtain point cloud data, so as to obtain denser point cloud data.

[0006] The embodiments of this application provide a method for processing electromagnetic signal image data based on a neural network. The method is applied to an assembly robot, and a detection radar is set on the assembly robot. The method includes:

[0007] Use the detection radar to scan the component to be assembled according to a preset scanning path, and obtain the echo electromagnetic signal returned by the scanning electromagnetic signal emitted by the detection radar when it hits the component to be assembled;

[0008] Perform mixing processing on the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image;

[0009] Divide the time-frequency image into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and generate input features according to the multiple time-frequency blocks and the positions of each time-frequency block;

[0010] Encode the input features using an encoder to obtain low-dimensional point cloud features; wherein, the low-dimensional point cloud features are used to represent position information and angle information;

[0011] Use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled.

[0012] In the above technical solution, after the detection radar receives the echo electromagnetic signal, mix the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, so as to analyze the echo electromagnetic signal in the time-frequency domain, convert the time-frequency signal into a time-frequency diagram, divide the time-frequency diagram into multiple time-frequency diagram blocks, obtain the positions of each time-frequency diagram block, generate input features according to the time-frequency diagram block and the positions of each time-frequency diagram block, use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to restore the low-dimensional point cloud features to obtain point cloud data, which can control the scale of the point cloud data output by the encoder, so as to ensure that the encoder can output point cloud data with higher density, thereby improving the accuracy of the point cloud data.

[0013] In a possible implementation manner, converting the time-frequency signal into a time-frequency image specifically includes:

[0014] Obtain the frequency range covered by the time-frequency signal, and segment the frequency range to obtain m frequency segments;

[0015] Obtain the time range covered by the time-frequency signal, and segment the time range to obtain n time periods;

[0016] Obtain the amplitude of the time-frequency signal in the i-th frequency segment and the j-th time period, calculate the average value of the amplitude of the time-frequency signal in the i-th frequency segment and the j-th time period, and use it as the pixel value of the i-th row and j-th column; let i traverse from 1 to m, and j traverse from 1 to n to obtain the time-frequency image.

[0017] In the above technical solution, by dividing the frequency range of the time-frequency signal to obtain multiple frequency segments, dividing the time range of the time-frequency signal to obtain multiple time periods, using the serial number of the frequency segment as the pixel row serial number, using the serial number of the time period as the pixel column serial number, and using the amplitude of the i-th frequency segment and the j-th time period as the pixel value of the corresponding pixel, so as to retain the time, frequency and amplitude signals in the time-frequency signal when converting the time-frequency signal into a time-frequency image. Furthermore, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0018] In a possible implementation manner, dividing the time-frequency image into multiple time-frequency diagram blocks and the positions of each time-frequency diagram block specifically includes:

[0019] Divide the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency diagram block array;

[0020] Obtain the frequency range corresponding to each time-frequency map block, calculate the average value of the frequency range corresponding to each time-frequency block, and use it as the ordinate corresponding to each time-frequency map block;

[0021] Obtain the time range corresponding to each time-frequency map block, calculate the average value of the time range corresponding to each time-frequency block, and use it as the abscissa corresponding to each time-frequency map block.

[0022] In the above technical solution, the time-frequency image is divided into multiple time-frequency map blocks, the frequency range and time range corresponding to each time-frequency map block are obtained, the frequency corresponding to each time block is used as the ordinate of the image block, and the time corresponding to each time block is used as the abscissa of the image block. Thus, input features are generated based on the image blocks and the positions of the image blocks, so that the input features contain amplitude, time, and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0023] In a possible implementation manner, generating input features according to multiple time-frequency map blocks and the positions of each time-frequency map block specifically includes:

[0024] Use multiple convolutional kernels to perform multiple convolutional operations on the time-frequency map blocks to obtain the amplitude features of the time-frequency map blocks;

[0025] Use a transformation function to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks;

[0026] Calculate the sum of the amplitude features of the time-frequency map blocks and the position features of the time-frequency map blocks to obtain the fusion features of the time-frequency map blocks;

[0027] Concatenate the fusion features of all time-frequency map blocks to generate input features.

[0028] In the above technical solution, use convolutional kernels to process the time-frequency map blocks to obtain the amplitude features of the time-frequency map blocks, use a transformation function to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks, obtain the fusion features of the time-frequency map blocks based on the amplitude features and coordinate features of the time-frequency map blocks, and concatenate the fusion features of each time-frequency map block to obtain input features, so that the input features contain amplitude, time, and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0029] In a possible implementation manner, using a transformation function to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks specifically includes:

[0030] Use a horizontal coordinate transformation function to transform the horizontal coordinates of the time-frequency map blocks to obtain the horizontal coordinate features of the time-frequency map blocks;

[0031] The ordinate of the time-frequency map block is transformed using the ordinate transformation function to obtain the ordinate feature of the time-frequency map block;

[0032] The abscissa feature and the ordinate feature of the time-frequency map block are concatenated to obtain the position feature of the time-frequency map block.

[0033] In the above technical solution, the abscissa of the time-frequency map block is transformed using the abscissa transformation function to obtain the abscissa feature of the time-frequency map block, the ordinate of the time-frequency map block is transformed using the ordinate transformation function to obtain the ordinate feature of the time-frequency map block, and the abscissa feature and the ordinate feature of the time-frequency map block are concatenated. In this way, the position feature of the generated time-frequency map block contains time and frequency information.

[0034] In a possible implementation manner, an encoder is used to encode the input feature to obtain a low-dimensional point cloud feature, specifically including:

[0035] The input feature is encoded using the encoding module of the first level to obtain the first intermediate encoding feature, the first intermediate encoding feature is processed using the encoding module of the second level to obtain the second intermediate encoding feature, the second intermediate encoding feature is processed using the encoding module of the third level to output the third intermediate encoding feature, and the third intermediate encoding feature is processed using the encoding module of the fourth level to output the low-dimensional point cloud feature;

[0036] Each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. In each encoding module, the multi-head attention mechanism layer processes the input feature of the encoding module, and the feed-forward neural network layer processes the data output by the multi-head attention mechanism layer.

[0037] In the above technical solution, the encoder includes multiple encoding modules, each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. After the input feature passes through multiple encoding modules for information extraction, a low-dimensional point cloud feature is obtained, and distance and angle information are extracted from the amplitude, frequency, and time information in the time-frequency image. In this way, the point cloud data can be restored based on the low-dimensional point cloud feature.

[0038] In a possible implementation manner, a decoder is used to decode the low-dimensional point cloud feature to obtain the point cloud data of the component to be assembled, specifically including:

[0039] The low-dimensional point cloud feature is decoded using the first multi-layer perceptron layer to obtain the first decoded data; the data dimension of the low-dimensional point cloud feature is smaller than the data dimension of the first decoded data;

[0040] The first decoded data is decoded using the second multi-layer perceptron layer to obtain the second decoded data; the data dimension of the second decoded data is twice the data dimension of the first decoded data;

[0041] The second decoded data is decoded using a third multi-layer perceptron layer to obtain third decoded data; the data dimension of the second decoded data is greater than that of the third decoded data;

[0042] The third decoded data is decoded using a fourth multi-layer perceptron layer to obtain point cloud data.

[0043] In the above technical solution, a first multi-layer perceptron layer and a second multi-layer perceptron layer are used to increase the dimension of the low-dimensional point cloud features, so that the features output by the first multi-layer perceptron layer and the features output by the second multi-layer perceptron layer contain more position information. A third multi-layer perceptron layer and a fourth multi-layer perceptron layer are used to reduce the dimension of the data, and three-dimensional position information is generated based on multi-dimensional position information to realize the restoration of the point cloud data.

[0044] An embodiment of the present application provides an assembly robot, and a detection radar is provided on the assembly robot;

[0045] The detection radar is used to scan the parts to be assembled according to a preset scanning path, and the echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the parts to be assembled is obtained;

[0046] The echo electromagnetic signal and the scanning electromagnetic signal are mixed to obtain an intermediate frequency signal, the intermediate frequency signal is subjected to wavelet transform to obtain a time-frequency signal, and the time-frequency signal is converted into a time-frequency image;

[0047] The time-frequency image is divided into blocks to obtain a plurality of time-frequency blocks and the positions of each time-frequency block, and input features are generated according to the plurality of time-frequency blocks and the positions of each time-frequency block;

[0048] An encoder is used to encode the input features to obtain low-dimensional point cloud features; wherein, the low-dimensional point cloud features are used to represent position information and angle information;

[0049] A decoder is used to decode the low-dimensional point cloud features to obtain the point cloud data of the parts to be assembled.

[0050] An embodiment of the present application provides an electronic device, including: a memory, a processor;

[0051] The memory stores computer execution instructions;

[0052] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above various possible implementation manners.

[0053] An embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the above various possible implementation manners.

[0054] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements various possible implementation manners as above. Description of the Drawings

[0055] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0056] Figure 1 It is a schematic flowchart of the electromagnetic signal image data processing method based on a neural network provided by the present application;

[0057] Figure 2 It is a schematic diagram of the principle of generating a time-frequency image based on a time-frequency signal provided by the present application;

[0058] Figure 3 It is a schematic diagram of the principle of dividing a time-frequency image into blocks provided by the present application;

[0059] Figure 4 It is a schematic diagram of the structures of an encoder and a decoder provided by the present application;

[0060] Through the above drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and the textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0061] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0062] The detection radar scans the to-be-assembled component to obtain the point cloud of the assembled component. More specifically, the detection radar continuously emits signals externally and receives the echo electromagnetic signals reflected by the to-be-assembled component. Based on the intensity and phase of the echo electromagnetic signals, a point position of the to-be-assembled component is obtained. By emitting signals at different angles, scanning of one surface of the to-be-assembled component is achieved. After a scanning cycle, the obtained multiple echo electromagnetic signals are filtered to obtain each echo electromagnetic signal, and then each echo electromagnetic signal is processed to obtain a coordinate data. By traversing each echo electromagnetic signal, the point cloud of one surface is obtained.

[0063] However, the point cloud obtained by the above processing method is relatively sparse, which in turn leads to insufficient accuracy of the obtained point cloud.

[0064] The electromagnetic signal data processing method provided by this application is applied to an assembly robot. A detection radar is provided on the assembly robot, and the detection radar is used to detect the parts to be assembled. More specifically, after the assembly robot receives the echo electromagnetic signal, the echo electromagnetic signal and the scanning battery signal are mixed to obtain an intermediate frequency signal, and the intermediate frequency signal is subjected to wavelet transform to obtain a time-frequency spectrum diagram. Input features are generated based on the time-frequency spectrum diagram, and an encoder is used to process the input features to obtain low-dimensional point cloud features. Subsequently, a decoder is used to restore the low-dimensional point cloud features to generate point cloud data. By designing the data dimension of the low-dimensional point cloud data, the density of the restored point cloud data can be controlled, so that the obtained point cloud data can be more accurate.

[0065] The following uses specific embodiments to elaborate in detail on the technical solution of this application and how the technical solution of this application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below in conjunction with the accompanying drawings.

[0066] Figure 1 It is a schematic flowchart of the electromagnetic signal image data processing method based on a neural network provided by this application. As Figure 1 shown, the data processing method is applied to an electronic device on an assembly robot. The method specifically includes the following steps:

[0067] S101. The electronic device uses the detection radar to scan the parts to be assembled according to a preset scanning path, and obtains the echo electromagnetic signal returned by the scanning electromagnetic signal emitted by the detection radar when it encounters the parts to be assembled.

[0068] Among them, a detection radar is set on the assembly robot, and the detection radar emits scanning electromagnetic signals outward according to a preset scanning path to realize scanning of the parts to be assembled. The scanning electromagnetic signal is reflected back after encountering the parts to be assembled, and then the echo electromagnetic signal is received by the detection radar.

[0069] S102. Mix the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image.

[0070] Among them, a mixer is used to mix the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform processing on the intermediate frequency signal to obtain a time-frequency signal, and convert the amplitudes at different times and different frequencies in the time-frequency signal into the pixel values of each pixel point, so as to obtain a time-frequency image.

[0071] Filter the high-frequency signal in the intermediate-frequency signal using a filter to obtain the low-frequency signal in the intermediate-frequency signal. Perform wavelet transform processing on the low-frequency signal in the intermediate-frequency signal to obtain a time-frequency signal.

[0072] S103. Divide the time-frequency image into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and generate input features based on the multiple time-frequency blocks and the positions of each time-frequency block.

[0073] Among them, divide the time-frequency image horizontally and vertically to obtain multiple time-frequency blocks, and extract the positions of each time-frequency block. For example: Use 9 dividing lines both horizontally and vertically to obtain 10×10 time-frequency blocks, and obtain the positions of each time-frequency block according to the time and frequency corresponding to each time-frequency block.

[0074] After obtaining each time-frequency block and the position of the time-frequency block, use a feature conversion module to perform feature conversion on each time-frequency block and the position of the time-frequency block to obtain input features.

[0075] S104. Use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to perform decoding processing on the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled.

[0076] Among them, the low-dimensional point cloud features are used to represent position information and angle information. The encoder includes multiple encoding modules, and each encoding module includes a multi-head attention mechanism and a feed-forward neural network. Use multiple encoding modules to encode the input features multiple times to obtain low-dimensional point cloud features. The parameters in the encoding module can be obtained through learning.

[0077] The encoder is used to output data of n×3. One row of data represents the coordinates of a point. By designing the number n, the scale of the point cloud data output by the encoder can be controlled, so as to ensure that the encoder can output point cloud data with higher density, thereby improving the accuracy of the point cloud data.

[0078] In the above technical solution, after the detection radar receives the echo electromagnetic signal, mix the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate-frequency signal, perform wavelet transform on the intermediate-frequency signal to obtain a time-frequency signal, so as to analyze the echo electromagnetic signal in the time-frequency domain, convert the time-frequency signal into a time-frequency image, divide the time-frequency image into multiple time-frequency blocks, obtain the positions of each time-frequency block, generate input features based on the time-frequency blocks and the positions of each time-frequency block, use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to restore the low-dimensional point cloud features to obtain point cloud data. The scale of the point cloud data output by the encoder can be controlled, so as to ensure that the encoder can output point cloud data with higher density, thereby improving the accuracy of the point cloud data.

[0079] Optionally, after receiving the scanned electromagnetic signal emitted by the detection radar and its returned echo electromagnetic signal, first perform mixing processing on the two to obtain an intermediate frequency signal. Subsequently, perform wavelet transform on the intermediate frequency signal to extract the time-frequency signal. This step effectively removes the high-frequency noise in the signal and retains the key low-frequency information, providing high-quality input data for subsequent processing. Through mixing processing and wavelet transform, the noise in the signal is effectively suppressed, and at the same time, the key features in the signal are retained, laying a solid foundation for subsequent time-frequency analysis and feature extraction.

[0080] Convert the time-frequency signal into a time-frequency image. The specific steps include segmenting the frequency range and time range covered by the time-frequency signal and calculating the average amplitude value of the time-frequency signal within each segment as the pixel value, thereby constructing the time-frequency image. The conversion of the time-frequency image enables the intuitive display of signal features in the time and space dimensions, facilitating subsequent feature extraction and analysis. At the same time, through the segmentation process, the key time-frequency information in the signal is retained, improving the accuracy and efficiency of feature extraction.

[0081] Perform block processing on the time-frequency image to obtain multiple time-frequency blocks and their position information. Subsequently, perform convolution processing on the time-frequency blocks using a convolution kernel to extract amplitude features, and use a transformation function to transform the position coordinates to obtain position features. Finally, splice the amplitude features and position features to generate input features. Through time-frequency image block processing and feature extraction, the input features contain rich amplitude, time, and frequency information, providing comprehensive and accurate data support for the subsequent processing of the encoder. At the same time, block processing reduces the complexity of data processing and improves the processing efficiency.

[0082] Use an encoder to perform encoding processing on the generated input features to obtain low-dimensional point cloud features. The encoder adopts a multi-level encoding module structure, and each module contains a multi-head attention mechanism layer and a feed-forward neural network layer. Through multi-layer encoding, position information and angle information are extracted. The encoder deeply mines the input features through multi-level encoding modules, effectively extracting low-dimensional point cloud features representing position information and angle information. This process not only reduces the dimension of the data but also retains the key information, providing efficient and accurate data input for the subsequent processing of the decoder.

[0083] Use a decoder to perform decoding processing on the low-dimensional point cloud features to restore the point cloud data of the component to be assembled. The decoder adopts a multi-layer perceptron structure. Through layer-by-layer dimension elevation and reduction processing, three-dimensional point cloud data is finally output. The decoder efficiently decodes the low-dimensional point cloud features through the multi-layer perceptron structure, restoring dense and accurate point cloud data. This process not only improves the density of the point cloud data but also retains the geometric shape and position information of the component, providing reliable data support for subsequent assembly operations.

[0084] By designing the output dimension of the encoder, the scale of the point cloud data output by the decoder can be controlled. By adjusting the parameters and structure of the encoder, the density and coverage of the point cloud data can be flexibly controlled. By controlling the scale of the point cloud data, the density and coverage of the point cloud data can be adjusted according to actual needs. This process not only improves the applicability of the point cloud data, but also reduces the costs of data storage and processing.

[0085] During the entire processing process, through the collaborative work of multiple steps such as mixing processing, wavelet transform, time-frequency image conversion, feature extraction, encoding and decoding, etc., the accuracy of the point cloud data is effectively improved. Each step processes and optimizes the key features in the signal, and finally generates high-quality point cloud data. Through the collaborative work of multiple steps, the accuracy of the point cloud data is significantly improved. This not only provides reliable data support for subsequent assembly operations, but also improves the efficiency and accuracy of the entire assembly process.

[0086] In a possible implementation, S102, converting the time-frequency signal into a time-frequency image, specifically includes:

[0087] S201, obtaining the frequency range covered by the time-frequency signal, and segmenting the frequency range to obtain m frequency segments.

[0088] Figure 2 FIG. is a schematic diagram of the principle for generating a time-frequency image based on a time-frequency signal provided by this application. As Figure 2 shown, the frequency range covered by the time-frequency signal is [fd, fu], and [fd, fu] is divided into m frequency bands, which are successively the 1st frequency band [fd, f1], the 2nd frequency band [f1, f2], the 3rd frequency band [f2, f3],..., the i-th frequency band [fi-1, fi],..., the (m-1)-th frequency band [fm-2, fm-1], and the m-th frequency band [fm-1, fu].

[0089] S202, obtaining the time range covered by the time-frequency signal, and segmenting the time range to obtain n time periods.

[0090] Among them, the time range covered by the time-frequency signal is [td, tu], and [td, tu] is divided into n frequency bands, which are successively the 1st time period [td, t1], the 2nd time period [t1, t2], the 3rd time period [t2, t3],..., the j-th time period [tj-1, tj],..., the (n-1)-th time period [tn-2, tn-1], and the n-th time period [tn-1, tu].

[0091] S203. Obtain the amplitude of the time-frequency signal in the \(j\)th time period of the \(i\)th frequency band, and calculate the average value of the amplitude of the time-frequency signal in the \(j\)th time period of the \(i\)th frequency band as the pixel value of the \(i\)th row and \(j\)th column; let \(i\) traverse from 1 to \(m\) and \(j\) traverse from 1 to \(n\) to obtain the time-frequency image.

[0092] Among them, sample the amplitude of the time-frequency signal in the \(j\)th time period \([t_{j - 1}, t_j]\) of the \(i\)th frequency band \([f_{i - 1}, f_i]\), and calculate the average value of the amplitude of the sampled time-frequency signal as the pixel value of the \(i\)th row and \(j\)th column. Let \(i\) traverse from 1 to \(m\) and \(j\) traverse from 1 to \(n\) to obtain a time-frequency image with a size of \(m\times n\). \(m\) and \(n\) are positive integers.

[0093] In the above technical solution, by dividing the frequency range of the time-frequency signal to obtain multiple frequency bands, dividing the time range of the time-frequency signal to obtain multiple time periods, using the serial number of the frequency band as the pixel row serial number, using the serial number of the time period as the pixel column serial number, and using the amplitude of the \(i\)th frequency band and \(j\)th time period as the pixel value of the corresponding pixel, so as to retain the time, frequency and amplitude signals in the time-frequency signal when converting the time-frequency signal into a time-frequency image. Furthermore, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0094] Optionally, obtain the frequency range covered by the time-frequency signal and accurately divide it into \(m\) frequency bands. This step ensures the effective retention of the signal characteristics within each frequency band through precise frequency division. The precise division of the frequency range enables the detailed display of the characteristics of the time-frequency signal in the frequency dimension. The change in the signal amplitude within each frequency band reflects the signal strength within that frequency range, providing rich frequency information for the subsequent construction of the time-frequency image.

[0095] Next, obtain the time range covered by the time-frequency signal and divide it into \(n\) time periods. This step enables the clear presentation of the changes in the signal in the time dimension through the division of time periods. The division of the time range enables the accurate capture of the characteristics of the time-frequency signal in the time dimension. The change in the signal amplitude within each time period reflects the signal dynamics within that time period, providing important time information for the construction of the time-frequency image.

[0096] For the intersection point of each frequency band and time period (i.e., the \(i\)th frequency band and \(j\)th time period), calculate the average value of the amplitude of the time-frequency signal in this area and use it as the pixel value of the corresponding pixel in the time-frequency image. This step effectively smooths the noise in the signal through the average calculation of the amplitude, improving the accuracy of the pixel value. The accurate conversion from amplitude to pixel value ensures that each pixel in the time-frequency image accurately reflects the signal strength within the corresponding frequency band and time period. This conversion not only retains the key amplitude information in the signal but also reduces the interference of noise through the average calculation, improving the quality of the time-frequency image.

[0097] By traversing the intersection points of all frequency bands and time periods, calculate and obtain the pixel values of each intersection point, and finally construct a complete time-frequency image. Through the construction of the pixel matrix, the characteristics of the time-frequency signal in the time and space dimensions are intuitively presented in the form of an image. The construction of the pixel matrix enables the comprehensive presentation of the time, frequency, and amplitude information in the time-frequency signal in the time-frequency image. This presentation method not only facilitates subsequent feature extraction and analysis but also provides a reliable data basis for generating point cloud data based on the time-frequency image.

[0098] By converting the time-frequency signal into a time-frequency image, the time, frequency, and amplitude information in the signal is effectively retained. These information play a key role in the subsequent process of generating point cloud data, ensuring the accuracy and integrity of the point cloud data. When performing feature extraction based on the time-frequency image, since the image contains rich time-frequency information, the features characterizing the position and shape of the parts to be assembled can be more accurately extracted. These features provide precise data support for the subsequent generation of point cloud data, further improving the accuracy of the point cloud data.

[0099] In a possible implementation, S103: Obtain multiple time-frequency patches and the positions of each time-frequency patch by partitioning the time-frequency image, specifically including:

[0100] S301: Partition the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency patch array.

[0101] Among them, the time-frequency image is partitioned using p row partition lines and q column partition lines to obtain a p×q time-frequency patch array. The m rows of pixels are divided into 10 regions, and the n columns of pixels are divided into 10 regions, so as to obtain a 10×10 time-frequency patch array.

[0102] Figure 3 Schematic diagram of the partitioning principle of the time-frequency image provided by this application, as Figure 3 shown, the size of each image block is a×b. In the time-frequency patch array, the position of the upper left corner point in the time-frequency patch in the (k + 1)-th row and the (h + 1)-th column is the pixel at the (k×a + 1)-th row and the (h×b + 1)-th column, the position of the upper right corner point in the time-frequency patch is the pixel at the (k×a + 1)-th row and the (h + 1)×b-th column, the position of the lower left corner point in the time-frequency patch is the pixel at the (k + 1)×a-th row and the h×b + 1-th column, and the position of the lower right corner point in the time-frequency patch is the pixel at the (k + 1)×a-th row and the (h + 1)×b-th column.

[0103] S302: Obtain the frequency range corresponding to each time-frequency patch, calculate the average value of the frequency range corresponding to each time-frequency block, and use it as the ordinate corresponding to each time-frequency patch.

[0104] Among them, obtain the frequency ranges corresponding to the pixels in the (k×a + 1)-th row, the frequency ranges corresponding to the pixels in the (k×a + 2)-th row, ……, the frequency ranges corresponding to the pixels in the ((k + 1)×a)-th row, and calculate the average value of the frequency ranges corresponding to the pixels in the (k×a + 1)-th row, the frequency ranges corresponding to the pixels in the (k×a + 2)-th row, ……, the frequency ranges corresponding to the pixels in the ((k + 1)×a)-th row.

[0105] Since the frequency ranges corresponding to the pixels in the (k×a + 1)-th row, the frequency ranges corresponding to the pixels in the (k×a + 2)-th row, ……, the frequency ranges corresponding to the pixels in the ((k + 1)×a)-th row are successively and continuously distributed frequency ranges, calculate the average value of the lower limit value of the frequency range corresponding to the pixels in the (k×a + 1)-th row and the upper limit value of the frequency range corresponding to the pixels in the ((k + 1)×a)-th row, and obtain the ordinate corresponding to the time-frequency patch at the (k + 1)-th row and the (h + 1)-th column.

[0106] S303. Obtain the time range corresponding to each time-frequency patch, and calculate the average value of the time range corresponding to each time-frequency patch as the abscissa corresponding to each time-frequency patch.

[0107] Among them, obtain the time ranges corresponding to the pixels in the (h×b + 1)-th column, the time ranges corresponding to the pixels in the (h×b + 2)-th column, ……, the time ranges corresponding to the pixels in the ((h + 1)×b)-th column, and calculate the average value of the time ranges corresponding to the pixels in the (h×b + 1)-th column, the time ranges corresponding to the pixels in the (h×b + 2)-th column, ……, the time ranges corresponding to the pixels in the ((h + 1)×b)-th column.

[0108] Since the time ranges corresponding to the pixels in the (h×b + 1)-th column, the time ranges corresponding to the pixels in the (h×b + 2)-th column, ……, the time ranges corresponding to the pixels in the ((h + 1)×b)-th column are successively and continuously distributed time ranges, calculate the average value of the lower limit value of the time range corresponding to the pixels in the (h×b + 1)-th column and the upper limit value of the time range corresponding to the pixels in the ((h + 1)×b)-th column, and obtain the ordinate corresponding to the time-frequency patch at the (k + 1)-th row and the (h + 1)-th column.

[0109] In the above technical solution, the time-frequency image is divided into multiple time-frequency patches, the frequency range and time range corresponding to each time-frequency patch are obtained, the frequency corresponding to each time patch is used as the ordinate of the image patch, and the time corresponding to each time patch is used as the abscissa of the image patch. Thus, input features are generated based on the image patches and the positions of the image patches, so that the input features contain amplitude, time, and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0110] Optionally, the time-frequency image is segmented along the pixel row direction and the pixel column direction to obtain a time-frequency tile array. In this step, by setting reasonable segmentation parameters (such as the number of row segments and the number of column segments), the time-frequency image is divided into multiple tiles of uniform or unequal sizes. The image tiling process effectively divides the information in the time-frequency image into multiple local regions, and each region (i.e., tile) contains a part of the time-frequency information of the original image. This segmentation method not only facilitates subsequent feature extraction but also improves the utilization rate of the overall information by retaining local information.

[0111] After obtaining the time-frequency tile array, the frequency range and time range corresponding to each tile are further calculated. Specifically, by calculating the average values of the frequencies and times corresponding to the pixel points within each tile, the vertical coordinate (frequency coordinate) and the horizontal coordinate (time coordinate) of the tile are obtained. The acquisition of the tile position information provides key data for the subsequent generation of input features. By using the frequency and time information as the coordinates of the tile, the input features not only contain amplitude information but also incorporate the dimensional information of time and frequency, thus enriching the connotation and expression ability of the features.

[0112] After obtaining the time-frequency tiles and their position information, a specific processing method (such as convolution processing, transfer function, etc.) is used to extract features from each tile, and the position information (vertical coordinate and horizontal coordinate) of the tile is incorporated as part of the feature. Finally, the features of all tiles are concatenated to generate the complete input features. The feature generation method based on tiles and position information enables the input features to contain rich amplitude, time, and frequency information. This multi-dimensional feature expression not only improves the discriminability and expression ability of the features but also provides more accurate and comprehensive data support for the subsequent encoding and decoding processes.

[0113] Through the tiling process and the incorporation of position information, the expression of the input features becomes more rich and accurate. Such features not only retain the key information in the original time-frequency image but also improve the utilization rate and accuracy of the information through the extraction and integration of local information. In the subsequent encoding and decoding processes, this rich feature expression will contribute to improving the generation quality and accuracy of the point cloud data.

[0114] The block processing of the time-frequency image enables the information in the original image to be utilized more effectively. Through the extraction and integration of local information, the input features contain more comprehensive and accurate time-frequency information. This information is fully utilized in the subsequent encoding and decoding processes, thereby improving the generation quality and accuracy of the point cloud data. The point cloud data generation method based on block processing and feature generation significantly enhances the accuracy of the point cloud data. Since the input features contain rich amplitude, time, and frequency information, the encoding and decoding processes can more accurately restore the geometric shape and position information of the components to be assembled. This improvement in accuracy not only provides reliable data support for subsequent assembly operations but also enhances the efficiency and accuracy of the entire assembly process.

[0115] In a possible implementation, S103. Generate input features according to multiple time-frequency patches and the positions of each time-frequency patch, specifically including:

[0116] S401. Use multiple convolutional kernels to perform multiple convolutional operations on the time-frequency patches to obtain the amplitude features of the time-frequency patches.

[0117] Among them, there are 2c convolutional kernels. Use the first convolutional kernel to perform a convolutional operation on the first time-frequency patch to obtain the first element of the amplitude feature of the first time-frequency patch. Use the second convolutional kernel to perform a convolutional operation on the first time-frequency patch to obtain the second element of the amplitude feature of the first time-frequency patch. …… Use the 2c-th convolutional kernel to perform a convolutional operation on the first time-frequency patch to obtain the 2c-th element of the amplitude feature of the first time-frequency patch.

[0118] Use the first convolutional kernel to perform a convolutional operation on the second time-frequency patch to obtain the first element of the amplitude feature of the second time-frequency patch. Use the second convolutional kernel to perform a convolutional operation on the second time-frequency patch to obtain the second element of the amplitude feature of the second time-frequency patch. …… Use the 2c-th convolutional kernel to perform a convolutional operation on the second time-frequency patch to obtain the 2c-th element of the amplitude feature of the second time-frequency patch.

[0119] And so on, until the processing of the last time-frequency patch is completed to obtain the amplitude features of the last time-frequency patch. The 2c convolutional kernels can be obtained through learning.

[0120] S402. Use a transformation function to transform the position coordinates of the time-frequency patches to obtain the position features of the time-frequency patches.

[0121] Among them, the conversion function includes multiple 2c conversion sub-functions. The first conversion sub-function converts the position coordinates of the first time-frequency map block to obtain the first element of the position feature of the first time-frequency map block. The second conversion sub-function converts the position coordinates of the first time-frequency map block to obtain the second element of the position feature of the first time-frequency map block.... The 2c-th conversion sub-function converts the position coordinates of the first time-frequency map block to obtain the 2c-th element of the position feature of the first time-frequency map block.

[0122] The first conversion sub-function converts the position coordinates of the second time-frequency map block to obtain the first element of the position feature of the second time-frequency map block. The second conversion sub-function converts the position coordinates of the second time-frequency map block to obtain the second element of the position feature of the second time-frequency map block.... The 2c-th conversion sub-function converts the position coordinates of the second time-frequency map block to obtain the 2c-th element of the position feature of the second time-frequency map block.

[0123] And so on, until the position coordinates of the last time-frequency map block are processed to obtain the position feature of the last time-frequency map block. The 2c conversion sub-functions can be obtained through learning.

[0124] S403. Calculate the sum of the amplitude feature of the time-frequency map block and the position feature of the time-frequency map block to obtain the fusion feature of the time-frequency map block.

[0125] Among them, calculate the sum of the amplitude feature and the coordinate feature of the first time-frequency map block to obtain the fusion feature of the first time-frequency map block. Calculate the sum of the amplitude feature and the coordinate feature of the second time-frequency map block to obtain the fusion feature of the second time-frequency map block. And so on, calculate the sum of the amplitude feature and the coordinate feature of the last time-frequency map block to obtain the fusion feature of the last time-frequency map block.

[0126] S404. Concatenate the fusion features of all time-frequency map blocks to generate the input feature.

[0127] Among them, concatenate the fusion feature of the first time-frequency map block, the fusion feature of the second time-frequency map block, the fusion feature of the third time-frequency map block,..., the fusion feature of the last time-frequency map block in order to obtain the input feature.

[0128] In the above technical solution, the convolution kernel is used to process the time-frequency map block to obtain the amplitude feature of the time-frequency map block, the conversion function is used to convert the position coordinates of the time-frequency map block to obtain the position feature of the time-frequency map block, the fusion feature of the time-frequency map block is obtained based on the amplitude feature and the coordinate feature of the time-frequency map block, and the fusion features of each time-frequency map block are concatenated to obtain the input feature, so that the input feature contains amplitude, time and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0129] Optionally, multiple convolutional kernels are used to perform multiple convolutional operations on the time-frequency map blocks to extract the amplitude features of the time-frequency map blocks. As the basic unit of feature extraction, the convolutional kernel performs local feature extraction on the time-frequency map blocks by means of a sliding window. Each convolutional kernel can capture different local patterns in the block. Convolutional processing can automatically extract representative amplitude features from the time-frequency map blocks, and these features reflect the spatial distribution and local variations of the signal intensity in the block. By using a combination of multiple convolutional kernels, more comprehensive and diverse amplitude features can be extracted, providing strong support for subsequent feature fusion and point cloud data generation.

[0130] Since the convolutional kernel can capture the local details and texture information in the block, the extracted amplitude features have high accuracy and robustness. These features are of great significance for characterizing the signal intensity changes in the time-frequency map blocks and provide an accurate data basis for subsequent point cloud data generation.

[0131] A transformation function is used to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks. The transformation function can be designed according to actual needs. For example, sine or cosine functions can be used to perform non-linear transformations on the position coordinates to extract specific patterns of the position information. The position coordinate transformation can extract the relative position information of the time-frequency map blocks in the time-frequency image, and these information are of great significance for characterizing the spatial and temporal relationships between the blocks. Through the non-linear transformation of the transformation function, the position coordinates can be transformed into more expressive feature vectors, providing strong support for subsequent feature fusion.

[0132] Since the position of each time-frequency map block in the time-frequency image is unique, the extracted position features are also unique. These features can distinguish different blocks and provide spatial position information for subsequent point cloud data generation, helping to improve the accuracy and integrity of the point cloud data.

[0133] The sum of the amplitude features and position features of the time-frequency map blocks is calculated to obtain the fusion features of the time-frequency map blocks. This step fuses the two features through a simple addition operation, generating a comprehensive feature that contains both amplitude information and position information. The feature fusion strategy can effectively combine the amplitude features and position features to generate fusion features with richer information. These features not only contain the signal intensity information in the time-frequency map blocks but also the position information of the blocks in the time-frequency image, providing comprehensive data support for subsequent point cloud data generation.

[0134] Concatenate the fusion features of all time-frequency patches to generate a complete input feature. This step combines the fusion features of multiple patches through a simple concatenation operation to form an input feature vector that contains information from all patches. The generation process of the input feature ensures that the information of all time-frequency patches is effectively integrated, forming a comprehensive and accurate feature vector. These feature vectors not only contain rich amplitude, time, and frequency information but also reflect the spatial and temporal relationships between patches, providing a reliable data basis for subsequent point cloud data generation.

[0135] Since the input feature contains rich amplitude, time, and frequency information, as well as the spatial and temporal relationships between patches, the point cloud data generated based on these features will have higher accuracy and integrity. This information provides more precise data support for subsequent assembly operations, helping to improve the efficiency and accuracy of assembly. By optimizing the generation process of the input feature, the quality of the point cloud data can be further improved. For example, by adjusting the size and number of convolutional kernels, the type and parameters of the transfer function, etc., more accurate and comprehensive feature information can be extracted, and then higher-quality point cloud data can be generated. This optimization process not only improves the accuracy of the point cloud data but also enhances its adaptability and robustness in complex environments.

[0136] In a possible implementation, in S402, use a transfer function to transform the position coordinates of the time-frequency patch to obtain the position feature of the time-frequency patch, which specifically includes:

[0137] In S501, use the abscissa transfer function to transform the abscissa of the time-frequency patch to obtain the abscissa feature of the time-frequency patch.

[0138] Among them, the transformation coordinates include an abscissa transfer function and an ordinate transfer function.

[0139] Taking any one time-frequency patch as an example. The abscissa transfer function includes c abscissa transfer sub-functions. The first abscissa transfer sub-function processes the abscissa of the time-frequency patch to obtain the first element of the abscissa feature of the time-frequency patch. The second abscissa transfer sub-function processes the abscissa of the time-frequency patch to obtain the second element of the abscissa feature of the time-frequency patch. And so on, the c-th abscissa transfer sub-function processes the abscissa of the time-frequency patch to obtain the c-th element of the abscissa feature of the time-frequency patch.

[0140] In S502, use the ordinate transfer function to transform the ordinate of the time-frequency patch to obtain the ordinate feature of the time-frequency patch.

[0141] Among them, the vertical coordinate conversion function includes c vertical coordinate conversion sub-functions. The first vertical coordinate conversion sub-function processes the vertical coordinate of the time-frequency map block to obtain the first element of the vertical coordinate feature of the time-frequency map block. The second vertical coordinate conversion sub-function processes the vertical coordinate of the time-frequency map block to obtain the second element of the vertical coordinate feature of the time-frequency map block. By analogy, the c-th vertical coordinate conversion sub-function processes the vertical coordinate of the time-frequency map block to obtain the c-th element of the vertical coordinate feature of the time-frequency map block.

[0142] The horizontal coordinate conversion sub-function and the vertical coordinate conversion sub-function use existing sine functions or cosine functions.

[0143] S503. Concatenate the horizontal coordinate feature and the vertical coordinate feature of the time-frequency map block to obtain the position feature of the time-frequency map block.

[0144] Among them, the size of the horizontal coordinate feature of the time-frequency map block is 1×c, the size of the vertical coordinate feature of the time-frequency map block is 1×c, and the horizontal coordinate feature and the vertical coordinate feature of the time-frequency map block are concatenated to obtain the position feature of the time-frequency map block. At this time, the size of the position feature of the time-frequency map block is 1×2c.

[0145] In the above technical solution, the horizontal coordinate of the time-frequency map block is converted using the horizontal coordinate conversion function to obtain the horizontal coordinate feature of the time-frequency map block, the vertical coordinate of the time-frequency map block is converted using the vertical coordinate conversion function to obtain the vertical coordinate feature of the time-frequency map block, and the horizontal coordinate feature and the vertical coordinate feature of the time-frequency map block are concatenated. In this way, the position feature of the generated time-frequency map block contains time and frequency information.

[0146] Optionally, the horizontal coordinate (i.e., the time coordinate) of the time-frequency map block is converted using the horizontal coordinate conversion function to obtain the horizontal coordinate feature of the time-frequency map block. The horizontal coordinate conversion function can be designed according to actual needs. For example, a sine function, a cosine function, or other non-linear functions are used to perform a non-linear transformation on the horizontal coordinate, so as to extract specific patterns or information in the horizontal coordinate. Through the processing of the horizontal coordinate conversion function, the horizontal coordinate of the time-frequency map block is converted into a more expressive feature vector. This conversion process not only retains the original information of the horizontal coordinate, but also introduces new feature dimensions through non-linear transformation, making the horizontal coordinate feature more discriminative and expressive in the subsequent feature fusion and point cloud data generation processes.

[0147] Similar to the abscissa conversion function, the ordinate conversion function is used to convert the ordinate (i.e., the frequency coordinate) of the time-frequency map block to obtain the ordinate feature of the time-frequency map block. The ordinate conversion function can also be designed according to actual needs to perform a non-linear transformation on the ordinate and extract specific information in the ordinate. The processing of the ordinate conversion function converts the ordinate of the time-frequency map block into a feature vector with rich information. This conversion process not only retains the original information of the ordinate but also introduces new feature dimensions through non-linear transformation, enhancing the expression ability of the ordinate feature in subsequent processing.

[0148] After obtaining the abscissa feature and ordinate feature of the time-frequency map block, the two are concatenated to form a complete position feature. This concatenation process can be achieved through a simple vector concatenation operation, combining the abscissa feature and ordinate feature in a certain order to form a comprehensive feature vector containing time and frequency information. The feature concatenation strategy enables the position feature of the time-frequency map block to contain both time and frequency information simultaneously. This multi-dimensional feature expression not only improves the discrimination and expression ability of the features but also provides more accurate and comprehensive data support for the subsequent encoding and decoding processes. By concatenating the abscissa feature and ordinate feature, the generated position feature can more comprehensively reflect the position information of the time-frequency map block in the time-frequency image, providing a key data basis for the subsequent generation of point cloud data.

[0149] Through the processing of the above conversion function and concatenation strategy, the generated position feature contains rich time and frequency information. These information not only reflect the specific position of the time-frequency map block in the time-frequency image but also introduce new feature dimensions through non-linear transformation, making the position feature more expressive and discriminative in subsequent processing. In the process of generating point cloud data based on the time-frequency image, this position feature that combines time and frequency information can more accurately characterize the geometric shape and position relationship of the parts to be assembled, thereby improving the accuracy and integrity of the point cloud data.

[0150] Since the position feature contains rich time and frequency information, the point cloud data generated based on these features will have higher accuracy and robustness. These information provide more precise data support for subsequent assembly operations, helping to improve the efficiency and accuracy of assembly. At the same time, by optimizing the conversion function and concatenation strategy, more expressive position features can be further extracted, thereby further improving the quality and accuracy of the point cloud data.

[0151] In a possible implementation, as Figure 4 shown, S104, encoding the input feature using an encoder to obtain a low-dimensional point cloud feature, specifically including:

[0152] Encode the input features using the encoding module of the first level to obtain the first intermediate encoded features, process the first intermediate encoded features using the encoding module of the second level to obtain the second intermediate encoded features, process the second intermediate encoded features using the encoding module of the third level to output the third intermediate encoded features, and process the third intermediate encoded features using the encoding module of the fourth level to output the low-dimensional point cloud features.

[0153] Among them, the encoder includes four encoding modules, and each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. In each encoding module, the multi-head attention mechanism layer processes the input features of the encoding module, and the feed-forward neural network layer processes the data output by the multi-head attention mechanism layer. The multi-head attention mechanism layer and the feed-forward neural network layer use existing network structures and will not be elaborated here.

[0154] There is a residual normalization layer after the multi-head attention mechanism layer. The features processed by the multi-head attention mechanism layer are processed by the residual normalization layer and then input to the feed-forward neural network layer. There is a residual normalization layer after the feed-forward neural network layer. The features processed by the feed-forward neural network layer are output to the residual normalization layer for processing to output the encoded features of the encoded features.

[0155] The input features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network layer, and the second residual normalization layer in the encoding module of the first level in sequence, and then the first intermediate encoded features are output. The second intermediate encoded features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network, and the second residual normalization layer in the encoding module of the second level in sequence, and then the second intermediate encoding layer is output. The second intermediate encoded features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network, and the second residual normalization layer in the encoding module of the third level in sequence, and then the third intermediate encoding layer is output. The third intermediate encoded features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network, and the second residual normalization layer in the encoding module of the fourth level in sequence, and then the low-dimensional point cloud features are output.

[0156] In the above technical solution, the encoder includes multiple encoding modules, and each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. The input features are processed by multiple encoding modules for information extraction to obtain low-dimensional point cloud features, and the distance and angle information are extracted from the amplitude, frequency, and time information in the time-frequency image. In this way, the point cloud data can be restored based on the low-dimensional point cloud features.

[0157] Optionally, the encoder consists of multiple cascaded encoding modules, and each encoding module is responsible for processing the input features step by step. This cascaded structure enables the encoder to gradually extract the deep-level information in the input features, laying the foundation for generating low-dimensional point cloud features. Through the design of multi-level encoding modules, the encoder can capture the complex patterns and structures in the input features more comprehensively. Each level of encoding module can further extract features based on the previous level, thereby generating more expressive intermediate encoded features.

[0158] Each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. The multi-head attention mechanism layer is responsible for performing attention-weighted processing on the input features to highlight the key information; the feed-forward neural network layer then performs a non-linear transformation on the output data of the attention mechanism layer to extract higher-level features. By simulating the attention mechanism of the human brain, the multi-head attention mechanism layer can dynamically adjust the weights between different features, enabling the key information to receive more attention in subsequent processing. The feed-forward neural network layer introduces new feature dimensions through non-linear transformation, enhancing the expressive ability of the features. The combination of the two enables the encoding module to more effectively extract the useful information in the input features.

[0159] The first-level encoding module performs preliminary processing on the input features to generate the first intermediate encoded features. In this process, the multi-head attention mechanism layer first identifies the key information in the input features and performs a preliminary non-linear transformation through the feed-forward neural network layer. The processing of the first-level encoding module effectively extracts and transforms the preliminary information in the input features, laying the foundation for subsequent processing.

[0160] Subsequently, the second-level, third-level, and fourth-level encoding modules sequentially process the intermediate encoded features of the previous level, gradually extracting deeper-level information. Each level of encoding module can further mine the useful information in the features based on the previous level and generate more expressive intermediate encoded features. Through the progressive processing of multi-level encoding modules, the encoder can gradually extract the complex patterns and structures in the input features and finally generate low-dimensional point cloud features containing rich information. This process not only improves the expressive ability of the features but also provides a reliable data basis for subsequent point cloud data restoration.

[0161] During the encoding process, the encoder extracts distance and angle information from the amplitude, frequency, and time information in the time-frequency image through the collaborative action of the multi-head attention mechanism layer and the feed-forward neural network layer. These information are gradually fused and transformed during the encoding process and finally generate low-dimensional point cloud features. Through this process, the encoder realizes the transformation from the time-frequency image to low-dimensional point cloud features. The low-dimensional point cloud features not only contain the key information in the input features but also introduce new feature dimensions and expression methods through the processing of the encoder, providing more accurate and comprehensive data support for subsequent point cloud data restoration.

[0162] Based on the generated low-dimensional point cloud features, the point cloud data can be further restored. Since the low-dimensional point cloud features contain rich distance and angle information, the restored point cloud data will have higher accuracy and integrity. This provides more accurate data support for subsequent assembly operations, helping to improve the efficiency and accuracy of assembly.

[0163] In a possible implementation, continue to refer to Figure 4 , S104. Use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled, specifically including:

[0164] S601. Use the first multi-layer perceptron layer to decode the low-dimensional point cloud features to obtain the first decoded data. The data dimension of the low-dimensional point cloud features is smaller than the data dimension of the first decoded data.

[0165] Among them, the first multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the data dimension of the first decoded data output by the output layer is made larger than the data dimension of the low-dimensional point cloud features. By expanding the low-dimensional point cloud features, more position information is included in the first decoded data. For example: the data dimension of the low-dimensional point cloud features is n×1. The data dimension of the second decoded data is n×256.

[0166] S602. Use the second multi-layer perceptron layer to decode the first decoded data to obtain the second decoded data. The data dimension of the second decoded data is twice the data dimension of the first decoded data.

[0167] Among them, the second multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the data dimension of the second decoded data output by the output layer is made twice the data dimension of the first decoded feature. By expanding the second decoded feature, more position information is included in the second decoded data. The data dimension of the second decoded data is n×512.

[0168] S603. Use the third multi-layer perceptron layer to decode the second decoded data to obtain the third decoded data. The data dimension of the second decoded data is larger than the data dimension of the third decoded data.

[0169] Among them, the third multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the data dimension of the third decoded data output by the output layer is made smaller than the data dimension of the second decoded feature, thus realizing the extraction of position information from the second decrypted feature.

[0170] S604. Use the fourth multi-layer perceptron layer to perform decoding processing on the third decoded data to obtain point cloud data.

[0171] Among them, the fourth multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the output data dimension of the output layer is n×3, where each row represents the three-dimensional coordinate information of a point, and n represents the number of points in the point cloud.

[0172] In the above technical solution, the first multi-layer perceptron layer and the second multi-layer perceptron layer are used to increase the dimension of the low-dimensional point cloud features, so that the features output by the first multi-layer perceptron layer and the features output by the second multi-layer perceptron layer contain more position information. The third multi-layer perceptron layer and the fourth multi-layer perceptron layer are used to reduce the dimension of the data, and three-dimensional position information is generated based on multi-dimensional position information to realize the restoration of the point cloud data.

[0173] Optionally, use the first multi-layer perceptron layer to perform decoding processing on the low-dimensional point cloud features to obtain the first decoded data. In this step, the data dimension of the low-dimensional features is expanded, providing a richer information basis for subsequent processing.

[0174] Immediately afterwards, the second multi-layer perceptron layer decodes the first decoded data and outputs the second decoded data. It should be noted that the data dimension of the second decoded data is twice that of the first decoded data, and this significant dimension increase helps to capture more position information and detailed features.

[0175] Subsequently, the third multi-layer perceptron layer processes the second decoded data to generate the third decoded data. At this stage, the data dimension decreases, but this is not a simple loss of information. Instead, through a carefully designed network structure, the key information crucial for three-dimensional reconstruction is refined from the high-dimensional features.

[0176] Finally, the fourth multi-layer perceptron layer performs decoding processing on the third decoded data and directly outputs the point cloud data of the component to be assembled. This step realizes the accurate mapping from the high-dimensional feature space to the three-dimensional physical space.

[0177] Through the continuous dimensionality increase operations of the first and second multi-layer perceptron layers, the low-dimensional point cloud features are effectively extended to a higher-dimensional space. During this process, the expressive power of the features is significantly enhanced, enabling them to contain more implicit information about the position, shape, and structural details of the point cloud, laying a solid foundation for subsequent point cloud reconstruction. The dimensionality reduction process of the third multi-layer perceptron layer is a process of selective information extraction, which can accurately screen out a feature subset crucial for three-dimensional position information reconstruction from the highly redundant high-dimensional features. This process effectively avoids information overload while ensuring the accuracy and integrity of the reconstructed point cloud data. The fourth multi-layer perceptron layer, as the final decoding step, directly maps the optimized features into three-dimensional space to generate high-precision point cloud data. Due to the flexible transformation of feature dimensions and the efficient integration of information in the previous steps, the generated point cloud data reaches a high level in terms of position accuracy, shape fidelity, and detail expressiveness. Through the designed multi-layer perceptron hierarchical structure and dimensionality transformation strategy, this technical solution not only performs well in processing specific low-dimensional point cloud features but also has a certain generalization ability, capable of adapting to the processing requirements of point cloud data with different complexities and diversities, enhancing the overall robustness of the system.

[0178] Some embodiments of the present application provide a method for processing electromagnetic signal image data based on a neural network. The method specifically includes the following steps:

[0179] S701. The electronic device scans the to-be-assembled component using a detection radar according to a preset scanning path, and obtains the echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the to-be-assembled component.

[0180] S702. Mix the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate-frequency signal, perform wavelet transform on the intermediate-frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image.

[0181] S703. Divide the time-frequency image into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and generate input features according to the multiple time-frequency blocks and the positions of each time-frequency block.

[0182] S704. Use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the to-be-assembled component.

[0183] S705. Obtain the three-dimensional model data of the to-be-assembled component in the assembly posture, and obtain the deflection angle of the to-be-assembled component according to the three-dimensional model data of the to-be-assembled component in the assembly posture and the point cloud data of the to-be-assembled component.

[0184] Wherein, to obtain the three-dimensional model data of the to-be-assembled component, determine the three-dimensional model data of the to-be-assembled component in the assembly posture according to the installation position of the assembly tooling on the assembly robot.

[0185] Two positioning structures are designed on the component to be assembled. The angle of the straight line where the two positioning structures are located in the assembly posture is calculated based on the three-dimensional model data of the component to be assembled in the assembly posture. More specifically, the theoretical positions of the first positioning structure and the second positioning structure are obtained according to the three-dimensional model data of the component to be assembled in the assembly posture, and the angle of the straight line where the two positioning structures are located in the assembly posture is calculated based on the theoretical positions of the first positioning structure and the second positioning structure.

[0186] The angle of the straight line where the two positioning structures are located in the actual posture is calculated based on the point cloud data of the component to be assembled. More specifically, the actual positions of the first positioning structure and the second positioning structure are obtained according to the point cloud data of the component to be assembled, and the angle of the straight line where the two positioning structures are located in the actual posture is calculated based on the actual positions of the first positioning structure and the second positioning structure.

[0187] Calculate the difference between the angle of the straight line where the positioning structure is located in the assembly posture and the angle of the straight line where the positioning structure is located in the actual posture, and use the difference as the deflection angle of the component to be assembled.

[0188] In the above technical solution, the angle of the straight line where the two positioning structures are located in the assembly posture is calculated based on the three-dimensional model data of the component to be assembled in the assembly posture, the angle of the straight line where the two positioning structures are located in the actual posture is calculated based on the point cloud data of the component to be assembled, and the difference between the two calculated angles is used as the deflection angle of the component to be assembled, so as to realize the determination of the deflection angle of the component to be assembled based on the positioning structure.

[0189] Optionally, by using the three-dimensional model data and the point cloud data, the technical solution can accurately calculate the straight line angles of the positioning structure in the assembly posture and the actual posture. This calculation method based on geometric features avoids the deviation of the deflection angle calculation caused by measurement errors or inaccurate data in the traditional method. Using two positioning structures for calculation can further improve the accuracy of the deflection angle. By comparing the angle differences of the two positioning structures in the assembly posture and the actual posture, the influence caused by the error of a single positioning structure can be effectively offset.

[0190] The accurate calculation of the deflection angle provides accurate direction guidance for the operation of the assembly robot. The robot can accurately adjust the clamping angle and grasping path of the manipulator according to the calculated deflection angle, ensure that the component will not be damaged during the grasping process, and maintain the grasping stability. By rotating the component to the correct assembly posture and then moving it, the assembly robot can avoid collisions or damages caused by incorrect component postures during the movement, thus improving the overall success rate of the assembly process.

[0191] Due to the adoption of an accurate deflection angle calculation method, the assembly robot does not need to make repeated adjustments and trial - and - error when grasping and moving the parts to be assembled. This greatly shortens the assembly cycle and improves the overall assembly efficiency. In addition, this technical solution is also applicable to various types and sizes of parts to be assembled. By adjusting the processing parameters of the 3D model data and point cloud data, it can quickly adapt to the assembly requirements of different products, further enhancing the flexibility and adaptability of the production line.

[0192] In the traditional assembly process, a large amount of manual participation is required to adjust the posture and position of the parts. This technical solution effectively reduces the dependence on manual intervention through automated calculation and precise robot operation. This not only reduces the labor cost but also improves the consistency and stability of the assembly process.

[0193] S706. Control the manipulator of the assembly robot to grasp the part to be assembled according to the deflection angle of the part to be assembled, rotate it to the assembly posture, and move the part to be assembled to the assembly fixture.

[0194] Among them, after obtaining the deflection angle of the part to be assembled, control the manipulator on the assembly robot according to the deflection angle of the part to be assembled, so that the manipulator can grip the part on the clamping surface of the part to be assembled. After the manipulator grips the part, make the manipulator return to the device configuration and move the part to be assembled to the assembly fixture.

[0195] In the above - mentioned technical solution, after obtaining the point cloud data of the part to be assembled, obtain the deflection angle of the part to be assembled according to the 3D model data of the part to be assembled in the assembly posture and the point cloud data of the part to be assembled. The assembly robot controls the manipulator to grip the part to be assembled according to the deflection angle, and returns to the default assembly posture after gripping the part to be assembled, so as to move the part to be assembled to the assembly fixture. The robot does not need to repeatedly adjust the relative relationship between the part to be assembled and the assembly fixture, improving the assembly efficiency of the assembly robot.

[0196] Optionally, obtaining high - precision point cloud data by decoding low - dimensional point cloud features provides an accurate data basis for subsequent deflection angle calculation. By using the comparison and analysis of 3D model data and point cloud data, the deflection angle of the part to be assembled is accurately calculated, avoiding the assembly accuracy problems caused by measurement errors in traditional methods. The assembly robot directly grasps and rotates the part to the assembly posture according to the deflection angle, without repeatedly adjusting the relative relationship between the part and the fixture, significantly improving the assembly efficiency. This technical solution organically combines point cloud data processing, 3D model comparison and analysis, and assembly robot control technology, realizing the intelligentization and automation of the assembly process. The assembly robot can independently decide the grasping path and assembly posture according to the real - time obtained point cloud data and 3D model data, enhancing the autonomous operation ability of the robot.

[0197] A large amount of manual participation is required in the traditional assembly process, which not only has a high labor intensity but also poses safety risks. This technical solution replaces manual operations with automated assembly robots, effectively reducing labor costs and safety risks. The assembly robot can perform precise operations according to a predetermined program, avoiding assembly errors and safety accidents caused by human factors. This technical solution is applicable to various types and sizes of components to be assembled, with strong flexibility and adaptability. By adjusting the decoder parameters and 3D model data, it can quickly adapt to the assembly requirements of different products and achieve rapid switching and adjustment of the production line. Through precise calculation of the deflection angle and automated assembly process, the errors and defective product rate in the assembly process are effectively reduced. The assembly robot can perform standardized operations according to a predetermined program, ensuring the consistency and stability of the assembly process, thereby improving the overall quality of the product.

[0198] Some embodiments of this application provide a method for processing electromagnetic signal image data based on a neural network. The method specifically includes the following steps:

[0199] S801. The electronic device uses a detection radar to scan the component to be assembled according to a preset scanning path, and obtains the echo electromagnetic signal returned after the scanning electromagnetic signal emitted by the detection radar hits the component to be assembled.

[0200] S802. Mix the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate-frequency signal, perform wavelet transform on the intermediate-frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image.

[0201] S803. Divide the time-frequency image into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and generate input features according to the multiple time-frequency blocks and the positions of each time-frequency block.

[0202] S804. Use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled.

[0203] S805. Obtain the 3D model data of the component to be assembled in the assembly posture, and obtain the deflection angle of the component to be assembled according to the 3D model data of the component to be assembled in the assembly posture and the point cloud data of the component to be assembled.

[0204] S806. Convert the point cloud data of the assembled component into the point cloud data in the assembly posture according to the deflection angle of the component to be assembled.

[0205] Among them, a rotation matrix is generated according to the deflection angle of the component to be assembled, and the inner product of the point cloud data of the component to be assembled and the rotation matrix is calculated to obtain the point cloud data in the assembly posture.

[0206] S807. Extract the theoretical data of each surface of the to-be-assembled part from the 3D model data of the to-be-assembled part in the assembly posture.

[0207] Among them, the to-be-assembled part has six surfaces, and the theoretical data of the six surfaces are extracted from the 3D model data of the to-be-assembled part in the assembly posture.

[0208] S808. Extract the actual data of each surface of the to-be-assembled part from the point cloud data of the to-be-assembled part in the assembly posture.

[0209] Among them, the actual data of the six surfaces are extracted from the point cloud data of the to-be-assembled part in the assembly posture.

[0210] S809. Perform homonymous point matching on the theoretical data of each surface of the to-be-assembled part and the actual data of each surface, and calculate the distance between each pair of homonymous points.

[0211] Among them, use the existing homonymous point matching algorithm to perform homonymous point matching on the theoretical data of each surface of the to-be-assembled part and the actual data of each surface, and calculate the distance between each pair of homonymous points according to the positions of the homonymous points.

[0212] S810. Calculate the variance of the distances between each pair of homonymous points. If the variance is greater than the preset threshold, determine that the to-be-assembled part has been knocked or bumped.

[0213] Among them, if the to-be-assembled part has been knocked or bumped, the surface similarity of the to-be-assembled part decreases, and the distances between each pair of homonymous points are relatively far apart, that is, the variance of the distances between each pair of homonymous points is relatively large. If the to-be-assembled part has not been knocked or bumped, the surface similarity of the to-be-assembled part is relatively high, and the distances between each pair of homonymous points are relatively close, that is, the variance of the distances between each pair of homonymous points is relatively small. Determine whether the to-be-assembled part has been knocked or bumped based on the variance of the distances between each pair of homonymous points.

[0214] In the above technical solution, after obtaining the point cloud data of the to-be-assembled part, convert the point cloud data of the to-be-assembled part to the point cloud data in the assembly posture according to the deflection angle of the to-be-assembled part. Extract the theoretical data of each surface of the to-be-assembled part from the 3D model data of the to-be-assembled part in the assembly posture, and extract the actual data of each surface of the to-be-assembled part from the point cloud data of the to-be-assembled part in the assembly posture. In this way, the theoretical data and the actual data of the same surface can be subjected to homonymous point matching, calculate the variance of the distances between each pair of homonymous points, determine whether the to-be-assembled part has been knocked or bumped according to the variance, reuse the point cloud data for appearance inspection, reduce the equipment cost, and thus realize appearance inspection again before assembly and improve the assembly efficiency.

[0215] Optionally, by performing an appearance inspection before assembly, components with bump problems can be promptly detected and processed, avoiding assembly failures or rework caused by component quality issues during the assembly process. This significantly improves the assembly efficiency and reduces the production cost. Using point cloud data for appearance inspection eliminates the need for additional detection equipment or complex detection processes, enabling fast and efficient detection.

[0216] Then, the point cloud data is reused for appearance inspection without the need for additional detection equipment or sensors. This reduces the equipment procurement cost and maintenance cost, improving the overall economic efficiency of the production line. By calculating the deflection angle and converting the point cloud data, the accuracy and reliability of the appearance inspection are ensured, avoiding misjudgments or omissions caused by insufficient equipment accuracy. By converting the point cloud data of the component to be assembled to the assembly posture, it is ensured that the theoretical data and the actual data are matched and compared in the same posture, improving the accuracy of the appearance inspection. Using homologous point matching and distance variance analysis, the difference between the theoretical data and the actual data can be accurately quantified, and it can be accurately determined whether the component has been bumped.

[0217] This technical solution organically combines technologies such as point cloud data processing, deflection angle calculation, surface data extraction and matching, and distance variance analysis, realizing automated appearance inspection before assembly. By integrating with the control system of the assembly robot, a fully automated process from component grasping, deflection angle calculation, appearance inspection to assembly can be achieved, improving the intelligent level and automation degree of the production line. By performing strict appearance inspection before assembly, it is ensured that the components assembled onto the product meet the quality requirements, improving the overall quality and reliability of the product.

[0218] The electronic device provided in this embodiment includes: at least one processor and a memory. Optionally, the device further includes a communication component. Among them, the processor, the memory, and the communication component are connected through a bus.

[0219] In the specific implementation process, at least one processor executes the computer execution instructions stored in the memory, causing at least one processor to execute the above method.

[0220] For the specific implementation process of the processor, reference can be made to the above method embodiment, and its implementation principle and technical effects are similar, so they will not be elaborated here in this embodiment.

[0221] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by the execution of the hardware processor, or by the combination of hardware and software modules in the processor.

[0222] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0223] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0224] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0225] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above method is implemented.

[0226] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disc. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0227] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0228] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed between each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0229] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0230] Furthermore, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0231] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0232] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0233] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. An electromagnetic signal data processing method based on a neural network, characterized in that The method is applied to an assembly robot, and a detection radar is provided on the assembly robot. The method includes: Scanning the to-be-assembled component using the detection radar according to a preset scanning path, and obtaining an echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the to-be-assembled component; Performing mixing processing on the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate-frequency signal, performing wavelet transform on the intermediate-frequency signal to obtain a time-frequency signal, and converting the time-frequency signal into a time-frequency image; Dividing the time-frequency image into blocks to obtain a plurality of time-frequency blocks and the positions of each time-frequency block, and generating input features according to the plurality of time-frequency blocks and the positions of each time-frequency block; Encoding the input features using an encoder to obtain low-dimensional point cloud features; wherein, the low-dimensional point cloud features are used to characterize position information and angle information; Decoding the low-dimensional point cloud features using a decoder to obtain the point cloud data of the to-be-assembled component; Dividing the time-frequency image into blocks to obtain a plurality of time-frequency blocks and the positions of each time-frequency block, specifically including: Dividing the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency block array; Obtaining the frequency range corresponding to each time-frequency block, calculating the average value of the frequency range corresponding to each time-frequency block, and using it as the ordinate corresponding to each time-frequency block; Obtaining the time range corresponding to each time-frequency block, calculating the average value of the time range corresponding to each time-frequency block, and using it as the abscissa corresponding to each time-frequency block; Generating input features according to the plurality of time-frequency blocks and the positions of each time-frequency block, specifically including: Performing multiple convolution processes on the time-frequency blocks using a plurality of convolution kernels to obtain the amplitude features of the time-frequency blocks; Converting the position coordinates of the time-frequency blocks using a conversion function to obtain the position features of the time-frequency blocks; Calculating the sum of the amplitude features of the time-frequency blocks and the position features of the time-frequency blocks to obtain the fusion features of the time-frequency blocks; Stitching the fusion features of all time-frequency blocks to generate the input features.

2. The electromagnetic signal data processing method according to claim 1, wherein Converting the time-frequency signal into a time-frequency image, specifically including: Obtaining the frequency range covered by the time-frequency signal, and segmenting the frequency range to obtain m frequency segments; Obtaining the time range covered by the time-frequency signal, and segmenting the time range to obtain n time periods; Obtaining the amplitude of the time-frequency signal in the i-th frequency segment and the j-th time period, calculating the average value of the amplitude of the time-frequency signal in the i-th frequency segment and the j-th time period, and using it as the pixel value of the i-th row and the j-th column; let i traverse from 1 to m, and j traverse from 1 to n to obtain the time-frequency image.

3. The electromagnetic signal data processing method according to claim 1, wherein Converting the position coordinates of the time-frequency blocks using a conversion function to obtain the position features of the time-frequency blocks, specifically including: Converting the abscissa of the time-frequency block using an abscissa conversion function to obtain the abscissa feature of the time-frequency block; Converting the ordinate of the time-frequency block using an ordinate conversion function to obtain the ordinate feature of the time-frequency block; Stitching the abscissa feature and the ordinate feature of the time-frequency block to obtain the position feature of the time-frequency block.

4. The electromagnetic signal data processing method according to claim 3, wherein Encoding the input features using an encoder to obtain low-dimensional point cloud features specifically includes: Encoding the input features using the encoding module of the first level to obtain first intermediate encoded features, processing the first intermediate encoded features using the encoding module of the second level to obtain second intermediate encoded features, processing the second intermediate encoded features using the encoding module of the third level to output third intermediate encoded features, and processing the third intermediate encoded features using the encoding module of the fourth level to output the low-dimensional point cloud features; Each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. In each encoding module, the multi-head attention mechanism layer processes the input features of the encoding module, and the feed-forward neural network layer processes the data output by the multi-head attention mechanism layer.

5. The electromagnetic signal data processing method according to any one of claims 1 to 4, characterized in that Using a decoder to perform decoding processing on the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled, specifically including: Performing decoding processing on the low-dimensional point cloud features using the first multi-layer perceptron layer to obtain first decoded data; the data dimension of the low-dimensional point cloud features is smaller than the data dimension of the first decoded data; Performing decoding processing on the first decoded data using the second multi-layer perceptron layer to obtain second decoded data; the data dimension of the second decoded data is twice the data dimension of the first decoded data; Performing decoding processing on the second decoded data using the third multi-layer perceptron layer to obtain third decoded data; the data dimension of the second decoded data is larger than the data dimension of the third decoded data; Performing decoding processing on the third decoded data using the fourth multi-layer perceptron layer to obtain the point cloud data.

6. An assembly robot, characterized in that, A detection radar is provided on the assembly robot; Scanning the component to be assembled using the detection radar according to a preset scanning path to obtain the echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the component to be assembled; Performing mixing processing on the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, performing wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and converting the time-frequency signal into a time-frequency image; Partitioning the time-frequency image to obtain a plurality of time-frequency patches and the positions of each time-frequency patch, and generating input features based on the plurality of time-frequency patches and the positions of each time-frequency patch; Encoding the input features using an encoder to obtain low-dimensional point cloud features; wherein, the low-dimensional point cloud features are used to represent position information and angle information; Using a decoder to perform decoding processing on the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled; Partitioning the time-frequency image to obtain a plurality of time-frequency patches and the positions of each time-frequency patch, specifically including: Dividing the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency patch array; Obtaining the frequency range corresponding to each time-frequency patch, calculating the average value of the frequency range corresponding to each time-frequency patch as the ordinate corresponding to each time-frequency patch; Obtaining the time range corresponding to each time-frequency patch, calculating the average value of the time range corresponding to each time-frequency patch as the abscissa corresponding to each time-frequency patch; Generate input features based on the multiple time-frequency map blocks and the positions of the respective time-frequency map blocks, specifically including: Perform multiple convolutional processes on the time-frequency map blocks using multiple convolutional kernels to obtain the amplitude features of the time-frequency map blocks; Use a conversion function to convert the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks; Calculate the sum of the amplitude features of the time-frequency map blocks and the position features of the time-frequency map blocks to obtain the fusion features of the time-frequency map blocks; Stitch together the fusion features of all the time-frequency map blocks to generate the input features.

7. An electronic device, characterized in that, Comprising: A processor; And, A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the method according to any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 5.

Images