Electromagnetic signal image data processing method based on neural network

Abstract

The invention provides an electromagnetic signal image data processing method based on a neural network, and the method comprises the steps: scanning a to-be-assembled part through a detection radar according to a preset scanning path, and obtaining an echo electromagnetic signal returned when a scanning electromagnetic signal transmitted by the detection radar encounters the to-be-assembled part; performing frequency mixing processing on the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate-frequency signal, performing wavelet transformation on the intermediate-frequency signal to obtain a time-frequency signal, converting the time-frequency signal into a time-frequency image, partitioning the time-frequency image to obtain a plurality of time-frequency image blocks and the position of each time-frequency image block, generating an input feature according to the plurality of time-frequency blocks and the position of each time-frequency block, encoding the input feature by using an encoder to obtain a low-dimensional point cloud feature, decoding the low-dimensional point cloud feature by using a decoder to obtain point cloud data of the to-be-assembled part, and controlling the scale of the point cloud data output by the encoder to obtain the to-be-assembled part. And therefore, the point cloud data with higher intensity can be output.

Description

Technical Field

[0001] The present application relates to the field of assembly robots, and in particular to a method for processing electromagnetic signal image data based on a neural network. Background Art

[0002] With the development of technology, assembly robots are widely used in industrial production. A detection radar is mounted on the assembly robot to detect the pose of the component to be assembled, and then assembly is performed based on the pose of the component.

[0003] The detection radar scans the component to be assembled to obtain the point cloud of the assembly component. More specifically, the detection radar continuously emits signals externally and receives the echo electromagnetic signals reflected by the component to be assembled. Based on the intensity and phase of the echo electromagnetic signals, a point position of the component to be assembled is obtained. By emitting signals at different angles, scanning of one surface of the component to be assembled is achieved. After a scanning cycle, the obtained multiple echo electromagnetic signals are filtered to obtain each echo electromagnetic signal, and then each echo electromagnetic signal is processed to obtain a coordinate data. By traversing each echo electromagnetic signal, the point cloud of one surface is obtained.

[0004] However, the point cloud obtained by the above processing method is relatively sparse, resulting in insufficient accuracy of the obtained point cloud. Summary of the Invention

[0005] The embodiments of the present application provide a method for processing electromagnetic signal image data based on a neural network, which is used to convert the intermediate frequency signal obtained by mixing the echo electromagnetic signal and the scanning electromagnetic signal into a time-frequency spectrogram, and process the time-frequency spectrum based on the model constructed by the neural network to obtain point cloud data, so as to obtain denser point cloud data.

[0006] The embodiments of the present application provide a method for processing electromagnetic signal image data based on a neural network. The method is applied to an assembly robot, and a detection radar is set on the assembly robot. The method includes:

[0007] Scanning the component to be assembled with the detection radar according to a preset scanning path, and obtaining the echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the component to be assembled;

[0008] Performing mixing processing on the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, performing wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and converting the time-frequency signal into a time-frequency image;

[0009] Dividing the time-frequency image into blocks to obtain a plurality of time-frequency blocks and the positions of each time-frequency block, and generating input features according to the plurality of time-frequency blocks and the positions of each time-frequency block;

[0010] Encode the input features using an encoder to obtain low-dimensional point cloud features; wherein, the low-dimensional point cloud features are used to represent position information and angle information;

[0011] Use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled.

[0012] In the above technical solution, after the detection radar receives the echo electromagnetic signal, mix the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, so as to analyze the echo electromagnetic signal in the time-frequency domain, convert the time-frequency signal into a time-frequency diagram, divide the time-frequency diagram into multiple time-frequency diagram blocks, obtain the positions of each time-frequency diagram block, generate input features according to the time-frequency diagram block and the positions of each time-frequency diagram block, use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to restore the low-dimensional point cloud features to obtain point cloud data, which can control the scale of the point cloud data output by the encoder, so as to ensure that the encoder can output point cloud data with higher density, thus improving the accuracy of the point cloud data.

[0013] In a possible implementation manner, converting the time-frequency signal into a time-frequency image specifically includes:

[0014] Obtain the frequency range covered by the time-frequency signal, and segment the frequency range to obtain m frequency segments;

[0015] Obtain the time range covered by the time-frequency signal, and segment the time range to obtain n time periods;

[0016] Obtain the amplitude of the time-frequency signal in the i-th frequency segment and the j-th time period, calculate the average value of the amplitude of the time-frequency signal in the i-th frequency segment and the j-th time period, and use it as the pixel value of the i-th row and the j-th column; let i traverse from 1 to m, and j traverse from 1 to n to obtain the time-frequency image.

[0017] In the above technical solution, by dividing the frequency range of the time-frequency signal to obtain multiple frequency segments, dividing the time range of the time-frequency signal to obtain multiple time periods, using the serial number of the frequency segment as the pixel row serial number, using the serial number of the time period as the pixel column serial number, and using the amplitude of the i-th frequency segment and the j-th time period as the pixel value of the corresponding pixel, so as to retain the time, frequency and amplitude signals in the time-frequency signal when converting the time-frequency signal into a time-frequency image. Furthermore, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0018] In a possible implementation manner, dividing the time-frequency image into blocks to obtain multiple time-frequency diagram blocks and the positions of each time-frequency diagram block specifically includes:

[0019] Divide the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency diagram block array;

[0020] Obtain the frequency range corresponding to each time-frequency map block, calculate the average value of the frequency range corresponding to each time-frequency block, and use it as the ordinate corresponding to each time-frequency map block;

[0021] Obtain the time range corresponding to each time-frequency map block, calculate the average value of the time range corresponding to each time-frequency block, and use it as the abscissa corresponding to each time-frequency map block.

[0022] In the above technical solution, the time-frequency image is divided into multiple time-frequency map blocks, the frequency range and time range corresponding to each time-frequency map block are obtained, the frequency corresponding to each time block is used as the ordinate of the image block, and the time corresponding to each time block is used as the abscissa of the image block. Thus, input features are generated based on the image blocks and their positions, enabling the input features to contain amplitude, time, and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0023] In a possible implementation manner, generating input features according to multiple time-frequency map blocks and the positions of each time-frequency map block specifically includes:

[0024] Use multiple convolutional kernels to perform multiple convolutional processes on the time-frequency map blocks to obtain the amplitude features of the time-frequency map blocks;

[0025] Use a transformation function to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks;

[0026] Calculate the sum of the amplitude features and the position features of the time-frequency map blocks to obtain the fusion features of the time-frequency map blocks;

[0027] Concatenate the fusion features of all time-frequency map blocks to generate input features.

[0028] In the above technical solution, use convolutional kernels to process the time-frequency map blocks to obtain the amplitude features of the time-frequency map blocks, use a transformation function to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks, obtain the fusion features of the time-frequency map blocks based on the amplitude features and coordinate features of the time-frequency map blocks, and concatenate the fusion features of each time-frequency map block to obtain input features, enabling the input features to contain amplitude, time, and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0029] In a possible implementation manner, using a transformation function to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks specifically includes:

[0030] Use a horizontal coordinate transformation function to transform the horizontal coordinates of the time-frequency map blocks to obtain the horizontal coordinate features of the time-frequency map blocks;

[0031] The ordinate of the time-frequency map block is transformed using the ordinate transformation function to obtain the ordinate feature of the time-frequency map block;

[0032] The abscissa feature and the ordinate feature of the time-frequency map block are concatenated to obtain the position feature of the time-frequency map block.

[0033] In the above technical solution, the abscissa of the time-frequency map block is transformed using the abscissa transformation function to obtain the abscissa feature of the time-frequency map block, the ordinate of the time-frequency map block is transformed using the ordinate transformation function to obtain the ordinate feature of the time-frequency map block, and the abscissa feature and the ordinate feature of the time-frequency map block are concatenated. In this way, the position feature of the generated time-frequency map block contains time and frequency information.

[0034] In a possible implementation manner, an encoder is used to encode the input feature to obtain a low-dimensional point cloud feature, specifically including:

[0035] The first-level encoding module is used to encode the input feature to obtain the first intermediate encoding feature, the second-level encoding module is used to process the first intermediate encoding feature to obtain the second intermediate encoding feature, the third-level encoding module is used to process the second intermediate encoding feature to output the third intermediate encoding feature, and the fourth-level encoding module is used to process the third intermediate encoding feature to output the low-dimensional point cloud feature;

[0036] Each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. In each encoding module, the multi-head attention mechanism layer processes the input feature of the encoding module, and the feed-forward neural network layer processes the data output by the multi-head attention mechanism layer.

[0037] In the above technical solution, the encoder includes multiple encoding modules, each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. After the input feature passes through multiple encoding modules for information extraction, a low-dimensional point cloud feature is obtained, and distance and angle information are extracted from the amplitude, frequency, and time information in the time-frequency image. In this way, the point cloud data can be restored based on the low-dimensional point cloud feature.

[0038] In a possible implementation manner, a decoder is used to decode the low-dimensional point cloud feature to obtain the point cloud data of the component to be assembled, specifically including:

[0039] The first multi-layer perceptron layer is used to decode the low-dimensional point cloud feature to obtain the first decoded data; the data dimension of the low-dimensional point cloud feature is smaller than the data dimension of the first decoded data;

[0040] The second multi-layer perceptron layer is used to decode the first decoded data to obtain the second decoded data; the data dimension of the second decoded data is twice the data dimension of the first decoded data;

[0041] The second decoded data is decoded using a third multi-layer perceptron layer to obtain third decoded data; the data dimension of the second decoded data is greater than that of the third decoded data.

[0042] The third decoded data is decoded using a fourth multi-layer perceptron layer to obtain point cloud data.

[0043] In the above technical solution, the first multi-layer perceptron layer and the second multi-layer perceptron layer are used to increase the dimension of the low-dimensional point cloud features, so that the features output by the first multi-layer perceptron layer and the second multi-layer perceptron layer contain more position information. The third multi-layer perceptron layer and the fourth multi-layer perceptron layer are used to reduce the dimension of the data, and three-dimensional position information is generated based on multi-dimensional position information to realize the restoration of the point cloud data.

[0044] An embodiment of the present application provides an assembly robot, and a detection radar is provided on the assembly robot;

[0045] The detection radar is used to scan the to-be-assembled part according to a preset scanning path to obtain an echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the to-be-assembled part;

[0046] The echo electromagnetic signal and the scanning electromagnetic signal are subjected to mixing processing to obtain an intermediate frequency signal, the intermediate frequency signal is subjected to wavelet transform to obtain a time-frequency signal, and the time-frequency signal is converted into a time-frequency image;

[0047] The time-frequency image is segmented to obtain a plurality of time-frequency blocks and the positions of each time-frequency block, and input features are generated according to the plurality of time-frequency blocks and the positions of each time-frequency block;

[0048] An encoder is used to encode the input features to obtain low-dimensional point cloud features; wherein, the low-dimensional point cloud features are used to characterize position information and angle information;

[0049] A decoder is used to decode the low-dimensional point cloud features to obtain the point cloud data of the to-be-assembled part.

[0050] An embodiment of the present application provides an electronic device, including: a memory, a processor;

[0051] The memory stores computer execution instructions;

[0052] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above various possible implementation manners.

[0053] An embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the above various possible implementation manners.

[0054] An embodiment of the present application provides a computer program product, including a computer program, which when executed by a processor implements various possible implementation manners as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0056] Figure 1 It is a schematic flowchart of a method for processing electromagnetic signal image data based on a neural network provided by the present application;

[0057] Figure 2 It is a schematic diagram of the principle of generating a time-frequency image based on a time-frequency signal provided by the present application;

[0058] Figure 3 It is a schematic diagram of the principle of dividing a time-frequency image into blocks provided by the present application;

[0059] Figure 4 It is a schematic diagram of the structures of an encoder and a decoder provided by the present application.

[0060] Through the above-mentioned accompanying drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These accompanying drawings and the written description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] Here, exemplary embodiments will be described in detail, and examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different accompanying drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0062] The detection radar scans the parts to be assembled to obtain the point cloud of the assembled parts. More specifically, the detection radar continuously emits signals externally and receives the echo electromagnetic signals reflected by the parts to be assembled. Based on the intensity and phase of the echo electromagnetic signals, a point position of the parts to be assembled is obtained. By emitting signals at different angles, scanning of one surface of the parts to be assembled is achieved. After a scanning cycle, the obtained multiple echo electromagnetic signals are filtered to obtain each echo electromagnetic signal, and then each echo electromagnetic signal is processed to obtain a coordinate data. By traversing each echo electromagnetic signal, the point cloud of one surface is obtained.

[0063] However, the point cloud obtained by the above processing method is relatively sparse, resulting in insufficient accuracy of the obtained point cloud.

[0064] The electromagnetic signal data processing method provided by this application is applied to an assembly robot. A detection radar is provided on the assembly robot, and the detection radar is used to detect the components to be assembled. More specifically, after the assembly robot receives the echo electromagnetic signal, the echo electromagnetic signal and the scanning battery signal are mixed to obtain an intermediate frequency signal, and the intermediate frequency signal is subjected to wavelet transform to obtain a time-frequency spectrogram. Input features are generated based on the time-frequency spectrogram, and an encoder is used to process the input features to obtain low-dimensional point cloud features. Subsequently, a decoder is used to restore the low-dimensional point cloud features to generate point cloud data. By designing the data dimension of the low-dimensional point cloud data, the density of the restored point cloud data can be controlled, so that the obtained point cloud data can be more accurate.

[0065] The technical solution of this application and how the technical solution of this application solves the above technical problems will be described in detail below with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below in conjunction with the drawings.

[0066] Figure 1 It is a schematic flowchart of the electromagnetic signal image data processing method based on a neural network provided by this application. As Figure 1 shown, the data processing method is applied to an electronic device on an assembly robot. The method specifically includes the following steps:

[0067] S101. The electronic device uses the detection radar to scan the components to be assembled according to a preset scanning path, and obtains the echo electromagnetic signal returned by the scanning electromagnetic signal emitted by the detection radar when it encounters the components to be assembled.

[0068] Among them, a detection radar is set on the assembly robot, and the detection radar emits scanning electromagnetic signals outward according to a preset scanning path to scan the components to be assembled. The scanning electromagnetic signal is reflected back after encountering the components to be assembled, and then the detection radar receives the echo electromagnetic signal.

[0069] S102. Mix the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image.

[0070] Among them, a mixer is used to mix the echo electromagnetic signal and the scanning electromagnetic signal to obtain an intermediate frequency signal, perform wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and convert the amplitudes at different times and different frequencies in the time-frequency signal into the pixel values of each pixel point, so as to obtain a time-frequency image.

[0071] The high-frequency signals in the intermediate-frequency signals are filtered using a filter to obtain the low-frequency signals in the intermediate-frequency signals. The low-frequency signals in the intermediate-frequency signals are subjected to wavelet transform processing to obtain time-frequency signals.

[0072] S103. The time-frequency image is divided into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and input features are generated according to the multiple time-frequency blocks and the positions of each time-frequency block.

[0073] Among them, the time-frequency image is divided into multiple time-frequency blocks along the horizontal and vertical directions, and the positions of each time-frequency block are extracted. For example: 9 dividing lines are used both horizontally and vertically to obtain 10×10 time-frequency blocks, and the positions of each time-frequency block are obtained according to the time and frequency corresponding to each time-frequency block.

[0074] After obtaining each time-frequency block and the position of the time-frequency block, a feature conversion module is used to perform feature conversion on each time-frequency block and the position of the time-frequency block to obtain input features.

[0075] S104. An encoder is used to encode the input features to obtain low-dimensional point cloud features, and a decoder is used to decode the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled.

[0076] Among them, the low-dimensional point cloud features are used to represent position information and angle information. The encoder includes multiple encoding modules, and each encoding module includes a multi-head attention mechanism and a feed-forward neural network. Multiple encoding modules are used to encode the input features multiple times to obtain low-dimensional point cloud features. The parameters in the encoding module can be obtained through learning.

[0077] The encoder is used to output data of n×3. One row of data represents the coordinates of a point. By designing the number n, the scale of the point cloud data output by the encoder can be controlled, so as to ensure that the encoder can output point cloud data with higher density, thereby improving the accuracy of the point cloud data.

[0078] In the above technical solution, after the detection radar receives the echo electromagnetic signal, the echo electromagnetic signal and the scanning electromagnetic signal are mixed to obtain an intermediate-frequency signal, and the intermediate-frequency signal is subjected to wavelet transform to obtain a time-frequency signal. Thus, the echo electromagnetic signal is analyzed in the time-frequency domain, the time-frequency signal is converted into a time-frequency image, the time-frequency image is divided into multiple time-frequency blocks, the positions of each time-frequency block are obtained, input features are generated according to the time-frequency blocks and the positions of each time-frequency block, an encoder is used to encode the input features to obtain low-dimensional point cloud features, and a decoder is used to restore the low-dimensional point cloud features to obtain point cloud data. The scale of the point cloud data output by the encoder can be controlled, so as to ensure that the encoder can output point cloud data with higher density, thereby improving the accuracy of the point cloud data.

[0079] Optionally, after receiving the scanning electromagnetic signal emitted by the detection radar and its returned echo electromagnetic signal, the two are first mixed to obtain an intermediate frequency signal. Subsequently, the intermediate frequency signal is subjected to wavelet transform to extract the time-frequency signal. This step effectively removes the high-frequency noise in the signal, retains the key low-frequency information, and provides high-quality input data for subsequent processing. Through mixing processing and wavelet transform, the noise in the signal is effectively suppressed, while retaining the key features in the signal, laying a solid foundation for subsequent time-frequency analysis and feature extraction.

[0080] The time-frequency signal is converted into a time-frequency image. The specific steps include segmenting the frequency range and time range covered by the time-frequency signal, and calculating the amplitude average of the time-frequency signal in each segment as the pixel value, thereby constructing a time-frequency image. The conversion of the time-frequency image allows the signal features to be intuitively displayed in the time and space dimensions, which facilitates subsequent feature extraction and analysis. At the same time, through segmentation processing, the key time-frequency information in the signal is retained, which improves the accuracy and efficiency of feature extraction.

[0081] The time-frequency image is processed in blocks to obtain multiple time-frequency blocks and their position information. Subsequently, the time-frequency blocks are convolved using a convolution kernel to extract amplitude features, and the position coordinates are transformed using a conversion function to obtain position features. Finally, the amplitude features and position features are concatenated to generate input features. Through time-frequency image segmentation and feature extraction, the input features contain rich amplitude, time, and frequency information, providing comprehensive and accurate data support for subsequent encoder processing. At the same time, block processing reduces the complexity of data processing and improves processing efficiency.

[0082] The encoder is used to encode the generated input features to obtain low-dimensional point cloud features. The encoder adopts a multi-level encoding module structure. Each module contains a multi-head attention mechanism layer and a feedforward neural network layer. The position information and angle information are extracted through multi-layer encoding. The encoder deeply mines the input features through multi-level encoding modules to effectively extract low-dimensional point cloud features that represent the position information and angle information. This process not only reduces the dimension of the data, but also retains key information, providing efficient and accurate data input for subsequent decoder processing.

[0083] The decoder is used to decode the low-dimensional point cloud features and restore the point cloud data of the parts to be assembled. The decoder uses a multi-layer perceptron structure, and finally outputs three-dimensional point cloud data through layer-by-layer dimensionality increase and decrease. The decoder uses a multi-layer perceptron structure to efficiently decode the low-dimensional point cloud features and restore dense and accurate point cloud data. This process not only improves the density of point cloud data, but also retains the geometric shape and position information of the parts, providing reliable data support for subsequent assembly operations.

[0084] By designing the output dimension of the encoder, the scale of the point cloud data output by the decoder can be controlled. By adjusting the parameters and structure of the encoder, the density and coverage of the point cloud data can be flexibly controlled. By controlling the scale of the point cloud data, the density and coverage of the point cloud data can be adjusted according to actual needs. This process not only improves the applicability of the point cloud data, but also reduces the costs of data storage and processing.

[0085] During the entire processing process, through the collaborative work of multiple steps such as mixing processing, wavelet transform, time-frequency image conversion, feature extraction, encoding, and decoding, the accuracy of the point cloud data is effectively improved. Each step processes and optimizes the key features in the signal, and finally generates high-quality point cloud data. Through the collaborative work of multiple steps, the accuracy of the point cloud data is significantly improved. This not only provides reliable data support for subsequent assembly operations, but also improves the efficiency and accuracy of the entire assembly process.

[0086] In a possible implementation, S102, converting the time-frequency signal into a time-frequency image, specifically includes:

[0087] S201, obtaining the frequency range covered by the time-frequency signal, and segmenting the frequency range to obtain m frequency segments.

[0088] Figure 2 This is a schematic diagram of the principle for generating a time-frequency image based on the time-frequency signal provided by this application. As Figure 2 shown, the frequency range covered by the time-frequency signal is [fd, fu], and [fd, fu] is divided into m frequency bands, which are successively the 1st frequency band [fd, f1], the 2nd frequency band [f1, f2], the 3rd frequency band [f2, f3],..., the i-th frequency band [fi-1, fi],..., the (m-1)-th frequency band [fm-2, fm-1], and the m-th frequency band [fm-1, fu].

[0089] S202, obtaining the time range covered by the time-frequency signal, and segmenting the time range to obtain n time periods.

[0090] Among them, the time range covered by the time-frequency signal is [td, tu], and [td, tu] is divided into n time bands, which are successively the 1st time period [td, t1], the 2nd time period [t1, t2], the 3rd time period [t2, t3],..., the j-th time period [tj-1, tj],..., the (n-1)-th time period [tn-2, tn-1], and the n-th time period [tn-1, tu].

[0091] S203. Obtain the amplitude of the time-frequency signal in the j-th time period of the i-th frequency band, and calculate the average value of the amplitudes of the time-frequency signals in the j-th time period of the i-th frequency band as the pixel value of the j-th column in the i-th row. Let i traverse from 1 to m, and j traverse from 1 to n to obtain the time-frequency image.

[0092] Among them, sample the amplitude of the time-frequency signal in the j-th time period [tj-1, tj] of the i-th frequency band [fi-1, fi], and calculate the average value of the amplitudes of the sampled time-frequency signals as the pixel value of the j-th column in the i-th row. Let i traverse from 1 to m, and j traverse from 1 to n to obtain a time-frequency image with a size of m×n. m and n are positive integers.

[0093] In the above technical solution, by dividing the frequency range of the time-frequency signal to obtain multiple frequency bands, dividing the time range of the time-frequency signal to obtain multiple time periods, using the serial number of the frequency band as the pixel row serial number, using the serial number of the time period as the pixel column serial number, and using the amplitude of the i-th frequency band and the j-th time period as the pixel value of the corresponding pixel, so as to retain the time, frequency, and amplitude signals in the time-frequency signal when converting the time-frequency signal into a time-frequency image. Furthermore, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0094] Optionally, obtain the frequency range covered by the time-frequency signal and accurately divide it into m frequency bands. This step ensures the effective retention of the signal characteristics within each frequency band through precise frequency division. The precise division of the frequency range enables the detailed display of the characteristics of the time-frequency signal in the frequency dimension. The change in the signal amplitude within each frequency band reflects the signal strength within that frequency range, providing rich frequency information for the subsequent construction of the time-frequency image.

[0095] Next, obtain the time range covered by the time-frequency signal and divide it into n time periods. This step enables the clear presentation of the changes in the signal in the time dimension through the division of time periods. The division of the time range enables the accurate capture of the characteristics of the time-frequency signal in the time dimension. The change in the signal amplitude within each time period reflects the signal dynamics within that time period, providing important time information for the construction of the time-frequency image.

[0096] For the intersection point of each frequency band and time period (i.e., the i-th frequency band and the j-th time period), calculate the average value of the amplitude of the time-frequency signal in this area and use it as the pixel value of the corresponding pixel in the time-frequency image. This step effectively smooths the noise in the signal through the average calculation of the amplitude, improving the accuracy of the pixel value. The accurate conversion from amplitude to pixel value ensures that each pixel in the time-frequency image accurately reflects the signal strength within the corresponding frequency band and time period. This conversion not only retains the key amplitude information in the signal but also reduces the interference of noise through the average calculation, improving the quality of the time-frequency image.

[0097] By traversing the intersection points of all frequency bands and time periods, calculating and obtaining the pixel values of each intersection point, and finally constructing a complete time-frequency image. This step intuitively displays the characteristics of the time-frequency signal in the time and space dimensions in the form of an image through the construction of a pixel matrix. The construction of the pixel matrix enables the comprehensive presentation of the time, frequency, and amplitude information in the time-frequency signal in the time-frequency image. This presentation method not only facilitates subsequent feature extraction and analysis but also provides a reliable data basis for generating point cloud data based on the time-frequency image.

[0098] By converting the time-frequency signal into a time-frequency image, the time, frequency, and amplitude information in the signal is effectively retained. These information play a key role in the subsequent process of generating point cloud data, ensuring the accuracy and integrity of the point cloud data. When performing feature extraction based on the time-frequency image, since the image contains rich time-frequency information, the features characterizing the position and shape of the parts to be assembled can be more accurately extracted. These features provide precise data support for the subsequent generation of point cloud data, further improving the accuracy of the point cloud data.

[0099] In a possible implementation, S103: Obtaining a plurality of time-frequency tiles and the positions of each time-frequency tile by partitioning the time-frequency image, specifically including:

[0100] S301: Dividing the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency tile array.

[0101] Among them, the time-frequency image is divided using p row dividing lines and q column dividing lines to obtain a p×q time-frequency tile array. The m rows of pixels are divided into 10 regions, and the n columns of pixels are divided into 10 regions, so as to obtain a 10×10 time-frequency tile array.

[0102] Figure 3 For the schematic diagram of the partitioning principle of the time-frequency image provided by this application, as Figure 3 shown, the size of each image block is a×b. In the time-frequency tile array, the position of the upper left corner point in the time-frequency tile in the (k + 1)-th row and the (h + 1)-th column is the pixel at the (k×a + 1)-th row and the (h×b + 1)-th column, the position of the upper right corner point in the time-frequency tile is the pixel at the (k×a + 1)-th row and the ((h + 1)×b)-th column, the position of the lower left corner point in the time-frequency tile is the pixel at the ((k + 1)×a)-th row and the (h×b + 1)-th column, and the position of the lower right corner point in the time-frequency tile is the pixel at the ((k + 1)×a)-th row and the ((h + 1)×b)-th column.

[0103] S302: Obtaining the frequency range corresponding to each time-frequency tile, calculating the average value of the frequency range corresponding to each time-frequency block, and using it as the ordinate corresponding to each time-frequency tile.

[0104] Among them, obtain the frequency ranges corresponding to the pixels in the (k×a + 1)-th row, the frequency ranges corresponding to the pixels in the (k×a + 2)-th row, ……, the frequency ranges corresponding to the pixels in the ((k + 1)×a)-th row, and calculate the average value of the frequency ranges corresponding to the pixels in the (k×a + 1)-th row, the frequency ranges corresponding to the pixels in the (k×a + 2)-th row, ……, the frequency ranges corresponding to the pixels in the ((k + 1)×a)-th row.

[0105] Since the frequency ranges corresponding to the pixels in the (k×a + 1)-th row, the frequency ranges corresponding to the pixels in the (k×a + 2)-th row, ……, the frequency ranges corresponding to the pixels in the ((k + 1)×a)-th row are successively and continuously distributed frequency ranges, calculate the average value of the lower limit value of the frequency range corresponding to the pixels in the (k×a + 1)-th row and the upper limit value of the frequency range corresponding to the pixels in the ((k + 1)×a)-th row to obtain the ordinate corresponding to the time-frequency patch at the (k + 1)-th row and the (h + 1)-th column.

[0106] S303. Obtain the time range corresponding to each time-frequency patch, and calculate the average value of the time range corresponding to each time-frequency patch as the abscissa corresponding to each time-frequency patch.

[0107] Among them, obtain the time ranges corresponding to the pixels in the (h×b + 1)-th column, the time ranges corresponding to the pixels in the (h×b + 2)-th column, ……, the time ranges corresponding to the pixels in the ((h + 1)×b)-th column, and calculate the average value of the time ranges corresponding to the pixels in the (h×b + 1)-th column, the time ranges corresponding to the pixels in the (h×b + 2)-th column, ……, the time ranges corresponding to the pixels in the ((h + 1)×b)-th column.

[0108] Since the time ranges corresponding to the pixels in the (h×b + 1)-th column, the time ranges corresponding to the pixels in the (h×b + 2)-th column, ……, the time ranges corresponding to the pixels in the ((h + 1)×b)-th column are successively and continuously distributed time ranges, calculate the average value of the lower limit value of the time range corresponding to the pixels in the (h×b + 1)-th column and the upper limit value of the time range corresponding to the pixels in the ((h + 1)×b)-th column to obtain the ordinate corresponding to the time-frequency patch at the (k + 1)-th row and the (h + 1)-th column.

[0109] In the above technical solution, the time-frequency image is divided into multiple time-frequency patches, the frequency range and time range corresponding to each time-frequency patch are obtained, the frequency corresponding to each time patch is used as the ordinate of the image patch, and the time corresponding to each time patch is used as the abscissa of the image patch. Thus, input features are generated based on the image patches and the positions of the image patches, so that the input features contain amplitude, time, and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0110] Optionally, the time-frequency image is segmented along the pixel row direction and the pixel column direction to obtain a time-frequency tile array. In this step, by setting reasonable segmentation parameters (such as the number of row segments and the number of column segments), the time-frequency image is divided into multiple tiles of uniform or unequal sizes. The image tiling process effectively divides the information in the time-frequency image into multiple local regions, and each region (i.e., tile) contains a part of the time-frequency information of the original image. This segmentation method not only facilitates subsequent feature extraction but also improves the utilization rate of the overall information by retaining local information.

[0111] After obtaining the time-frequency tile array, the frequency range and time range corresponding to each tile are further calculated. Specifically, by calculating the average values of the frequencies and times corresponding to the pixel points within each tile, the ordinate (frequency coordinate) and abscissa (time coordinate) of the tile are obtained. The acquisition of the tile position information provides key data for the subsequent generation of input features. By using the frequency and time information as the coordinates of the tile, the input features not only contain amplitude information but also incorporate the dimensional information of time and frequency, thus enriching the connotation and expression ability of the features.

[0112] After obtaining the time-frequency tiles and their position information, a specific processing method (such as convolution processing, transfer function, etc.) is used to extract features from each tile, and the position information (ordinate and abscissa) of the tile is incorporated as part of the features. Finally, the features of all tiles are concatenated to generate the complete input features. The feature generation method based on tiles and position information enables the input features to contain rich amplitude, time, and frequency information. This multi-dimensional feature expression not only improves the discriminability and expression ability of the features but also provides more accurate and comprehensive data support for the subsequent encoding and decoding processes.

[0113] Through the tiling process and the incorporation of position information, the expression of the input features becomes more rich and accurate. Such features not only retain the key information in the original time-frequency image but also improve the utilization rate and accuracy of the information through the extraction and integration of local information. In the subsequent encoding and decoding processes, this rich feature expression will contribute to improving the generation quality and accuracy of the point cloud data.

[0114] The block processing of the time-frequency image enables more effective utilization of the information in the original image. Through the extraction and integration of local information, the input features contain more comprehensive and accurate time-frequency information. This information is fully utilized in the subsequent encoding and decoding processes, thereby improving the generation quality and accuracy of the point cloud data. The point cloud data generation method based on block processing and feature generation significantly enhances the accuracy of the point cloud data. Since the input features contain rich amplitude, time, and frequency information, the encoding and decoding processes can more accurately restore the geometric shape and position information of the components to be assembled. This improvement in accuracy not only provides reliable data support for subsequent assembly operations but also enhances the efficiency and accuracy of the entire assembly process.

[0115] In a possible implementation, S103. Generate input features according to multiple time-frequency patches and the positions of each time-frequency patch, specifically including:

[0116] S401. Perform multiple convolution operations on the time-frequency patches using multiple convolutional kernels to obtain the amplitude features of the time-frequency patches.

[0117] Among them, there are 2c convolutional kernels. Use the first convolutional kernel to perform a convolution operation on the first time-frequency patch to obtain the first element of the amplitude feature of the first time-frequency patch. Use the second convolutional kernel to perform a convolution operation on the first time-frequency patch to obtain the second element of the amplitude feature of the first time-frequency patch. ……. Use the 2c-th convolutional kernel to perform a convolution operation on the first time-frequency patch to obtain the 2c-th element of the amplitude feature of the first time-frequency patch.

[0118] Use the first convolutional kernel to perform a convolution operation on the second time-frequency patch to obtain the first element of the amplitude feature of the second time-frequency patch. Use the second convolutional kernel to perform a convolution operation on the second time-frequency patch to obtain the second element of the amplitude feature of the second time-frequency patch. ……. Use the 2c-th convolutional kernel to perform a convolution operation on the second time-frequency patch to obtain the 2c-th element of the amplitude feature of the second time-frequency patch.

[0119] And so on, until the processing of the last time-frequency patch is completed to obtain the amplitude features of the last time-frequency patch. The 2c convolutional kernels can be obtained through learning.

[0120] S402. Use a transformation function to transform the position coordinates of the time-frequency patches to obtain the position features of the time-frequency patches.

[0121] Among them, the conversion function includes multiple 2c conversion sub-functions. The first conversion sub-function converts the position coordinates of the first time-frequency map block to obtain the first element of the position feature of the first time-frequency map block. The second conversion sub-function converts the position coordinates of the first time-frequency map block to obtain the second element of the position feature of the first time-frequency map block.... The 2c-th conversion sub-function converts the position coordinates of the first time-frequency map block to obtain the 2c-th element of the position feature of the first time-frequency map block.

[0122] The first conversion sub-function converts the position coordinates of the second time-frequency map block to obtain the first element of the position feature of the second time-frequency map block. The second conversion sub-function converts the position coordinates of the second time-frequency map block to obtain the second element of the position feature of the second time-frequency map block.... The 2c-th conversion sub-function converts the position coordinates of the second time-frequency map block to obtain the 2c-th element of the position feature of the second time-frequency map block.

[0123] By analogy, until the position coordinates of the last time-frequency map block are processed to obtain the position feature of the last time-frequency map block. The 2c conversion sub-functions can be obtained through learning.

[0124] S403. Calculate the sum of the amplitude feature of the time-frequency map block and the position feature of the time-frequency map block to obtain the fusion feature of the time-frequency map block.

[0125] Among them, calculate the sum of the amplitude feature and the coordinate feature of the first time-frequency map block to obtain the fusion feature of the first time-frequency map block. Calculate the sum of the amplitude feature and the coordinate feature of the second time-frequency map block to obtain the fusion feature of the second time-frequency map block. By analogy, calculate the sum of the amplitude feature and the coordinate feature of the last time-frequency map block to obtain the fusion feature of the last time-frequency map block.

[0126] S404. Concatenate the fusion features of all time-frequency map blocks to generate the input feature.

[0127] Among them, concatenate the fusion feature of the first time-frequency map block, the fusion feature of the second time-frequency map block, the fusion feature of the third time-frequency map block,..., the fusion feature of the last time-frequency map block in order to obtain the input feature.

[0128] In the above technical solution, use the convolution kernel to process the time-frequency map block to obtain the amplitude feature of the time-frequency map block, use the conversion function to convert the position coordinates of the time-frequency map block to obtain the position feature of the time-frequency map block, obtain the fusion feature of the time-frequency map block based on the amplitude feature and the coordinate feature of the time-frequency map block, and concatenate the fusion features of each time-frequency map block to obtain the input feature, so that the input feature contains amplitude, time and frequency information. In this way, when generating point cloud data based on the time-frequency image, the accuracy of the point cloud data can be improved.

[0129] Optionally, multiple convolutional kernels are used to perform multiple convolutional operations on the time-frequency map blocks to extract the amplitude features of the time-frequency map blocks. As the basic unit for feature extraction, the convolutional kernel performs local feature extraction on the time-frequency map blocks in a sliding window manner. Each convolutional kernel can capture different local patterns in the block. The convolutional operation can automatically extract representative amplitude features from the time-frequency map blocks, and these features reflect the spatial distribution and local variations of the signal intensity in the block. By combining multiple convolutional kernels, more comprehensive and diverse amplitude features can be extracted, providing strong support for subsequent feature fusion and point cloud data generation.

[0130] Since the convolutional kernel can capture local details and texture information in the block, the extracted amplitude features have high accuracy and robustness. These features are of great significance for characterizing the signal intensity changes in the time-frequency map blocks and provide an accurate data basis for subsequent point cloud data generation.

[0131] A transformation function is used to transform the position coordinates of the time-frequency map blocks to obtain the position features of the time-frequency map blocks. The transformation function can be designed according to actual needs. For example, sine or cosine functions can be used to perform non-linear transformations on the position coordinates to extract specific patterns of the position information. The position coordinate transformation can extract the relative position information of the time-frequency map blocks in the time-frequency image, and these information are of great significance for characterizing the spatial and temporal relationships between the blocks. Through the non-linear transformation of the transformation function, the position coordinates can be transformed into more expressive feature vectors, providing strong support for subsequent feature fusion.

[0132] Since the position of each time-frequency map block in the time-frequency image is unique, the extracted position features are also unique. These features can distinguish different blocks and provide spatial position information for subsequent point cloud data generation, helping to improve the accuracy and integrity of the point cloud data.

[0133] The sum of the amplitude features and position features of the time-frequency map blocks is calculated to obtain the fused features of the time-frequency map blocks. This step combines the two features through a simple addition operation to generate a comprehensive feature that contains both amplitude information and position information. The feature fusion strategy can effectively combine the amplitude features and position features to generate fused features with richer information. These features not only contain the signal intensity information in the time-frequency map blocks but also the position information of the blocks in the time-frequency image, providing comprehensive data support for subsequent point cloud data generation.

[0134] The fused features of all time-frequency patches are concatenated to generate a complete input feature. This step combines the fused features of multiple patches through a simple concatenation operation to form an input feature vector that contains information from all patches. The generation process of the input feature ensures that the information of all time-frequency patches is effectively integrated, forming a comprehensive and accurate feature vector. These feature vectors not only contain rich amplitude, time, and frequency information but also reflect the spatial and temporal relationships between patches, providing a reliable data basis for subsequent point cloud data generation.

[0135] Since the input feature contains rich amplitude, time, and frequency information, as well as the spatial and temporal relationships between patches, the point cloud data generated based on these features will have higher accuracy and integrity. This information provides more precise data support for subsequent assembly operations, helping to improve the efficiency and accuracy of assembly. By optimizing the generation process of the input feature, the quality of the point cloud data can be further improved. For example, by adjusting the size and number of convolutional kernels, the type and parameters of transfer functions, etc., more accurate and comprehensive feature information can be extracted, and then higher-quality point cloud data can be generated. This optimization process not only improves the accuracy of the point cloud data but also enhances its adaptability and robustness in complex environments.

[0136] In a possible implementation, in S402, the position coordinates of the time-frequency patches are transformed using a transfer function to obtain the position features of the time-frequency patches, which specifically includes:

[0137] In S501, the abscissa of the time-frequency patch is transformed using an abscissa transfer function to obtain the abscissa feature of the time-frequency patch.

[0138] Among them, the transformation coordinates include an abscissa transfer function and an ordinate transfer function.

[0139] Taking any time-frequency patch as an example. The abscissa transfer function includes c abscissa transfer sub-functions. The first abscissa transfer sub-function processes the abscissa of the time-frequency patch to obtain the first element of the abscissa feature of the time-frequency patch. The second abscissa transfer sub-function processes the abscissa of the time-frequency patch to obtain the second element of the abscissa feature of the time-frequency patch. And so on, the c-th abscissa transfer sub-function processes the abscissa of the time-frequency patch to obtain the c-th element of the abscissa feature of the time-frequency patch.

[0140] In S502, the ordinate of the time-frequency patch is transformed using an ordinate transfer function to obtain the ordinate feature of the time-frequency patch.

[0141] Among them, the vertical coordinate conversion function includes c vertical coordinate conversion sub-functions. The first vertical coordinate conversion sub-function processes the vertical coordinate of the time-frequency diagram block to obtain the first element of the vertical coordinate feature of the time-frequency diagram block. The second vertical coordinate conversion sub-function processes the vertical coordinate of the time-frequency diagram block to obtain the second element of the vertical coordinate feature of the time-frequency diagram block. By analogy, the c-th vertical coordinate conversion sub-function processes the vertical coordinate of the time-frequency diagram block to obtain the c-th element of the vertical coordinate feature of the time-frequency diagram block.

[0142] The horizontal coordinate conversion sub-function and the vertical coordinate conversion sub-function use existing sine functions or cosine functions.

[0143] S503. Concatenate the horizontal coordinate feature and the vertical coordinate feature of the time-frequency diagram block to obtain the position feature of the time-frequency diagram block.

[0144] Among them, the size of the horizontal coordinate feature of the time-frequency diagram block is 1×c, the size of the vertical coordinate feature of the time-frequency diagram block is 1×c, and the horizontal coordinate feature and the vertical coordinate feature of the time-frequency diagram block are concatenated to obtain the position feature of the time-frequency diagram block. At this time, the size of the position feature of the time-frequency diagram block is 1×2c.

[0145] In the above technical solution, the horizontal coordinate of the time-frequency diagram block is converted using the horizontal coordinate conversion function to obtain the horizontal coordinate feature of the time-frequency diagram block, the vertical coordinate of the time-frequency diagram block is converted using the vertical coordinate conversion function to obtain the vertical coordinate feature of the time-frequency diagram block, and the horizontal coordinate feature and the vertical coordinate feature of the time-frequency diagram block are concatenated. In this way, the position feature of the generated time-frequency diagram block contains time and frequency information.

[0146] Optionally, the horizontal coordinate (i.e., the time coordinate) of the time-frequency diagram block is converted using the horizontal coordinate conversion function to obtain the horizontal coordinate feature of the time-frequency diagram block. The horizontal coordinate conversion function can be designed according to actual needs. For example, sine functions, cosine functions or other non-linear functions are used to perform non-linear transformation on the horizontal coordinate, so as to extract specific patterns or information in the horizontal coordinate. Through the processing of the horizontal coordinate conversion function, the horizontal coordinate of the time-frequency diagram block is converted into a more expressive feature vector. This conversion process not only retains the original information of the horizontal coordinate, but also introduces new feature dimensions through non-linear transformation, making the horizontal coordinate feature more distinguishable and expressive in the subsequent feature fusion and point cloud data generation processes.

[0147] Similar to the abscissa conversion function, the ordinate conversion function is used to convert the ordinate (i.e., the frequency coordinate) of the time-frequency map block to obtain the ordinate feature of the time-frequency map block. The ordinate conversion function can also be designed according to actual needs to perform non-linear transformation on the ordinate and extract specific information in the ordinate. The processing of the ordinate conversion function converts the ordinate of the time-frequency map block into a feature vector with rich information. This conversion process not only retains the original information of the ordinate but also introduces new feature dimensions through non-linear transformation, enhancing the expression ability of the ordinate feature in subsequent processing.

[0148] After obtaining the abscissa feature and ordinate feature of the time-frequency map block, the two are concatenated to form a complete position feature. This concatenation process can be achieved through a simple vector concatenation operation, combining the abscissa feature and ordinate feature in a certain order to form a comprehensive feature vector containing time and frequency information. The feature concatenation strategy enables the position feature of the time-frequency map block to contain both time and frequency information simultaneously. This multi-dimensional feature expression not only improves the discrimination and expression ability of the features but also provides more accurate and comprehensive data support for subsequent encoding and decoding processes. By concatenating the abscissa feature and ordinate feature, the generated position feature can more comprehensively reflect the position information of the time-frequency map block in the time-frequency image, providing a key data basis for subsequent point cloud data generation.

[0149] Through the processing of the above conversion function and concatenation strategy, the generated position feature contains rich time and frequency information. These information not only reflect the specific position of the time-frequency map block in the time-frequency image but also introduce new feature dimensions through non-linear transformation, making the position feature more expressive and discriminative in subsequent processing. In the process of generating point cloud data based on the time-frequency image, this position feature that combines time and frequency information can more accurately characterize the geometric shape and position relationship of the parts to be assembled, thereby improving the accuracy and integrity of the point cloud data.

[0150] Since the position feature contains rich time and frequency information, the point cloud data generated based on these features will have higher accuracy and robustness. These information provide more precise data support for subsequent assembly operations, helping to improve the efficiency and accuracy of assembly. At the same time, by optimizing the conversion function and concatenation strategy, more expressive position features can be further extracted, thereby further improving the quality and accuracy of the point cloud data.

[0151] In a possible implementation, as Figure 4 shown, S104. Encode the input features using an encoder to obtain low-dimensional point cloud features, specifically including:

[0152] Encode the input features using the encoding module at the first level to obtain the first intermediate encoded features, process the first intermediate encoded features using the encoding module at the second level to obtain the second intermediate encoded features, process the second intermediate encoded features using the encoding module at the third level to output the third intermediate encoded features, and process the third intermediate encoded features using the encoding module at the fourth level to output the low-dimensional point cloud features.

[0153] Among them, the encoder includes four encoding modules, each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. In each encoding module, the multi-head attention mechanism layer processes the input features of the encoding module, and the feed-forward neural network layer processes the data output by the multi-head attention mechanism layer. The multi-head attention mechanism layer and the feed-forward neural network layer use existing network structures and will not be elaborated here.

[0154] There is a residual normalization layer after the multi-head attention mechanism layer. The features processed by the multi-head attention mechanism layer are processed by the residual normalization layer and then input to the feed-forward neural network layer. There is a residual normalization layer after the feed-forward neural network layer. The features processed by the feed-forward neural network layer are output to the residual normalization layer for processing and then the encoded features of the encoded features are output.

[0155] The input features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network layer, and the second residual normalization layer in the encoding module at the first level in sequence, and then the first intermediate encoded features are output. The second intermediate encoded features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network, and the second residual normalization layer in the encoding module at the second level in sequence, and then the second intermediate encoding layer is output. The second intermediate encoded features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network, and the second residual normalization layer in the encoding module at the third level in sequence, and then the third intermediate encoding layer is output. The third intermediate encoded features are processed by the multi-head attention mechanism layer, the first residual normalization layer, the feed-forward neural network, and the second residual normalization layer in the encoding module at the fourth level in sequence, and then the low-dimensional point cloud features are output.

[0156] In the above technical solution, the encoder includes multiple encoding modules, each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. The input features are processed by multiple encoding modules for information extraction to obtain low-dimensional point cloud features, and the distance and angle information are extracted from the amplitude, frequency, and time information in the time-frequency image. In this way, the point cloud data can be restored based on the low-dimensional point cloud features.

[0157] Optionally, the encoder consists of multiple cascaded encoding modules, and each encoding module is responsible for processing the input features step by step. This cascaded structure enables the encoder to gradually extract the deep-level information in the input features, laying the foundation for generating low-dimensional point cloud features. Through the design of multi-level encoding modules, the encoder can more comprehensively capture the complex patterns and structures in the input features. Each level of encoding module can further extract features based on the previous level, thereby generating more expressive intermediate encoded features.

[0158] Each encoding module includes a multi-head attention mechanism layer and a feed-forward neural network layer. The multi-head attention mechanism layer is responsible for performing attention-weighted processing on the input features to highlight the key information; the feed-forward neural network layer then performs a non-linear transformation on the output data of the attention mechanism layer to extract higher-level features. By simulating the human brain's attention mechanism, the multi-head attention mechanism layer can dynamically adjust the weights between different features, enabling the key information to receive more attention in subsequent processing. The feed-forward neural network layer introduces new feature dimensions through non-linear transformation, enhancing the expressive ability of the features. The combination of the two enables the encoding module to more effectively extract the useful information in the input features.

[0159] The first-level encoding module performs preliminary processing on the input features to generate the first intermediate encoded features. In this process, the multi-head attention mechanism layer first identifies the key information in the input features and performs a preliminary non-linear transformation through the feed-forward neural network layer. The processing of the first-level encoding module effectively extracts and transforms the preliminary information in the input features, laying the foundation for subsequent processing.

[0160] Subsequently, the second-level, third-level, and fourth-level encoding modules sequentially process the intermediate encoded features of the previous level, gradually extracting deeper-level information. Each level of encoding module can further mine the useful information in the features based on the previous level and generate more expressive intermediate encoded features. Through the progressive processing of multi-level encoding modules, the encoder can gradually extract the complex patterns and structures in the input features and finally generate low-dimensional point cloud features containing rich information. This process not only improves the expressive ability of the features but also provides a reliable data basis for subsequent point cloud data restoration.

[0161] During the encoding process, the encoder extracts distance and angle information from the amplitude, frequency, and time information in the time-frequency image through the collaborative action of the multi-head attention mechanism layer and the feed-forward neural network layer. These information are gradually fused and transformed during the encoding process and finally generate low-dimensional point cloud features. Through this process, the encoder realizes the transformation from the time-frequency image to low-dimensional point cloud features. The low-dimensional point cloud features not only contain the key information in the input features but also introduce new feature dimensions and expression methods through the processing of the encoder, providing more accurate and comprehensive data support for subsequent point cloud data restoration.

[0162] Based on the generated low-dimensional point cloud features, the point cloud data can be further restored. Since the low-dimensional point cloud features contain rich distance and angle information, the restored point cloud data will have higher accuracy and integrity. This provides more accurate data support for subsequent assembly operations, helping to improve the efficiency and accuracy of assembly.

[0163] In a possible implementation, continue to refer to Figure 4 , S104. Use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the component to be assembled, specifically including:

[0164] S601. Use the first multi-layer perceptron layer to decode the low-dimensional point cloud features to obtain the first decoded data. The data dimension of the low-dimensional point cloud features is smaller than the data dimension of the first decoded data.

[0165] Among them, the first multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the data dimension of the first decoded data output by the output layer is made larger than the data dimension of the low-dimensional point cloud features. By expanding the low-dimensional point cloud features, more position information is included in the first decoded data. For example: the data dimension of the low-dimensional point cloud features is n×1. The data dimension of the second decoded data is n×256.

[0166] S602. Use the second multi-layer perceptron layer to decode the first decoded data to obtain the second decoded data. The data dimension of the second decoded data is twice the data dimension of the first decoded data.

[0167] Among them, the second multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the data dimension of the second decoded data output by the output layer is made twice the data dimension of the first decoded feature. By expanding the second decoded feature, more position information is included in the second decoded data. The data dimension of the second decoded data is n×512.

[0168] S603. Use the third multi-layer perceptron layer to decode the second decoded data to obtain the third decoded data. The data dimension of the second decoded data is larger than the data dimension of the third decoded data.

[0169] Among them, the third multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the data dimension of the third decoded data output by the output layer is made smaller than the data dimension of the second decoded feature, thus realizing the extraction of position information from the second decrypted feature.

[0170] S604. Use the fourth multi-layer perceptron layer to perform decoding processing on the third decoded data to obtain point cloud data.

[0171] Among them, the fourth multi-layer perceptron layer includes an input layer, at least one hidden layer, and an output layer. Neurons in each layer are connected to all neurons in the next layer, and a non-linear activation function is introduced into the neurons. By designing the number of neurons in each hidden layer, the output data dimension of the output layer is n×3, where each row represents the three-dimensional coordinate information of a point, and n represents the number of points in the point cloud.

[0172] In the above technical solution, the first multi-layer perceptron layer and the second multi-layer perceptron layer are used to increase the dimension of the low-dimensional point cloud features, so that the features output by the first multi-layer perceptron layer and the second multi-layer perceptron layer contain more position information. The third multi-layer perceptron layer and the fourth multi-layer perceptron layer are used to reduce the dimension of the data, and three-dimensional position information is generated based on multi-dimensional position information to realize the restoration of the point cloud data.

[0173] Optionally, use the first multi-layer perceptron layer to perform decoding processing on the low-dimensional point cloud features to obtain the first decoded data. In this step, the data dimension of the low-dimensional features is expanded, providing a richer information basis for subsequent processing.

[0174] Immediately afterwards, the second multi-layer perceptron layer decodes the first decoded data and outputs the second decoded data. It should be noted that the data dimension of the second decoded data is twice that of the first decoded data, and this significant dimension increase helps to capture more position information and detailed features.

[0175] Subsequently, the third multi-layer perceptron layer processes the second decoded data to generate the third decoded data. At this stage, the data dimension decreases, but this is not simply information loss. Instead, through a carefully designed network structure, the key information crucial for three-dimensional reconstruction is extracted from the high-dimensional features.

[0176] Finally, the fourth multi-layer perceptron layer performs decoding processing on the third decoded data and directly outputs the point cloud data of the component to be assembled. This step realizes the accurate mapping from the high-dimensional feature space to the three-dimensional physical space.

[0177] Through the continuous dimensionality increase operations of the first and second multi-layer perceptron layers, the low-dimensional point cloud features are effectively extended to a higher-dimensional space. During this process, the expressive power of the features is significantly enhanced, capable of containing more implicit information about the position, shape, and structural details of the point cloud, laying a solid foundation for subsequent point cloud reconstruction. The dimensionality reduction process of the third multi-layer perceptron layer is a selective information extraction process, which can accurately screen out the feature subset crucial for three-dimensional position information reconstruction from the highly redundant high-dimensional features. This process effectively avoids information overload while ensuring the accuracy and integrity of the reconstructed point cloud data. The fourth multi-layer perceptron layer, as the final decoding step, directly maps the optimized features into three-dimensional space to generate high-precision point cloud data. Due to the flexible transformation of feature dimensions and the efficient integration of information in the previous steps, the generated point cloud data reaches a high level in terms of position accuracy, shape fidelity, and detail expressiveness. Through the designed multi-layer perceptron hierarchical structure and dimensionality transformation strategy, this technical solution not only performs well in processing specific low-dimensional point cloud features but also has a certain generalization ability, capable of adapting to the processing requirements of point cloud data with different complexities and diversities, enhancing the overall robustness of the system.

[0178] Some embodiments of this application provide a method for processing electromagnetic signal image data based on a neural network. The method specifically includes the following steps:

[0179] S701. The electronic device uses a detection radar to scan the to-be-assembled component according to a preset scanning path, and obtains the echo electromagnetic signal returned when the scanning electromagnetic signal emitted by the detection radar hits the to-be-assembled component.

[0180] S702. Mix the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate-frequency signal, perform wavelet transform on the intermediate-frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image.

[0181] S703. Divide the time-frequency image into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and generate input features according to the multiple time-frequency blocks and the positions of each time-frequency block.

[0182] S704. Use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the to-be-assembled component.

[0183] S705. Obtain the three-dimensional model data of the to-be-assembled component in the assembly posture, and obtain the deflection angle of the to-be-assembled component according to the three-dimensional model data of the to-be-assembled component in the assembly posture and the point cloud data of the to-be-assembled component.

[0184] Among them, to obtain the three-dimensional model data of the to-be-assembled component, determine the three-dimensional model data of the to-be-assembled component in the assembly posture according to the installation position of the assembly tooling on the assembly robot.

[0185] Two positioning structures are designed on the component to be assembled, and the angle of the straight line where the two positioning structures are located in the assembly posture is calculated according to the three-dimensional model data of the component to be assembled in the assembly posture. More specifically, the theoretical positions of the first positioning structure and the second positioning structure are obtained according to the three-dimensional model data of the component to be assembled in the assembly posture, and the angle of the straight line where the two positioning structures are located in the assembly posture is calculated according to the theoretical positions of the first positioning structure and the second positioning structure.

[0186] The angle of the straight line where the two positioning structures are located in the actual posture is calculated according to the point cloud data of the component to be assembled. More specifically, the actual positions of the first positioning structure and the second positioning structure are obtained according to the point cloud data of the component to be assembled, and the angle of the straight line where the two positioning structures are located in the actual posture is calculated according to the actual positions of the first positioning structure and the second positioning structure.

[0187] Calculate the difference between the angle of the straight line where the positioning structure is located in the assembly posture and the angle of the straight line where the positioning structure is located in the actual posture, and use the difference as the deflection angle of the component to be assembled.

[0188] In the above technical solution, the angle of the straight line where the two positioning structures are located in the assembly posture is calculated according to the three-dimensional model data of the component to be assembled in the assembly posture, the angle of the straight line where the two positioning structures are located in the actual posture is calculated according to the point cloud data of the component to be assembled, and the difference between the two calculated angles is used as the deflection angle of the component to be assembled, so as to realize the determination of the deflection angle of the component to be assembled based on the positioning structure.

[0189] Optionally, by using the three-dimensional model data and the point cloud data, the technical solution can accurately calculate the straight line angles of the positioning structure in the assembly posture and the actual posture. This calculation method based on geometric features avoids the deviation of the deflection angle calculation caused by measurement errors or inaccurate data in the traditional method. Using two positioning structures for calculation can further improve the accuracy of the deflection angle. By comparing the angle differences of the two positioning structures in the assembly posture and the actual posture, the influence caused by the error of a single positioning structure can be effectively offset.

[0190] The accurate calculation of the deflection angle provides accurate direction guidance for the operation of the assembly robot. The robot can accurately adjust the clamping angle and grasping path of the manipulator according to the calculated deflection angle, ensure that the component will not be damaged during the grasping process, and maintain the grasping stability. By rotating the component to the correct assembly posture and then moving it, the assembly robot can avoid collisions or damages caused by incorrect component postures during the moving process, thereby improving the overall success rate of the assembly process.

[0191] Due to the adoption of an accurate deflection angle calculation method, the assembly robot does not need to make repeated adjustments and trial-and-error when grasping and moving the parts to be assembled. This greatly shortens the assembly cycle and improves the overall assembly efficiency. In addition, this technical solution is also applicable to various types and sizes of parts to be assembled. By adjusting the processing parameters of the 3D model data and point cloud data, it can quickly adapt to the assembly requirements of different products, further enhancing the flexibility and adaptability of the production line.

[0192] In the traditional assembly process, a large amount of manual participation is required to adjust the posture and position of the parts. This technical solution effectively reduces the dependence on manual intervention through automated calculation and precise robot operation. This not only reduces the labor cost but also improves the consistency and stability of the assembly process.

[0193] S706. Control the manipulator of the assembly robot to grasp the part to be assembled according to the deflection angle of the part to be assembled, rotate it to the assembly posture, and move the part to be assembled to the assembly fixture.

[0194] Among them, after obtaining the deflection angle of the part to be assembled, control the manipulator on the assembly robot according to the deflection angle of the part to be assembled, so that the manipulator can grip the part on the clamping surface of the part to be assembled. After the manipulator grips the part, make the manipulator return to the device configuration and move the part to be assembled to the assembly fixture.

[0195] In the above technical solution, after obtaining the point cloud data of the part to be assembled, obtain the deflection angle of the part to be assembled according to the 3D model data of the part to be assembled in the assembly posture and the point cloud data of the part to be assembled. The assembly robot controls the manipulator to grip the part to be assembled according to the deflection angle, and returns to the default assembly posture after gripping the part to be assembled, so as to realize moving the part to be assembled to the assembly fixture. The robot does not need to repeatedly adjust the relative relationship between the part to be assembled and the assembly fixture, improving the assembly efficiency of the assembly robot.

[0196] Optionally, obtaining high-precision point cloud data by decoding low-dimensional point cloud features provides an accurate data basis for subsequent deflection angle calculation. Using the comparison and analysis of 3D model data and point cloud data, accurately calculate the deflection angle of the part to be assembled, avoiding the assembly accuracy problems caused by measurement errors in the traditional method. The assembly robot directly grasps and rotates the part to the assembly posture according to the deflection angle, without repeatedly adjusting the relative relationship between the part and the fixture, significantly improving the assembly efficiency. This technical solution organically combines point cloud data processing, 3D model comparison and analysis, and assembly robot control technology, realizing the intelligence and automation of the assembly process. The assembly robot can independently decide the grasping path and assembly posture according to the real-time obtained point cloud data and 3D model data, enhancing the robot's autonomous operation ability.

[0197] In the traditional assembly process, a large amount of manual participation is required, which not only has a high labor intensity but also poses safety risks. This technical solution replaces manual operations with automated assembly robots, effectively reducing labor costs and safety risks. The assembly robot can perform precise operations according to a predetermined program, avoiding assembly errors and safety accidents caused by human factors. This technical solution is applicable to various types and sizes of parts to be assembled, with strong flexibility and adaptability. By adjusting the decoder parameters and three-dimensional model data, it can quickly adapt to the assembly requirements of different products and achieve rapid switching and adjustment of the production line. Through precise calculation of the deflection angle and automated assembly process, the errors and defective product rate in the assembly process are effectively reduced. The assembly robot can perform standardized operations according to a predetermined program, ensuring the consistency and stability of the assembly process, thereby improving the overall quality of the product.

[0198] Some embodiments of this application provide a method for processing electromagnetic signal image data based on a neural network. The method specifically includes the following steps:

[0199] S801. The electronic device uses a detection radar to scan the parts to be assembled according to a preset scanning path, and obtains the echo electromagnetic signal returned after the scanning electromagnetic signal emitted by the detection radar hits the parts to be assembled.

[0200] S802. Mix the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate-frequency signal, perform wavelet transform on the intermediate-frequency signal to obtain a time-frequency signal, and convert the time-frequency signal into a time-frequency image.

[0201] S803. Divide the time-frequency image into blocks to obtain multiple time-frequency blocks and the positions of each time-frequency block, and generate input features according to the multiple time-frequency blocks and the positions of each time-frequency block.

[0202] S804. Use an encoder to encode the input features to obtain low-dimensional point cloud features, and use a decoder to decode the low-dimensional point cloud features to obtain the point cloud data of the parts to be assembled.

[0203] S805. Obtain the three-dimensional model data of the parts to be assembled in the assembly posture, and obtain the deflection angle of the parts to be assembled according to the three-dimensional model data of the parts to be assembled in the assembly posture and the point cloud data of the parts to be assembled.

[0204] S806. Convert the point cloud data of the assembled parts into the point cloud data in the assembly posture according to the deflection angle of the parts to be assembled.

[0205] Among them, a rotation matrix is generated according to the deflection angle of the parts to be assembled, and the inner product of the point cloud data of the parts to be assembled and the rotation matrix is calculated to obtain the point cloud data in the assembly posture.

[0206] S807. Extract the theoretical data of each surface of the to-be-assembled component from the 3D model data of the to-be-assembled component in the assembly posture.

[0207] Among them, the to-be-assembled component has six surfaces, and the theoretical data of the six surfaces are extracted from the 3D model data of the to-be-assembled component in the assembly posture.

[0208] S808. Extract the actual data of each surface of the to-be-assembled component from the point cloud data of the to-be-assembled component in the assembly posture.

[0209] Among them, the actual data of the six surfaces are extracted from the point cloud data of the to-be-assembled component in the assembly posture.

[0210] S809. Perform homonymous point matching on the theoretical data of each surface of the to-be-assembled component and the actual data of each surface, and calculate the distance between each pair of homonymous points.

[0211] Among them, use the existing homonymous point matching algorithm to perform homonymous point matching on the theoretical data of each surface of the to-be-assembled component and the actual data of each surface, and calculate the distance between each pair of homonymous points according to the positions of the homonymous points.

[0212] S810. Calculate the variance of the distances between each pair of homonymous points. If the variance is greater than the preset threshold, it is determined that the to-be-assembled component has been knocked or bumped.

[0213] Among them, if the to-be-assembled component has been knocked or bumped, the surface similarity of the to-be-assembled component decreases, and the distances between each pair of homonymous points are relatively far apart, that is, the variance of the distances between each pair of homonymous points is relatively large. If the to-be-assembled component has not been knocked or bumped, the surface similarity of the to-be-assembled component is relatively high, and the distances between each pair of homonymous points are relatively close, that is, the variance of the distances between each pair of homonymous points is relatively small. Based on the variance of the distances between each pair of homonymous points, it is determined whether the to-be-assembled component has been knocked or bumped.

[0214] In the above technical solution, after obtaining the point cloud data of the to-be-assembled component, the point cloud data of the to-be-assembled component is converted into the point cloud data in the assembly posture according to the deflection angle of the to-be-assembled component. The theoretical data of each surface of the to-be-assembled component are extracted from the 3D model data of the to-be-assembled component in the assembly posture, and the actual data of each surface of the to-be-assembled component are extracted from the point cloud data of the to-be-assembled component in the assembly posture. In this way, the theoretical data and the actual data of the same surface can be subjected to homonymous point matching, the variance of the distances between each pair of homonymous points can be calculated, and it can be determined whether the to-be-assembled component has been knocked or bumped according to the variance. The point cloud data is reused for appearance inspection, the equipment cost is reduced, and in this way, the appearance inspection is performed again before assembly, and the assembly efficiency is improved.

[0215] Optionally, by performing an appearance inspection before assembly, components with knocking problems can be promptly detected and processed, avoiding assembly failures or rework caused by component quality issues during the assembly process. This significantly improves the assembly efficiency and reduces the production cost. Using point cloud data for appearance inspection eliminates the need for additional detection equipment or complex detection processes, enabling fast and efficient detection.

[0216] Then, reuse the point cloud data for appearance inspection without additional detection equipment or sensors. This reduces the equipment procurement cost and maintenance cost, improving the overall economic efficiency of the production line. By calculating the deflection angle and point cloud data conversion, the accuracy and reliability of the appearance inspection are ensured, avoiding misjudgment or missed judgment caused by insufficient equipment accuracy. By converting the point cloud data of the component to be assembled to the assembly posture, it is ensured that the theoretical data and the actual data are matched and compared in the same posture, improving the accuracy of the appearance inspection. Using homologous point matching and distance variance analysis, the difference between the theoretical data and the actual data can be accurately quantified, and it can be accurately determined whether the component has been knocked.

[0217] This technical solution organically combines technologies such as point cloud data processing, deflection angle calculation, surface data extraction and matching, and distance variance analysis to achieve automated appearance inspection before assembly. By integrating with the control system of the assembly robot, a fully automated process from component grasping, deflection angle calculation, appearance inspection to assembly can be realized, improving the intelligent level and automation degree of the production line. By performing strict appearance inspection before assembly, it is ensured that the components assembled onto the product meet the quality requirements, improving the overall quality and reliability of the product.

[0218] The electronic device provided in this embodiment includes: at least one processor and a memory. Optionally, the device further includes a communication component. Among them, the processor, the memory, and the communication component are connected through a bus.

[0219] In the specific implementation process, at least one processor executes the computer execution instructions stored in the memory, causing at least one processor to execute the above method.

[0220] For the specific implementation process of the processor, reference can be made to the above method embodiment, and its implementation principle and technical effects are similar, so they will not be elaborated here in this embodiment.

[0221] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0222] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0223] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0224] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0225] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above method is implemented.

[0226] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0227] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0228] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed among each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0229] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0230] Furthermore, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0231] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0232] Those of ordinary skill in the art will understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program code.

[0233] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for processing electromagnetic signal image data based on neural network, characterized in that: The method is applied to an assembly robot, which is provided with a detection radar, and the method comprises: Scan the component to be assembled using the detection radar according to a preset scanning path, and obtain an echo electromagnetic signal returned by the scanning electromagnetic signal emitted by the detection radar when it encounters the component to be assembled; Mixing the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate frequency signal, performing wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and converting the time-frequency signal into a time-frequency image; Divide the time-frequency image into blocks to obtain a plurality of time-frequency blocks and positions of the time-frequency blocks, and generate input features according to the plurality of time-frequency blocks and positions of the time-frequency blocks; Using an encoder to encode the input features to obtain low-dimensional point cloud features; wherein the low-dimensional point cloud features are used to represent position information and angle information; The decoder is used to decode the low-dimensional point cloud features to obtain point cloud data of the components to be assembled.

2. The electromagnetic signal data processing method according to claim 1, characterized in that: Converting the time-frequency signal into a time-frequency image specifically includes: Acquire a frequency range covered by the time-frequency signal, and segment the frequency range to obtain m frequency segments; Acquire the time range covered by the time-frequency signal, and segment the time range to obtain n time periods; Get the amplitude of the time-frequency signal in the jth time period of the i-th frequency segment, calculate the average value of the amplitude of the time-frequency signal in the jth time period of the i-th frequency segment, and use it as the pixel value of the i-th row and j-th column; let i traverse from 1 to m and j traverse from 1 to n to obtain the time-frequency image.

3. The electromagnetic signal data processing method according to claim 2, characterized in that: Dividing the time-frequency image into blocks to obtain a plurality of time-frequency blocks and the position of each time-frequency block specifically includes: Segmenting the time-frequency image along the pixel row direction and the pixel column direction to obtain a time-frequency image block array; Obtain the frequency range corresponding to each time-frequency block, and calculate the average value of the frequency range corresponding to each time-frequency block as the ordinate corresponding to each time-frequency block; The time range corresponding to each time-frequency block is obtained, and the average value of the time range corresponding to each time-frequency block is calculated as the horizontal coordinate corresponding to each time-frequency block.

4. The electromagnetic signal data processing method according to claim 3, characterized in that: Generating input features according to the multiple time-frequency image blocks and the positions of the time-frequency image blocks specifically includes: Using multiple convolution kernels to perform multiple convolution processes on the time-frequency image block to obtain amplitude features of the time-frequency image block; Using a conversion function to convert the position coordinates of the time-frequency image block to obtain the position features of the time-frequency image block; Calculating the sum of the amplitude feature of the time-frequency image block and the position feature of the time-frequency image block to obtain a fusion feature of the time-frequency image block; The fused features of all time-frequency blocks are concatenated to generate the input features.

5. The electromagnetic signal data processing method according to claim 4, characterized in that: The position coordinates of the time-frequency image block are converted using a conversion function to obtain the position features of the time-frequency image block, specifically including: Using a horizontal coordinate conversion function to convert the horizontal coordinate of the time-frequency image block to obtain the horizontal coordinate feature of the time-frequency image block; Using a ordinate conversion function to convert the ordinate of the time-frequency image block to obtain a ordinate feature of the time-frequency image block; The position feature of the time-frequency image block is obtained by concatenating the horizontal coordinate feature and the vertical coordinate feature of the time-frequency image block.

6. The electromagnetic signal data processing method according to claim 5, characterized in that: The using an encoder to encode the input features to obtain low-dimensional point cloud features specifically includes: Using a first-level encoding module to encode the input feature to obtain a first intermediate encoding feature, using a second-level encoding module to process the first intermediate encoding feature to obtain a second intermediate encoding feature, using a third-level encoding module to process the second intermediate encoding feature to output a third intermediate encoding feature, and using a fourth-level encoding module to process the third intermediate encoding feature to output the low-dimensional point cloud feature; Each of the encoding modules includes a multi-head attention mechanism layer and a feedforward neural network layer. In each of the encoding modules, the multi-head attention mechanism layer processes the input features of the encoding module, and the feedforward neural network layer processes the output data of the multi-head attention mechanism layer.

7. The electromagnetic signal data processing method according to any one of claims 1 to 6, characterized in that: Using a decoder to decode the low-dimensional point cloud features to obtain point cloud data of the components to be assembled specifically includes: Using a first multi-layer perceptron layer to decode the low-dimensional point cloud feature to obtain first decoded data; the data dimension of the low-dimensional point cloud feature is smaller than the data dimension of the first decoded data; Using a second multi-layer perceptron layer to decode the first decoded data to obtain second decoded data; the data dimension of the second decoded data is twice the data dimension of the first decoded data; Using a third multi-layer perceptron layer to decode the second decoded data to obtain third decoded data; the data dimension of the second decoded data is greater than the data dimension of the third decoded data; The third decoded data is decoded using a fourth multi-layer perceptron layer to obtain the point cloud data.

8. An assembly robot, characterized in that: The assembly robot is provided with a detection radar; Scan the component to be assembled using the detection radar according to a preset scanning path, and obtain an echo electromagnetic signal returned by the scanning electromagnetic signal emitted by the detection radar when it encounters the component to be assembled; Mixing the echo electromagnetic signal with the scanning electromagnetic signal to obtain an intermediate frequency signal, performing wavelet transform on the intermediate frequency signal to obtain a time-frequency signal, and converting the time-frequency signal into a time-frequency image; Divide the time-frequency image into blocks to obtain a plurality of time-frequency blocks and positions of the time-frequency blocks, and generate input features according to the plurality of time-frequency blocks and positions of the time-frequency blocks; Using an encoder to encode the input features to obtain low-dimensional point cloud features; wherein the low-dimensional point cloud features are used to represent position information and angle information; The decoder is used to decode the low-dimensional point cloud features to obtain point cloud data of the components to be assembled.

9. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images