Space collision warning method and device

Abstract

The embodiment of the application provides a kind of space collision early warning method and device, method includes: through the three-dimensional coordinate of target space, object speed and object pose information of multi-modal sensor acquisition, the multi-modal data is normalized, filtering and alignment processing, obtain space-time sequence data;Space-time sequence data is input into convolution network to extract space-time feature of target, and space-time feature is input into bidirectional LSTM network to extract time series enhancement feature, construct fully connected layer to map space-time feature and time series enhancement feature to the same spatial dimension, attention weight matrix is constructed using multi-head attention mechanism in fully connected layer, according to the instantaneous motion characteristic of space-time feature, the degree of motion intensity is evaluated, according to the motion state score after evaluation, attention weight matrix is corrected, to obtain attention weight value, based on the value, feature fusion is obtained to obtain fusion feature, the collision probability of target space is evaluated and early warning is carried out, the precision and efficiency of space collision early warning can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a space collision early warning method and device. Background Technology

[0002] With the increasing frequency of space exploration and utilization, space collision early warning technology has become crucial for ensuring spacecraft safety and maintaining the stability of the space environment. Traditionally, space collision early warning relies on precise physical models, such as orbital mechanics equations. These models predict the trajectory of spacecraft based on a series of assumptions. However, with the increasing complexity of the space environment, multibody disturbances (such as atmospheric drag and nonlinear changes in solar radiation pressure) limit the prediction accuracy of traditional physical models, leading to significant long-term accumulation of prediction errors.

[0003] To overcome these limitations, researchers have begun exploring the application of neural network models in space collision early warning. Single neural network models, such as Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs), have improved prediction accuracy to some extent. However, each of these models has its own limitations. Furthermore, when faced with unknown space targets (such as sudden disintegration debris) or non-cooperative targets (such as spacecraft with irregular orbital changes), existing models often suffer from overfitting or missed detections, exhibiting weak generalization ability.

[0004] In summary, existing space collision warning technologies have significant shortcomings in terms of algorithm limitations, real-time performance, generalization ability, and data utilization. There is an urgent need for a space collision warning method to improve the accuracy and real-time performance of space collision warnings, providing stronger protection for space safety. Summary of the Invention

[0005] To address the problems in the prior art, this application provides a space collision early warning method and apparatus, which can improve the accuracy and efficiency of space collision early warning.

[0006] To solve at least one of the above problems, this application provides the following technical solution:

[0007] Firstly, this application provides a space collision early warning method, including:

[0008] Raw spatial data of a space target is acquired by a multimodal sensor. The raw spatial data is then normalized, noise filtered, and time aligned to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensor includes a radar, an optical sensor, and a satellite navigation receiving system. The raw spatial data includes three-dimensional spatial coordinates, object velocity, and object attitude information.

[0009] The standardized spatiotemporal sequence data is input into a preset convolutional network. Feature extraction is performed by synchronously sliding multiple convolutional kernels in the time dimension and multiple spatial dimensions to determine the spatiotemporal features of the corresponding spatial target. The spatiotemporal features are flattened by time step and converted into temporal input data and input into a preset bidirectional LSTM network. Temporal dependencies are modeled by forward LSTM and backward LSTM respectively to determine the corresponding temporal enhancement features.

[0010] The spatiotemporal features and the temporal enhancement features are mapped to the same spatial dimension through a fully connected layer. The instantaneous motion features of the spatiotemporal features are extracted by global average pooling, and the motion intensity score is calculated. A multi-head attention mechanism is used to calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features to determine the corresponding initial attention weight matrix. The initial attention weight matrix is ​​then adjusted according to the motion intensity score to determine the corresponding adjusted attention weights. The spatiotemporal features and the temporal enhancement features are fused according to the attention weights to determine the corresponding fused features. Based on the fused features, a collision probability distribution of the target space is generated by feature sampling according to a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, a warning signal is output to issue a collision warning.

[0011] Furthermore, the acquisition of raw spatial data of space targets through multimodal sensors includes:

[0012] The target's three-dimensional spatial coordinates and velocity are obtained using radar.

[0013] The object's attitude information in the target space is acquired through optical sensors;

[0014] Global positioning data of the target space is obtained through a satellite navigation receiving system.

[0015] Further, the normalization, noise filtering, and time alignment processing of the original spatial data to determine the corresponding standardized spatiotemporal sequence data includes:

[0016] A unified spatiotemporal reference system is established, and the raw spatial data collected by each modal sensor is transformed to the same coordinate system. The linear normalization formula is used to map the raw data of each modality to the [0,1] interval, and the normalized data corresponding to each modality is determined.

[0017] The normalized data is filtered for noise using a Kalman filter multi-sensor data fusion algorithm. A consistency verification model for multimodal data is constructed based on Mahalanobis distance. Outlier detection is performed on the noise-filtered multimodal data according to the consistency verification model, and the detected outlier data is removed to determine the verification data corresponding to the original spatial data of each modality.

[0018] Based on the verification data from radar and satellite navigation receiving systems, the verification data from optical sensors is aligned by linear interpolation. When a missing data is detected, the verification data from the optical sensors is completed using the quaternion spherical linear interpolation method to determine the corresponding standardized spatiotemporal sequence data.

[0019] Further, the step of inputting the standardized spatiotemporal sequence data into a preset convolutional network, and extracting features by synchronously sliding multiple convolutional kernels across the time and multiple spatial dimensions to determine the spatiotemporal features of the corresponding spatial target includes:

[0020] The standardized spatiotemporal sequence data is input into a preset 3D convolutional network. Multiple layers of 3D convolutional kernels slide synchronously in the time and space dimensions to extract convolutional response vectors. The convolutional response vectors are used to characterize the motion pattern and relative position features of spatial targets.

[0021] The convolutional response vector is subjected to batch normalization and time-dimension-preserving max pooling operations to determine the spatiotemporal features of the corresponding spatial target, wherein the spatiotemporal features of the spatial target are represented in the form of a high-dimensional feature tensor.

[0022] Further, the step of flattening the spatiotemporal features into temporal input data by time step and inputting it into a preset bidirectional LSTM network, and modeling temporal dependencies through forward LSTM and backward LSTM respectively to determine the corresponding temporal enhancement features includes:

[0023] The spatiotemporal features are flattened into two-dimensional sequence data according to time steps, while preserving the correspondence between time steps and feature channels;

[0024] The flattened sequence data is input into a preset bidirectional LSTM network. The forward LSTM models historical dependencies in chronological order, and the reverse LSTM models potential future associations in reverse chronological order. Finally, the bidirectional hidden states are spliced ​​together to determine the corresponding temporal enhancement features.

[0025] Furthermore, the step of extracting the instantaneous motion features of the spatiotemporal features through global average pooling and calculating the motion intensity score includes:

[0026] Perform global average pooling on the spatiotemporal features along the time dimension to determine the corresponding pooled feature vector;

[0027] Within the fully connected layer, a motion intensity score is generated based on the pooled feature vector and a preset sigmoid activation function.

[0028] Further, the step of using a multi-head attention mechanism to calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features, determining the corresponding initial attention weight matrix, and adjusting the initial attention weight matrix according to the motion intensity score to determine the corresponding adjusted attention weights includes:

[0029] A multi-head attention mechanism is adopted, with the spatiotemporal features as query vectors and the temporal enhancement features as key vectors and value vectors, to determine the corresponding initial attention weight matrix;

[0030] Set a threshold for classifying motion states, and determine the motion state based on the threshold and the motion intensity score. If the motion state is determined to be a violent motion state, increase the weight of the temporal enhancement feature and determine the corresponding adjusted attention weight.

[0031] If the motion is determined to be in a steady state, the weight of the spatiotemporal features is increased, and the corresponding adjusted attention weight is determined.

[0032] Secondly, this application provides a space collision early warning device, comprising:

[0033] The multimodal data preprocessing module is used to collect raw spatial data of space targets through multimodal sensors, and to perform normalization, noise filtering and time alignment processing on the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems, and the raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information.

[0034] The spatiotemporal feature extraction module is used to input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multiple convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model the temporal dependency relationship by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features;

[0035] The spatiotemporal feature fusion module is used to map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer. It extracts the instantaneous motion features of the spatiotemporal features using global average pooling and calculates the motion intensity score. A multi-head attention mechanism is used to calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features, determining the corresponding initial attention weight matrix. The initial attention weight matrix is ​​then adjusted based on the motion intensity score to determine the adjusted attention weights. Feature fusion is performed on the spatiotemporal features and the temporal enhancement features based on the attention weights to determine the corresponding fused features. A collision probability distribution of the target space is generated based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, a warning signal is output for collision warning.

[0036] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the space collision warning method.

[0037] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the space collision warning method described above.

[0038] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the space collision warning method.

[0039] As can be seen from the above technical solution, this application provides a space collision early warning method and device. It collects the three-dimensional coordinates, object velocity, and object pose information of the target space using multimodal sensors. The multimodal data is normalized, filtered, and aligned to obtain spatiotemporal sequence data. This spatiotemporal sequence data is then input into a convolutional network to extract the spatiotemporal features of the spatial target. These features are then input into a bidirectional LSTM network to extract temporal enhancement features. A fully connected layer is constructed to map the spatiotemporal features and temporal enhancement features to the same spatial dimension. A multi-head attention mechanism is used in the fully connected layer to construct an attention weight matrix. The intensity of motion is evaluated based on the instantaneous motion features of the spatiotemporal features. The attention weight matrix is ​​then corrected based on the evaluated motion state score to obtain attention weight values. Based on these values, feature fusion is performed to obtain fused features. The collision probability of the target space is evaluated, and an early warning is issued, thereby improving the accuracy and efficiency of space collision early warning. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts illustrating the space collision warning method in the embodiments of this application;

[0042] Figure 2 This is a structural diagram of the space collision warning device in the embodiments of this application;

[0043] Figure 3 This is a schematic diagram of the structure of the electronic device in the embodiments of this application.

[0044] Figure label:

[0045] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0048] Considering the limitations of individual neural network models in space collision early warning, and the insufficient ability of fused neural network models to process multimodal data, this application provides a space collision early warning method and apparatus. It collects three-dimensional coordinates, object velocity, and object pose information in the target space using multimodal sensors. The multimodal data is normalized, filtered, and aligned to obtain spatiotemporal sequence data. This spatiotemporal sequence data is input into a convolutional network to extract the spatiotemporal features of the spatial target. These features are then input into a bidirectional LSTM network to extract temporal enhancement features. A fully connected layer is constructed to map the spatiotemporal features and temporal enhancement features to the same spatial dimension. A multi-head attention mechanism is used in the fully connected layer to construct an attention weight matrix. The intensity of motion is evaluated based on the instantaneous motion characteristics of the spatiotemporal features. The attention weight matrix is ​​then corrected based on the evaluated motion state score to obtain attention weight values. Based on these values, feature fusion is performed to obtain fused features. The collision probability of the target space is evaluated, and an early warning is issued, thereby improving the accuracy and efficiency of space collision early warning.

[0049] To improve the accuracy and efficiency of space collision early warning, this application provides an embodiment of a space collision early warning method, see [link to embodiment]. Figure 1 The space collision early warning method specifically includes the following:

[0050] Step S101: Collect raw spatial data of the space target through multimodal sensors, normalize, filter noise and align the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems. The raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information.

[0051] Optionally, in this embodiment, this step is a data acquisition and preprocessing step. In order to realize spatial collision early warning based on multi-source sensor multimodal data, this step uses multi-source data acquisition, unified dataset spatial system, normalization, noise filtering, data alignment and other means to standardize multimodal data into spatiotemporal sequence data, laying the foundation for subsequent data processing through models.

[0052] Optionally, in this embodiment, to perform collision warning for moving objects in space, we first need to collect spatial coordinates and the velocity of the moving object to predict its trajectory. To provide more accurate warnings, we also collect the pose image of the moving object.

[0053] Specifically, a high-precision radar system is used to measure the target's three-dimensional spatial coordinates (x, y, z) and velocity (vx, vy, vz) through electromagnetic wave reflection. The radar data is characterized by all-weather operation and high precision, making it suitable for long-range target monitoring.

[0054] Specifically, high-resolution optical sensors are used to acquire the target's attitude information, and image processing techniques are used to extract the target's rotation state. Optical data can provide more refined attitude features during close-range observation. The attitude information can be Euler angles or quaternions.

[0055] Understandably, in close-range encounter scenarios, the target's attitude directly affects the collision probability calculation. Optical data can avoid misjudgments caused by unknown attitude. Furthermore, for targets where radar is ineffective, radar and GNSS may not be able to acquire valid signals, while optical sensors can achieve target detection and tracking through visual features (such as shape and texture).

[0056] Specifically, global positioning data of the target is obtained through the Global Navigation Satellite System (GNSS data), providing high-precision absolute position reference and enhancing data reliability.

[0057] Optionally, in this embodiment, after obtaining the above-mentioned multi-source data, data normalization and preprocessing are performed.

[0058] Specifically, radar, optical, and GNSS data are converted to the same coordinate system to ensure data spatial consistency. Preferably, the coordinate system chosen is the geocentric inertial coordinate system (ECI), the orbital coordinate system, or the ECEF coordinate system.

[0059] Optionally, in this embodiment, a linear normalization formula is applied to the data for each modality:

[0060]

[0061] Here, dmin and dmax represent the minimum and maximum values ​​of the data in this dimension, respectively. After normalization, the data is mapped to the interval [0,1].

[0062] Normalization eliminates the dimensional differences between different sensor data, avoids the interference of numerical range on model training, and improves the stability and convergence speed of subsequent neural network training.

[0063] Optionally, in this embodiment, a Kalman filter algorithm based on multi-sensor fusion is employed to iteratively optimize the data through state equations and observation equations. The prediction phase estimates the state based on a motion model, while the update phase incorporates sensor observations to correct errors and reduce the noise impact of data from each sensor.

[0064] Specifically, the multi-sensor fusion Kalman filter algorithm is an extended Kalman filter (EKF) method that improves data accuracy and robustness by integrating observation data from multiple independent sensors, combining them with a target motion model, and iteratively optimizing state estimation.

[0065] In this scheme, the Extended Kalman Filter (EKF) is used as the framework, and the radar, GNSS and optical observation vectors are used as external observations. The state vector is selected as the target's six-dimensional position, velocity and attitude.

[0066] In the prediction phase, the state at the next time step is extrapolated based on the dynamic model; in the update phase, the state estimate is jointly updated using the observations from each sensor.

[0067] Optionally, in this embodiment, to detect and remove abnormal observations, a Mahalanobis distance test is constructed. For each time-varying multimodal observation vector, its Mahalanobis distance with the predicted observation vector is calculated. If the distance exceeds a threshold, the observation is determined to be abnormal and discarded directly. Through the above filtering and removal, highly reliable verification data is obtained.

[0068] Optionally, in this embodiment, the verification data of radar and GNSS are used as the time reference, and their sampling points are strictly aligned; the optical sensor requires interpolation alignment due to its low frame rate and the risk of frame loss. If missing optical attitude data is detected, quaternion spherical linear interpolation (Slerp) is used to smoothly fill in the missing data on the time axis, maintaining the continuity of rotational motion, ensuring strict synchronization of multimodal data, avoiding feature extraction deviations caused by temporal misalignment, and obtaining complete spatiotemporal sequence data after filling in the missing data.

[0069] This step involves collaborative acquisition from multiple sensors to cover all dimensions of the target's motion parameters. Data is normalized to unify the scale, filtered, and anomaly detected to improve data reliability. Time alignment and completion ensure spatiotemporal sequence consistency. Through the above data processing, high-quality, low-noise, and spatiotemporally aligned input data is provided for subsequent CNN-RNN models, laying the foundation for the accuracy of the collision warning algorithm.

[0070] Step S102: Input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multi-layer convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model the temporal dependency relationship by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features;

[0071] Optionally, in this embodiment, in the space collision warning task, multimodal sensor data has both spatial sparsity and temporal continuity, which cannot be taken into account by a single model. This solution uses a CNN-RNN hybrid architecture to jointly model geometric and dynamic characteristics and extract rich motion features for subsequent collision warning.

[0072] Optionally, in this embodiment, the preprocessed spatiotemporal sequence data is input into a 3D convolutional network.

[0073] Assume the input data is a spatiotemporal sequence data block with the following shape:

[0074] [Time step, spatial dimension 1, spatial dimension 2, spatial dimension 3, number of channels]

[0075] 3D convolution operations are performed simultaneously in the time step and spatial dimension. The convolution kernel size is set to (kt,ks1,ks2,ks3,cin)×cout, where kt is the convolution kernel size in the time dimension, ks1, ks2, and ks3 are the convolution kernel sizes in each spatial dimension, cin is the number of input channels, and cout is the number of output channels.

[0076] The formula for calculating convolution is as follows:

[0077]

[0078] Where Y is the convolution output, X is the input data, W is the convolution kernel weight, b is the bias term, and σ is the activation function.

[0079] The convolutional kernel slides synchronously across the temporal and three spatial dimensions. Each sliding window performs a weighted summation of the values ​​within the local spatiotemporal cube and activates them using an activation function to extract local spatiotemporal features. During the sliding process, the convolutional kernel calculates the convolutional response vector at each location, which reflects the motion pattern of the spatial target (such as velocity changes and acceleration) and its relative position features (such as distance and rate of change of angle).

[0080] The response vector output by the convolution is standardized to stabilize the training process and avoid gradient vanishing or exploding. After all sliding windows are completed, downsampling is performed in the spatial dimensions (height, width, depth) to reduce computation while preserving the continuity of the temporal dimension and ensuring the integrity of temporal information. The final output is a spatial feature representation with higher-level semantic information, represented in the form of a high-dimensional feature tensor with the shape: [batch, time step, feature channel].

[0081] Optionally, in this embodiment, the spatial feature sequence extracted by the CNN is input into a bidirectional LSTM network. The bidirectional LSTM includes a forward LSTM and a backward LSTM, which can utilize both past and future temporal information simultaneously.

[0082] In this step, the high-dimensional feature tensor [batch, time step, feature channel] output by 3D convolution is flattened into two-dimensional sequence data ([batch × time step, feature channel]) according to the time step, preserving the correspondence between the time step and the feature to adapt to the input format of RNN. In this scheme, the RNN adopts a bidirectional LSTM network.

[0083] Next, the bidirectional LSTM processing procedure.

[0084] Forward LSTM processes sequences in time step order (t = 1 → T), models historical dependencies, and captures the gradual motion trend of the target (such as continuous acceleration or orbit change).

[0085] Inverse LSTM processes sequences in reverse time step order (t = T → 1) to model potential future associations and capture sudden motion changes (such as emergency braking or collision avoidance actions).

[0086] The final hidden states of the forward LSTM and the backward LSTM are concatenated along the feature dimension to form a temporal augmentation feature with dimensions of [batch, hidden layer dimension × 2], which integrates temporal information from the past and the future.

[0087] In this step, spatial collision is essentially a spatiotemporal coupled event, requiring joint modeling of geometric and dynamic characteristics. The 3D convolutional kernel slides synchronously in the time and space dimensions to directly capture short-term motion patterns. Regardless of where the target appears in the sensor's field of view, the CNN can stably extract features with the same semantics. The spatial dimension is gradually compressed through pooling layers to retain key features (such as max pooling to highlight "nearest distance"), reducing the computational burden on subsequent RNNs. The LSTM selectively retains long-term key information (such as the trend of "target continuously moving away") through a gating mechanism, filtering noise. It is irreplaceable in scenarios of unknown target warning and low-latency response.

[0088] Step S103: Map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer. Extract the instantaneous motion features of the spatiotemporal features using global average pooling and calculate the motion intensity score. Calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features using a multi-head attention mechanism to determine the corresponding initial attention weight matrix. Adjust the weights of the initial attention weight matrix according to the motion intensity score to determine the corresponding adjusted attention weights. Perform feature fusion on the spatiotemporal features and the temporal enhancement features based on the attention weights to determine the corresponding fused features. Generate a collision probability distribution of the target space based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, output a warning signal for collision warning.

[0089] Optionally, in this embodiment, the spatiotemporal features extracted by the CNN and the temporal enhancement features output by the RNN are input into the fully connected layer and mapped to the same dimensional space to ensure consistent feature scale.

[0090] Optionally, in this embodiment, global average pooling (GAP) is performed on the spatiotemporal features along the time dimension to compress the time dimension information and generate a pooled feature vector. The pooled feature vector is input into a fully connected layer, and the motion intensity score Smotion∈[0,1] of the CNN features is extracted through global average pooling. The calculation formula is as follows:

[0091] S motion =σ(W m ·Pool global (F CNN )+b m )

[0092] Among them, W m and b m Here are the trainable parameters, and σ is the Sigmoid function;

[0093] Optionally, in this embodiment, a multi-head attention mechanism is used, where the spatiotemporal features output by the CNN are split into multiple heads, which are used as query vectors respectively. Similarly, the temporal enhancement features output by the RNN are split into corresponding heads, which are used as key vectors and value vectors respectively. The dot product of each query-key pair is calculated and scaled, and then the initial attention weight matrix of each head is obtained through Softmax: α_initial∈[0,1]^{T×T}

[0094] The initial α_initial is dynamically adjusted based on the Smotion score, which indicates the intensity of the activity, to generate the final attention weight matrix α_final.

[0095] αfinal=αinitial·Smotion+(1-Smotion)·αbase

[0096] Here, αbase is a preset base weight matrix used to ensure feature stability during smooth motion.

[0097] Optionally, in this embodiment, the process of adjusting the weight matrix is ​​as follows:

[0098] A threshold for classifying motion states is pre-set. If the motion intensity score is greater than or equal to the threshold, it is considered a violent motion state. In this case, in the multi-head attention weights, the weights corresponding to RNN features in all heads are multiplied by a gain coefficient greater than 1, and the weights corresponding to CNN features are multiplied by a decay coefficient less than 1, making the fusion more dependent on temporal enhancement features.

[0099] If the intensity score of the motion is less than the threshold, it is determined to be a stable motion state. In this case, the multi-head attention weights are adjusted in reverse: the weights corresponding to CNN features are multiplied by a gain coefficient, and the weights corresponding to RNN features are multiplied by a decay coefficient, making the fusion more dependent on spatiotemporal features. The adjusted attention weight matrix is ​​the final weight, used for subsequent feature fusion.

[0100] The spatiotemporal features and temporal enhancement features are weighted and summed according to the adjusted attention weights to generate a joint spatiotemporal feature F. Based on the fused feature, Monte Carlo simulation is used to randomly sample future trajectory points to generate a collision probability distribution. The collision probability is calculated to see if it exceeds a preset threshold; if it does, a warning signal is triggered.

[0101] Specifically, collision probability calculation:

[0102] Based on the fused features F, the collision probability P_collision is calculated using a pre-trained collision probability calculation model (such as a logistic regression model or a neural network model). For example, when using a logistic regression model, the formula is as follows:

[0103] P collision =σ(W f F+b f )

[0104] Among them, W f b represents the model weights. f σ is the bias term, and σ is the sigmoid function.

[0105] The calculated collision probability is compared with a set threshold. If the collision probability is greater than the threshold, an obstacle avoidance command is triggered; otherwise, no obstacle avoidance measures are taken.

[0106] This example demonstrates how this embodiment preprocesses multi-source data in space, unifies spatial coordinates, eliminates data noise by combining Kalman and Mahalanobis distance outlier algorithms, achieves sequential fusion of multi-source data, extracts features from multi-source spatial data based on CNN+RNN networks, and combines Monte Carlo simulation for collision probability prediction.

[0107] As described above, the space collision warning method provided in this application can collect the three-dimensional coordinates, object velocity, and object pose information of the target space through multimodal sensors, normalize, filter, and align the multimodal data to obtain spatiotemporal sequence data; input the spatiotemporal sequence data into a convolutional network to extract the spatiotemporal features of the spatial target, and input the spatiotemporal features into a bidirectional LSTM network to extract temporal enhancement features; construct a fully connected layer to map the spatiotemporal features and temporal enhancement features to the same spatial dimension; construct an attention weight matrix in the fully connected layer using a multi-head attention mechanism; evaluate the intensity of motion based on the instantaneous motion features of the spatiotemporal features; correct the attention weight matrix based on the evaluated motion state score to obtain attention weight values; perform feature fusion based on these values ​​to obtain fused features; evaluate the collision probability of the target space and issue a warning, thereby improving the accuracy and efficiency of space collision warning.

[0108] In one embodiment of the space collision warning method of this application, it may further include the following:

[0109] Step S201: Obtain the target's three-dimensional spatial coordinates and velocity using radar;

[0110] Step S202: Acquire object attitude information in the target space using an optical sensor;

[0111] Step S203: Obtain global positioning data of the target space through a satellite navigation receiving system.

[0112] Optionally, in this embodiment, raw information about the space target is acquired in real time from different types of sensors.

[0113] Specifically, the radar transmits millimeter-wave pulses towards the target within a fixed sampling period. By measuring the round-trip time difference and phase difference, it directly obtains the target's radial range, azimuth, elevation angle, and radial velocity. Subsequently, it uses a rotation matrix from the radar coordinate system to the Earth-fixed coordinate system (ECEF) to convert the polar coordinates into three-dimensional rectangular coordinates (x, y, z) and velocity vectors (vx, vy, vz). The radar output frequency is high, reaching 10-100 Hz, thus providing high-time-resolution dynamic information.

[0114] Specifically, a high frame rate optical camera continuously captures visible light or infrared images of the target. A star point extraction algorithm is used to locate the target's light spot on the image plane. Combining the camera's intrinsic and extrinsic parameter matrices, the image plane coordinates are calculated into the target's azimuth, pitch, and roll angles in the J2000 inertial frame. Using the stellar background or attitude reference provided by a gyroscope, the angle information is further converted into quaternions q = (q0, q1, q2, q3). The optical sensor has a sampling rate of 1-10Hz, providing higher spatial resolution than radar, enabling accurate estimation of the target's attitude and motion.

[0115] Preferably, the star point extraction algorithm is either the centroid method or the Gaussian fitting method.

[0116] Specifically, the GNSS receiver of the satellite navigation receiving system locks onto signals from four or more satellites and uses pseudorange and carrier phase difference techniques to calculate the three-dimensional coordinates (x, y, z) of the target antenna phase center in the WGS-84 coordinate system. Simultaneously, it outputs the velocity (vx, vy, vz) based on Doppler frequency shift. Absolute positioning accuracy can reach the centimeter level, providing a global absolute reference.

[0117] In this embodiment, radar provides high-precision motion parameters, optical sensors supplement attitude information, and GNSS ensures global positioning consistency. Through multi-source data acquisition, accurate monitoring of the target's motion state is achieved.

[0118] Through step S203, this embodiment successfully achieves multi-source data acquisition, and improves the robustness of target monitoring by complementing multi-source data.

[0119] In one embodiment of the space collision warning method of this application, it may further include the following:

[0120] Step S301: Establish a unified spatiotemporal reference system, transform the raw spatial data collected by each modal sensor to the same coordinate system, and use the linear normalization formula to map the raw data of each modality to the [0,1] interval to determine the normalized data corresponding to each mode.

[0121] Step S302: Noise filtering is performed on each of the normalized data according to the Kalman filter multi-sensor data fusion algorithm. A consistency verification model for multimodal data is constructed based on Mahalanobis distance. Outlier detection is performed on the noise-filtered multimodal data according to the consistency verification model, and the detected outlier data is removed to determine the verification data corresponding to the original spatial data of each modality.

[0122] Step S303: Using the verification data from the radar and satellite navigation receiving system as a reference, the verification data from the optical sensor is aligned by linear interpolation. When a missing data is detected, the verification data of the optical sensor is completed by using the quaternion spherical linear interpolation method to determine the corresponding standardized spatiotemporal sequence data.

[0123] Optionally, in this embodiment, this step performs data preprocessing on the multi-source data collected in step S202 above, laying the foundation for subsequent model applications.

[0124] Optionally, in this embodiment, the ECEF coordinate system is first used as a common spatiotemporal reference system to establish a rotation matrix from the radar local coordinate system, the optical sensor inertial system to the ECEF; all three types of original coordinates are transformed to the ECEF so that all data are in the same coordinate frame.

[0125] Optionally, in this embodiment, linear normalization is performed for each dimension d:

[0126]

[0127] Here, dmin and dmax are the minimum and maximum values ​​of the data in this dimension, respectively. Mapping the original value d to the interval [0,1] eliminates differences in different units and numerical ranges, avoiding gradient explosion or slow convergence during subsequent network training.

[0128] Optionally, in this embodiment, Kalman filtering is used to filter noise in the normalized data for each dimension. This scheme employs multi-sensor fusion Kalman filtering, which significantly improves the accuracy of spatial target state estimation by integrating observation data from multiple independent sensors and combining them with a target motion model.

[0129] Specifically, we define the state equation and the observation equation.

[0130] 1. State equations (motion model), using orbital dynamics model:

[0131] xk=Axk-1+Buk-1+wk-1

[0132] in,

[0133] xk: State vector (e.g., position, velocity, attitude);

[0134] A: State transition matrix (derived from the physical model);

[0135] Wk-1: Process noise (follows a Gaussian distribution).

[0136] 2. Observation equations (sensor model): Construct the relationship between the observations and the state vector of each sensor (radar, optical, GNSS):

[0137]

[0138] in,

[0139] The observation value of the i-th sensor (such as radar ranging, optical attitude angle);

[0140] H (i) Observation matrix (maps the state to the sensor output);

[0141] Observation noise (sensor-specific).

[0142] Specifically, the prediction phase uses a motion model to predict the state and error covariance at the next moment.

[0143]

[0144] P k|k-1 =AP k-1|k-1 A T +Q

[0145] in,

[0146] Prior state estimation;

[0147] P k|k-1 Prior error covariance;

[0148] Q: Process noise covariance matrix.

[0149] Specifically, during the update phase, when the observation data from the i-th sensor arrives, the data is fused according to the following steps:

[0150] 1. Calculate the Kalman gain:

[0151]

[0152] Among them, R (i) : Observation noise covariance of the i-th sensor

[0153] 2. Update state estimate:

[0154]

[0155] 3. Update the error covariance:

[0156]

[0157] By following the steps described above, the observations of each sensor are processed sequentially, and the output of the previous update is used as the input for the next sensor.

[0158] Optionally, in this embodiment, the rationality of the observed data is verified by Mahalanobis distance after the Kalman filter described above.

[0159] Mahalanobis distance model:

[0160]

[0161] S = H (i) P k|k-1 H (i)T +R (i)

[0162] If the value of D is greater than the distance threshold, it is judged as an outlier and discarded.

[0163] Optionally, in this embodiment, time alignment and missing value supplementation are performed on the sensor data after distance verification. The radar and GNSS verification data are used as the time reference, and their sampling points are strictly aligned. Due to the lower frame rate and risk of frame loss, optical sensor data requires interpolation alignment. For each missing time point t in the optical data, quaternion spherical linear interpolation is performed between the two preceding and following valid attitude quaternions q_prev and q_next. If two or more consecutive segments of optical data are missing, the attitude predicted by the Kalman filter is used as the supplementary value. Finally, the verification data of all modes form a continuous, synchronous, and anomaly-free standardized spatiotemporal sequence on a unified time axis, which can be directly used as the input tensor of the CNN+RNN network.

[0164] In this scheme, considering that radar data can provide high-precision range and velocity but is susceptible to multipath interference, and optical sensor data can capture fine attitude information but is dependent on lighting conditions, this scheme is designed to simultaneously acquire data from multiple sensors, forming a multi-source data complementarity enhancement strategy to accurately observe the motion state of the target object. To further improve the observation accuracy, a Kalman filter combining multi-sensor fusion with Mahalanobis distance outlier removal is adopted. Its core value lies in:

[0165] 1. Dynamically remove abnormal observations to prevent erroneous sensor data from contaminating state estimation.

[0166] 2. Adaptively adjust the filter confidence level and dynamically adjust the Kalman gain based on data reliability.

[0167] 3. Improve the system's fault tolerance, and maintain stable output even when some sensors fail.

[0168] Understandably, the Kalman gain dynamically adjusts the contribution weights of each sensor based on the sensor noise level. Observations from high-noise sensors are automatically downweighted, while data from high-precision sensors are given higher confidence. Through a "prediction-update" cycle, the state estimate is gradually corrected, suppressing random noise (such as radar ranging jitter and optical image noise) and short-term interference.

[0169] However, the limitation of Kalman filtering is that outliers may occur in real-world scenarios for the following reasons:

[0170] Temporary sensor failure (such as optical obstruction or loss of GNSS signal).

[0171] Environmental interference (such as electromagnetic pulses affecting radar data).

[0172] Data conflicts between multiple sensors (such as significant differences between radar and optical ranging).

[0173] Without an anomaly detection mechanism, erroneous data will be forcibly merged, causing the state estimate to deviate significantly from the true value.

[0174] Therefore, this scheme introduces an outlier detection mechanism based on Mahalanobis distance, and uses statistical significance tests to determine whether the observed data is reasonable.

[0175] For example, if a radar suddenly outputs a ranging value that is far from the predicted trajectory due to multipath effects, Mahalanobis distance will mark it as an anomaly to avoid using the data in the Kalman update phase.

[0176] Through step S303, this embodiment combines Kalman filtering and Mahalanobis distance outlier removal algorithms in a unified coordinate system to remove data noise from multi-source sensor data and achieve data alignment and imputation, laying a solid data foundation for subsequent data applications of neural network models.

[0177] In one embodiment of the space collision warning method of this application, it may further include the following:

[0178] Step S401: Input the standardized spatiotemporal sequence data into a preset 3D convolutional network, and extract the convolutional response vector by synchronously sliding multiple layers of 3D convolutional kernels in the time and spatial dimensions. The convolutional response vector is used to characterize the motion pattern and relative position features of the spatial target.

[0179] Step S402: Perform batch normalization and time-dimension-preserving max pooling operations on the convolutional response vector to determine the spatiotemporal features of the corresponding spatial target, wherein the spatiotemporal features of the spatial target are represented in the form of a high-dimensional feature tensor.

[0180] Optionally, in this embodiment, the preprocessed spatiotemporal sequence data is input into a 3D convolutional network.

[0181] Assume the input data is a spatiotemporal sequence data block with the following shape:

[0182] [Time step, spatial dimension 1, spatial dimension 2, spatial dimension 3, number of channels]

[0183] 3D convolution operations are performed simultaneously in the time step and spatial dimension. The convolution kernel size is set to (kt,ks1,ks2,ks3,cin)×cout, where kt is the convolution kernel size in the time dimension, ks1, ks2, and ks3 are the convolution kernel sizes in each spatial dimension, cin is the number of input channels, and cout is the number of output channels.

[0184] The formula for calculating convolution is as follows:

[0185]

[0186] Where Y is the convolution output, X is the input data, W is the convolution kernel weight, b is the bias term, and σ is the activation function.

[0187] The convolutional kernel slides synchronously across the temporal and three spatial dimensions. Each sliding window performs a weighted summation of the values ​​within the local spatiotemporal cube and activates them using an activation function to extract local spatiotemporal features. During the sliding process, the convolutional kernel calculates the convolutional response vector at each location, which reflects the motion pattern of the spatial target (such as velocity changes and acceleration) and its relative position features (such as distance and rate of change of angle).

[0188] The response vector output by the convolution is standardized to stabilize the training process and avoid gradient vanishing or exploding. After all sliding windows are completed, downsampling is performed in the spatial dimensions (height, width, depth) to reduce computation while preserving the continuity of the temporal dimension and ensuring the integrity of temporal information. The final output is a spatial feature representation with higher-level semantic information, represented in the form of a high-dimensional feature tensor with the shape: [batch, time step, feature channel].

[0189] Through step S402, this embodiment successfully captures the local motion patterns of spatial targets and the interaction features of multiple targets simultaneously by using the spatiotemporal synchronous sliding of 3D convolutional kernels. It also improves the training stability of the model and reduces redundant computation through batch normalization and pooling operations, making the features more discriminative.

[0190] In one embodiment of the space collision warning method of this application, it may further include the following:

[0191] Step S501: Flatten the spatiotemporal features into two-dimensional sequence data according to the time steps, and retain the correspondence between the time steps and the feature channels;

[0192] Step S502: Input the flattened sequence data into a preset bidirectional LSTM network. The forward LSTM models historical dependencies in the order of time steps, and the reverse LSTM models potential future associations in the reverse order of time steps. Finally, the bidirectional hidden states are spliced ​​together to determine the corresponding temporal enhancement features.

[0193] Optionally, in this embodiment, the spatial feature sequence extracted by the CNN is input into a bidirectional LSTM network. The bidirectional LSTM includes a forward LSTM and a backward LSTM, which can utilize both past and future temporal information simultaneously.

[0194] In this step, the high-dimensional feature tensor [batch, time step, feature channel] output by 3D convolution is flattened into two-dimensional sequence data ([batch × time step, feature channel]) according to the time step, preserving the correspondence between the time step and the feature to adapt to the input format of RNN. In this scheme, the RNN adopts a bidirectional LSTM network.

[0195] Next, the bidirectional LSTM processing procedure.

[0196] Forward LSTM processes sequences in time step order (t = 1 → T), models historical dependencies, and captures the gradual motion trend of the target (such as continuous acceleration or trajectory change);

[0197] Inverse LSTM processes sequences in reverse time step order (t = T → 1) to model potential future associations and capture sudden motion changes (such as emergency braking or collision avoidance actions).

[0198] The final hidden states of the forward LSTM and the backward LSTM are concatenated along the feature dimension to form a temporal augmentation feature with dimensions of [batch, hidden layer dimension × 2], which integrates temporal information from the past and the future.

[0199] 3D CNN extracts spatial features, and bidirectional LSTM models temporal dependencies. The combination of the two overcomes the limitations of a single model (such as pure CNN or pure RNN) and achieves more accurate collision risk prediction.

[0200] Through step S502, this embodiment successfully improves the robustness of trajectory prediction by jointly modeling historical and future information, which is especially suitable for non-cooperative targets (such as spacecraft with irregular orbit changes) or sudden debris scenarios. The flattening operation ensures that spatiotemporal features are seamlessly transitioned to the time series modeling stage, avoiding information loss.

[0201] In one embodiment of the space collision warning method of this application, it may further include the following:

[0202] Step S601: Perform global average pooling on the spatiotemporal features along the time dimension to determine the corresponding pooling feature vector;

[0203] Step S602: Within the fully connected layer, a motion intensity score is generated based on the pooled feature vector and the preset Sigmoid activation function.

[0204] Optionally, in this embodiment, global average pooling is performed on the spatiotemporal feature tensor output by the CNN along the time dimension. That is, the average is calculated over all time steps at each spatial-channel location to obtain a pooled feature vector with the dimension of "space × channel". This step compresses the temporal information into an instantaneous motion description while preserving the spatial structure.

[0205] By using a fully connected layer and a sigmoid function, pooling vectors are mapped to scalar scores that reflect the intensity of the exercise.

[0206] The intensity score of exercise, Smotion ∈ [0,1], is calculated using the following formula:

[0207] S motion =σ(W m ·Pool global (F CNN )+b m )

[0208] Among them, W m and b m Here are the trainable parameters, and σ is the Sigmoid function.

[0209] Through step S602, this embodiment successfully obtains the motion mechanism credit score, which is directly related to the target dynamics and supports subsequent logical judgment.

[0210] In one embodiment of the space collision warning method of this application, it may further include the following:

[0211] Step S701: Using a multi-head attention mechanism, with the spatiotemporal features as the query vector and the temporal enhancement features as the key vector and value vector, determine the corresponding initial attention weight matrix;

[0212] Step S702: Set a motion state classification threshold, and determine the motion state based on the motion state classification threshold and the motion intensity score. If it is determined to be a violent motion state, increase the weight of the temporal enhancement feature and determine the corresponding adjusted attention weight.

[0213] Step S703: If the motion is determined to be in a steady state, the weight of the spatiotemporal features is increased, and the corresponding adjusted attention weight is determined.

[0214] Optionally, in this embodiment, a multi-head attention mechanism is used, where the spatiotemporal features output by the CNN are split into multiple heads, which are used as query vectors respectively. Similarly, the temporal enhancement features output by the RNN are split into corresponding heads, which are used as key vectors and value vectors respectively. The dot product of each query-key pair is calculated and scaled, and then the initial attention weight matrix of each head is obtained through Softmax: α_initial∈[0,1]^{T×T}

[0215] The initial α_initial is dynamically adjusted based on the Smotion score, which indicates the intensity of the activity, to generate the final attention weight matrix α_final.

[0216] αfinal=αinitial·Smotion+(1-Smotion)·αbase

[0217] Here, αbase is a preset base weight matrix used to ensure feature stability during smooth motion.

[0218] Optionally, in this embodiment, the process of adjusting the weight matrix is ​​as follows:

[0219] A threshold for classifying motion states is pre-set. If the motion intensity score is greater than or equal to the threshold, it is considered a violent motion state. In this case, in the multi-head attention weights, the weights corresponding to RNN features in all heads are multiplied by a gain coefficient greater than 1, and the weights corresponding to CNN features are multiplied by a decay coefficient less than 1, making the fusion more dependent on temporal enhancement features.

[0220] If the intensity score of the motion is less than the threshold, it is determined to be a stable motion state. In this case, the multi-head attention weights are adjusted in reverse: the weights corresponding to CNN features are multiplied by a gain coefficient, and the weights corresponding to RNN features are multiplied by a decay coefficient, making the fusion more dependent on spatiotemporal features. The adjusted attention weight matrix is ​​the final weight, used for subsequent feature fusion.

[0221] Through step S703, this embodiment successfully dynamically adjusts the feature contribution according to the motion state, thereby improving the sensitivity to sudden motion (such as emergency track change).

[0222] To improve the accuracy and efficiency of space collision warning, this application provides an embodiment of a space collision warning device for implementing all or part of the aforementioned space collision warning method. See [link to embodiment]. Figure 2 The space collision early warning device specifically includes the following components:

[0223] The multimodal data preprocessing module 10 is used to collect raw spatial data of spatial targets through multimodal sensors, and to perform normalization, noise filtering and time alignment processing on the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems, and the raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information.

[0224] The spatiotemporal feature extraction module 20 is used to input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multi-layer convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model temporal dependencies by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features;

[0225] The spatiotemporal feature fusion module 30 is used to map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer, extract the instantaneous motion features of the spatiotemporal features through global average pooling and calculate the motion intensity score, calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features using a multi-head attention mechanism, determine the corresponding initial attention weight matrix, adjust the weights of the initial attention weight matrix according to the motion intensity score, determine the corresponding adjusted attention weights, perform feature fusion on the spatiotemporal features and the temporal enhancement features according to the attention weights, determine the corresponding fused features, and generate a collision probability distribution of the target space based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, a warning signal is output to issue a collision warning.

[0226] As described above, the space collision warning device provided in this application can collect three-dimensional coordinates, object velocity, and object pose information of the target space through multimodal sensors. It then normalizes, filters, and aligns the multimodal data to obtain spatiotemporal sequence data. The spatiotemporal sequence data is input into a convolutional network to extract the spatiotemporal features of the spatial target. These features are then input into a bidirectional LSTM network to extract temporal enhancement features. A fully connected layer is constructed to map the spatiotemporal features and temporal enhancement features to the same spatial dimension. A multi-head attention mechanism is used in the fully connected layer to construct an attention weight matrix. The intensity of motion is evaluated based on the instantaneous motion features of the spatiotemporal features. The attention weight matrix is ​​corrected based on the evaluated motion state score to obtain attention weight values. Feature fusion is then performed based on these values ​​to obtain fused features. The collision probability of the target space is evaluated, and a warning is issued, thereby improving the accuracy and efficiency of space collision warning.

[0227] From a hardware perspective, in order to improve the accuracy and efficiency of space collision warning, this application provides an embodiment of an electronic device for implementing all or part of the space collision warning method, wherein the electronic device specifically includes the following:

[0228] The system comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to transmit information between the space collision warning method and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the space collision warning method in the present embodiment, and the contents of the embodiments of the space collision warning method are incorporated herein, and repeated details will not be described again.

[0229] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0230] In practical applications, parts of the space collision warning method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0231] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0232] Figure 3 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 3 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 3 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0233] In one embodiment, the space collision warning method function can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0234] Step S101: Collect raw spatial data of the space target through multimodal sensors, normalize, filter noise and align the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems. The raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information.

[0235] Step S102: Input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multi-layer convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model the temporal dependency relationship by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features;

[0236] Step S103: Map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer. Extract the instantaneous motion features of the spatiotemporal features using global average pooling and calculate the motion intensity score. Calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features using a multi-head attention mechanism to determine the corresponding initial attention weight matrix. Adjust the weights of the initial attention weight matrix according to the motion intensity score to determine the corresponding adjusted attention weights. Perform feature fusion on the spatiotemporal features and the temporal enhancement features based on the attention weights to determine the corresponding fused features. Generate a collision probability distribution of the target space based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, output a warning signal for collision warning.

[0237] As described above, the electronic device provided in this application collects the three-dimensional coordinates, object velocity, and object pose information of the target space through a multimodal sensor. The multimodal data is normalized, filtered, and aligned to obtain spatiotemporal sequence data. This spatiotemporal sequence data is then input into a convolutional network to extract the spatiotemporal features of the spatial target. These features are then input into a bidirectional LSTM network to extract temporal enhancement features. A fully connected layer is constructed to map the spatiotemporal features and temporal enhancement features to the same spatial dimension. A multi-head attention mechanism is used in the fully connected layer to construct an attention weight matrix. The intensity of motion is evaluated based on the instantaneous motion features of the spatiotemporal features. The attention weight matrix is ​​then corrected based on the evaluated motion state score to obtain attention weight values. Feature fusion is performed based on these values ​​to obtain fused features. The collision probability of the target space is evaluated, and a warning is issued, thereby improving the accuracy and efficiency of spatial collision warning.

[0238] In another embodiment, the space collision warning method can be configured separately from the central processing unit 9100. For example, the space collision warning method can be configured as a chip connected to the central processing unit 9100, and the space collision warning method function can be implemented through the control of the central processing unit.

[0239] like Figure 3 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 3 All components shown; in addition, the electronic device 9600 may also include Figure 3 For components not shown, please refer to existing technologies.

[0240] like Figure 3As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0241] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0242] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0243] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0244] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0245] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which is the same as in a conventional mobile communication terminal.

[0246] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is also coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0247] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the space collision warning method with a server or client execution subject in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the space collision warning method with a server or client execution subject in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0248] Step S101: Collect raw spatial data of the space target through multimodal sensors, normalize, filter noise and align the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems. The raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information.

[0249] Step S102: Input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multi-layer convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model the temporal dependency relationship by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features;

[0250] Step S103: Map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer. Extract the instantaneous motion features of the spatiotemporal features using global average pooling and calculate the motion intensity score. Calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features using a multi-head attention mechanism to determine the corresponding initial attention weight matrix. Adjust the weights of the initial attention weight matrix according to the motion intensity score to determine the corresponding adjusted attention weights. Perform feature fusion on the spatiotemporal features and the temporal enhancement features based on the attention weights to determine the corresponding fused features. Generate a collision probability distribution of the target space based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, output a warning signal for collision warning.

[0251] As described above, the computer-readable storage medium provided in this application collects three-dimensional coordinates, object velocity, and object pose information of the target space through a multimodal sensor. The multimodal data is normalized, filtered, and aligned to obtain spatiotemporal sequence data. This spatiotemporal sequence data is then input into a convolutional network to extract the spatiotemporal features of the spatial target. These features are then input into a bidirectional LSTM network to extract temporal enhancement features. A fully connected layer is constructed to map the spatiotemporal features and temporal enhancement features to the same spatial dimension. A multi-head attention mechanism is used in the fully connected layer to construct an attention weight matrix. The intensity of motion is evaluated based on the instantaneous motion features of the spatiotemporal features. The attention weight matrix is ​​then corrected based on the evaluated motion state score to obtain attention weight values. Feature fusion is performed based on these values ​​to obtain fused features. The collision probability of the target space is evaluated, and an early warning is issued, thereby improving the accuracy and efficiency of spatial collision early warning.

[0252] Embodiments of this application also provide a computer program product capable of implementing all steps of the space collision warning method with the execution subject being a server or client in the above embodiments. When executed by a processor, this computer program / instruction implements the steps of the space collision warning method. For example, the computer program / instruction implements the following steps:

[0253] Step S101: Collect raw spatial data of the space target through multimodal sensors, normalize, filter noise and align the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems. The raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information.

[0254] Step S102: Input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multi-layer convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model the temporal dependency relationship by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features;

[0255] Step S103: Map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer. Extract the instantaneous motion features of the spatiotemporal features using global average pooling and calculate the motion intensity score. Calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features using a multi-head attention mechanism to determine the corresponding initial attention weight matrix. Adjust the weights of the initial attention weight matrix according to the motion intensity score to determine the corresponding adjusted attention weights. Perform feature fusion on the spatiotemporal features and the temporal enhancement features based on the attention weights to determine the corresponding fused features. Generate a collision probability distribution of the target space based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, output a warning signal for collision warning.

[0256] As described above, the computer program product provided in this application collects the three-dimensional coordinates, object velocity, and object pose information of the target space through a multimodal sensor. It then normalizes, filters, and aligns the multimodal data to obtain spatiotemporal sequence data. This spatiotemporal sequence data is input into a convolutional network to extract the spatiotemporal features of the spatial target. These features are then input into a bidirectional LSTM network to extract temporal enhancement features. A fully connected layer is constructed to map the spatiotemporal features and temporal enhancement features to the same spatial dimension. A multi-head attention mechanism is used in the fully connected layer to construct an attention weight matrix. The intensity of motion is evaluated based on the instantaneous motion features of the spatiotemporal features. The attention weight matrix is ​​then corrected based on the evaluated motion state score to obtain attention weight values. Feature fusion is performed based on these values ​​to obtain fused features. The collision probability of the target space is evaluated, and a warning is issued, thereby improving the accuracy and efficiency of spatial collision warning.

[0257] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0258] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0259] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0260] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0261] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A space collision early warning method, characterized in that, The method includes: Raw spatial data of a space target is acquired by a multimodal sensor. The raw spatial data is then normalized, noise filtered, and time aligned to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensor includes a radar, an optical sensor, and a satellite navigation receiving system. The raw spatial data includes three-dimensional spatial coordinates, object velocity, and object attitude information. The standardized spatiotemporal sequence data is input into a preset convolutional network. Feature extraction is performed by synchronously sliding multiple convolutional kernels in the time dimension and multiple spatial dimensions to determine the spatiotemporal features of the corresponding spatial target. The spatiotemporal features are flattened by time step and converted into temporal input data and input into a preset bidirectional LSTM network. Temporal dependencies are modeled by forward LSTM and backward LSTM respectively to determine the corresponding temporal enhancement features. The spatiotemporal features and the temporal enhancement features are mapped to the same spatial dimension through a fully connected layer. The instantaneous motion features of the spatiotemporal features are extracted by global average pooling, and the motion intensity score is calculated. A multi-head attention mechanism is used to calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features to determine the corresponding initial attention weight matrix. The initial attention weight matrix is ​​then adjusted according to the motion intensity score to determine the corresponding adjusted attention weights. The spatiotemporal features and the temporal enhancement features are fused according to the attention weights to determine the corresponding fused features. Based on the fused features, a collision probability distribution of the target space is generated by feature sampling according to a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, a warning signal is output to issue a collision warning.

2. The space collision early warning method according to claim 1, characterized in that, The acquisition of raw space data of space targets through multimodal sensors includes: The target's three-dimensional spatial coordinates and velocity are obtained using radar. The object's attitude information in the target space is acquired through optical sensors; Global positioning data of the target space is obtained through a satellite navigation receiving system.

3. The space collision early warning method according to claim 2, characterized in that, The process of normalizing, filtering noise, and aligning the original spatial data to determine the corresponding standardized spatiotemporal sequence data includes: A unified spatiotemporal reference system is established, and the raw spatial data collected by each modal sensor is transformed to the same coordinate system. The linear normalization formula is used to map the raw data of each modality to the [0,1] interval, and the normalized data corresponding to each modality is determined. The normalized data is filtered for noise using a Kalman filter multi-sensor data fusion algorithm. A consistency verification model for multimodal data is constructed based on Mahalanobis distance. Outlier detection is performed on the noise-filtered multimodal data according to the consistency verification model, and the detected outlier data is removed to determine the verification data corresponding to the original spatial data of each modality. Based on the verification data from radar and satellite navigation receiving systems, the verification data from optical sensors is aligned by linear interpolation. When a missing data is detected, the verification data from the optical sensors is completed using the quaternion spherical linear interpolation method to determine the corresponding standardized spatiotemporal sequence data.

4. The space collision early warning method according to claim 1, characterized in that, The step of inputting the standardized spatiotemporal sequence data into a preset convolutional network, and extracting features by synchronously sliding multiple convolutional kernels across the time and multiple spatial dimensions to determine the spatiotemporal features of the corresponding spatial target includes: The standardized spatiotemporal sequence data is input into a preset 3D convolutional network. Multiple layers of 3D convolutional kernels slide synchronously in the time and space dimensions to extract convolutional response vectors. The convolutional response vectors are used to characterize the motion pattern and relative position features of spatial targets. The convolutional response vector is subjected to batch normalization and time-dimension-preserving max pooling operations to determine the spatiotemporal features of the corresponding spatial target, wherein the spatiotemporal features of the spatial target are represented in the form of a high-dimensional feature tensor.

5. The space collision early warning method according to claim 1, characterized in that, The process of flattening the spatiotemporal features into time-series input data and inputting it into a preset bidirectional LSTM network, modeling temporal dependencies using forward LSTM and backward LSTM respectively, and determining the corresponding temporal enhancement features includes: The spatiotemporal features are flattened into two-dimensional sequence data according to time steps, while preserving the correspondence between time steps and feature channels; The flattened sequence data is input into a preset bidirectional LSTM network. The forward LSTM models historical dependencies in chronological order, and the reverse LSTM models potential future associations in reverse chronological order. Finally, the bidirectional hidden states are spliced ​​together to determine the corresponding temporal enhancement features.

6. The space collision early warning method according to claim 1, characterized in that, The process of extracting instantaneous motion features from the spatiotemporal features using global average pooling and calculating a motion intensity score includes: Perform global average pooling on the spatiotemporal features along the time dimension to determine the corresponding pooled feature vector; Within the fully connected layer, a motion intensity score is generated based on the pooled feature vector and a preset sigmoid activation function.

7. The space collision early warning method according to claim 6, characterized in that, The process involves employing a multi-head attention mechanism to calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features, determining the corresponding initial attention weight matrix, and adjusting the initial attention weight matrix based on the motion intensity score to determine the corresponding adjusted attention weights. This includes: A multi-head attention mechanism is adopted, with the spatiotemporal features as query vectors and the temporal enhancement features as key vectors and value vectors, to determine the corresponding initial attention weight matrix; Set a threshold for classifying motion states, and determine the motion state based on the threshold and the motion intensity score. If the motion state is determined to be a violent motion state, increase the weight of the temporal enhancement feature and determine the corresponding adjusted attention weight. If the motion is determined to be in a steady state, the weight of the spatiotemporal features is increased, and the corresponding adjusted attention weight is determined.

8. A space collision early warning device, characterized in that, The device includes: The multimodal data preprocessing module is used to collect raw spatial data of space targets through multimodal sensors, and to perform normalization, noise filtering and time alignment processing on the raw spatial data to determine the corresponding standardized spatiotemporal sequence data. The multimodal sensors include radar, optical sensors and satellite navigation receiving systems, and the raw spatial data includes spatial three-dimensional coordinates, object velocity and object attitude information. The spatiotemporal feature extraction module is used to input the standardized spatiotemporal sequence data into a preset convolutional network, extract features by synchronously sliding multiple convolutional kernels in the time dimension and multiple spatial dimensions, determine the spatiotemporal features of the corresponding spatial target, flatten the spatiotemporal features by time step and convert them into temporal input data and input them into a preset bidirectional LSTM network, model the temporal dependency relationship by forward LSTM and backward LSTM respectively, and determine the corresponding temporal enhancement features; The spatiotemporal feature fusion module is used to map the spatiotemporal features and the temporal enhancement features to the same spatial dimension through a fully connected layer. It extracts the instantaneous motion features of the spatiotemporal features using global average pooling and calculates the motion intensity score. A multi-head attention mechanism is used to calculate the dynamic weights of the spatiotemporal features and the temporal enhancement features, determining the corresponding initial attention weight matrix. The initial attention weight matrix is ​​then adjusted based on the motion intensity score to determine the adjusted attention weights. Feature fusion is performed on the spatiotemporal features and the temporal enhancement features based on the attention weights to determine the corresponding fused features. A collision probability distribution of the target space is generated based on the fused features using a preset Monte Carlo simulation sampling algorithm. If the collision probability distribution is greater than a preset collision probability threshold, a warning signal is output for collision warning.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the space collision warning method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the space collision warning method according to any one of claims 1 to 7.

Images