A method for space target radar / visible light joint perception information processing

Abstract

The present invention discloses a method for processing space target radar / visible light joint perception information. When the target is in a long-distance stage with a distance of 100km to 20km, the target is jointly positioned by radar and visible light; when the target is in a close-distance stage with a distance of less than 20km, the target is imaged by radar and imaged by a visible light camera respectively; on the basis of radar imaging and visible light camera imaging, four recognition and measurement modes are synchronously executed, wherein mode 1 is radar recognition and measurement, mode 2 is radar fusion recognition and radar measurement, mode 3 is radar single source recognition and radar and visible light fusion measurement, and mode 4 is radar and visible light fusion recognition and measurement; the measurement results of the four modes are fused at the decision level, and finally the attitude measurement result of the target at the current moment is output. The present invention realizes all-day and all-weather target perception by combining radar and visible light sensors, greatly improving the reliability of the application.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-orbit space target recognition and measurement, and in particular relates to a method for space target radar / visible light joint perception information processing. Background Art

[0002] With the rapid development of space technology, spacecraft with various functions have been deployed into space, and spacecraft situational awareness technology has gradually become an important research direction in the field of space technology. Space surveillance systems mainly include ground-based radars, optical sensors, and various space-based sensors. Optical sensors have advantages such as high resolution and relatively mature technology, but their detection performance is easily affected by light and weather. Radar systems can work effectively under all-day and all-weather conditions and provide accurate speed and distance information of targets, so radar can make up for the limitations of optical sensors.

[0003] In recent years, the observation performance of radar and optical sensors has been greatly improved, and significant progress has been made in the field of data fusion. Joint processing of multi-source data has become a development trend in the field of signal processing and image processing. However, most of the existing research is based on the study of spacecraft situational awareness of a single type of sensor, namely optical sensors and ISAR (Inverse Synthetic Aperture Radar). For optical sensors, Feng et al. (FENG Q, LIU Y, ZHU ZH, et al. Vision-based relative state estimation for a non-cooperative target [C] / / Proceedings of the 2018 AIAA Guidance, Navigation, and Control Conference, Florida, US, 2018) proposed a new attitude determination method based on convolutional neural networks to address the lack of reliable visual features and the need for robustness to the lighting environment. The method achieves real-time estimation of the airborne attitude and gets rid of the reliance on image features and prior relative state information. SHARMA et al. (SHARMA S, D'AMICO S.Poseestimation for non-cooperative rendezvous using neural networks[C] / / Proceedings of AAS / AIAA Astrodynamics Specialist Conference Ka'anapali:AIAA, 2019) used the YOLO network to identify the satellite body, solar panels, satellite antennas, nozzles, cameras and star sensors of non-cooperative targets, and the recognition accuracy reached more than 90%. For ISAR, F. Wang et al. (WANG F, XU F, JIN Y.3-D information of a space target retrieved from a sequence of high-resolution 2-D ISAR images[C]. IEEE International Geoscience and Remote Sensing Symposium, Beijing, China, 2016: 5000-5002) combined the compressed sensing technology and applied the method to sparse imaging scenes, and reconstructed the target in three dimensions according to the distribution of scattering points in the sparse ISAR image sequence of the space target.K.Suwa et al. (SUWAK, WAKAYAMAT, IWAMOTO M. Three-dimensional target geometry and target motion estimation method using multistatic ISAR movies and its performance [J]. IEEE Transactions Geoscience and Remote Sensing, 2011, 49 (6): 2361-2373.) proposed to use the change characteristics of space targets in multi-base ISAR observation image sequences to invert their three-dimensional information in space. Wang Zhihui et al. (Wang Zhihui, Wang Zhuang, Jiang Libing. Space target attitude estimation method based on line feature differential projection [J]. Signal Processing, 2017, 33 (10): 1377-1384) proposed to optimize the target on-orbit attitude parameter estimation based on the line structure differential projection characteristics of space targets in ISAR observation image sequences. The fusion of radar / visible light multi-source data mainly adopts the method of joint utilization of decision-making layer information, which fails to achieve true collaborative perception. Therefore, it is necessary to design an information processing system for radar / visible light joint perception, which uses radar and visible light information from multiple angles and dimensions to efficiently and reliably complete space target recognition and measurement tasks. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method for space target radar / visible light joint perception information processing, performs radar imaging and visible light camera imaging on the target, executes four recognition and measurement modes, fuses the measurement results of the four modes at the decision level, and outputs the posture measurement result of the target at the current moment, effectively fusing radar and visible light sensor data, and improving the accuracy and robustness of target component recognition and measurement.

[0005] In order to achieve the above functions, the present invention adopts the following technical solutions:

[0006] A method for space target radar / visible light joint perception information processing, comprising the following steps:

[0007] S1. When the target is at a long distance of 100km to 20km, the target is located jointly by radar and visible light.

[0008] S2. When the target is at a close distance of less than 20 km, radar imaging and visible light camera imaging are performed on the target respectively.

[0009] S3. Based on step S2, four identification and measurement modes are synchronously executed, wherein mode 1 is radar identification and measurement, mode 2 is radar fusion identification and radar measurement, mode 3 is radar single source identification and radar and visible light fusion measurement, and mode 4 is radar and visible light fusion identification and measurement.

[0010] S4. Based on step S3, the measurement results of the four modes are fused at the decision level, and finally the posture measurement result of the target at the current moment is output.

[0011] Further, in step S3, mode 1 includes a radar image component recognition module and a radar image component measurement module; mode 2 includes a feature-level fusion component recognition module and a radar image component measurement module; mode 3 includes a radar image component recognition module and a fusion component measurement module; mode 4 includes a feature-level fusion component recognition module and a fusion component measurement module.

[0012] Furthermore, in step S3, in the radar image component recognition module of mode 1 and mode 3, the Mask-RCNN network is used to recognize the sailboard components of the radar image; in the feature-level fusion component recognition module of mode 2 and mode 4, the radar and visible light surface target image feature-level fusion network based on cooperative learning is used to recognize the target sailboard. During the training phase, this network must simultaneously input the paired radar image and visible light image for network training for sailboard recognition, but in the application phase, only the radar image can be input for sailboard recognition, or the radar image and visible light image can be input for sailboard recognition at the same time.

[0013] Furthermore, Mode 1 includes the following sub-steps:

[0014] Step 1. Execute the radar image component recognition module: Use Mask R-CNN (Mask Region-based Convolutional Neural Network) to recognize the sailboard components of the radar image, and output the segmented binary sailboard mask and the confidence of the segmentation result.

[0015] Step 2, execute the radar image component measurement module: use the LSD (Line Segment Detector) algorithm to extract the straight line features of the long side of the sailboard in the binary sailboard mask image, output the projection length of the sailboard in the radar range direction and Doppler direction, and output the confidence of the sailboard feature extraction.

[0016] Step 3: Combine the radar observation geometry and feature extraction results to establish a three-dimensional hybrid observation equation, and output the analytical relationship between the image features and the size and attitude of the sailboard. The observation equation formula is expressed as follows:

[0017]

[0018] In the formula, i LOS and i d Respectively represent the radar sight direction vector and Doppler direction vector of the current frame, · represents the inner product, represents the attitude vector of the sailboard, α and β represent the pitch angle and azimuth angle of the sailboard in the radar reference coordinate system, r represents the projection length of the sailboard in the radar range direction, and d represents the projection length of the sailboard in the Doppler direction.

[0019] Step 4: Convert the equation in step 3 into the optimization problem of solving the attitude angle parameters α and β of the windsurfing board. The optimization objective function is constructed as follows:

[0020]

[0021] The particle swarm optimization algorithm is used to solve the objective function and obtain the optimization parameters. in and Represent the measured values ​​of α and β in mode 1, respectively.

[0022] Step 5: Determine the attitude vector of the windsurfing board based on the optimization parameters of step 4. The specific formula is:

[0023]

[0024] in, yes The measured value in mode 1, that is, the measurement result of the windsurfing attitude in mode 1.

[0025] Based on the attitude vector, the length of the sailboard is further calculated. The specific formula is:

[0026]

[0027] in, Indicates the measured length of the board in mode 1.

[0028] Furthermore, mode 3 includes the following sub-steps:

[0029] Step 1: In the radar image component recognition module, the Mask-RCNN network is used to identify the sailboard components of the radar image, output the segmented binary sailboard mask, and output the confidence of the segmentation result.

[0030] Step 2: In the fusion component measurement module, the straight line segment detection algorithm is used to extract the projection length of the long side straight line feature of the sailboard in the radar binary sailboard mask image in the distance direction, which is expressed as r 1; Use the straight line segment detection algorithm to extract the horizontal projection length of the long side straight line feature of the sailboard in the visible light image of the first observation angle at the same time, expressed as u 1 , the vertical projection length, denoted as v 1 ; Use the straight line segment detection algorithm to extract the horizontal projection length of the long side straight line feature of the sailboard in the visible light image of the second observation angle at the same time, expressed as u 2 , the vertical projection length, denoted as v 2 ; At the same time, the confidence of the sailboard feature extraction in each image is output.

[0031] Step 3: Combine the radar observation geometry and feature extraction results to establish a three-dimensional hybrid observation equation, output the analytical relationship between the image features and the size and attitude of the sailboard, and the observation equation formula is expressed as follows;

[0032]

[0033] In the formula, i u,1 and i v,1 Respectively represent the unit vectors in the horizontal and vertical directions of the visible light image I; i u,2 and i v,2 They represent the unit vectors in the horizontal and vertical directions of the visible light image II respectively, and L is the actual length of the sailboard.

[0034] Step 4: Convert the equation in step 3 into an optimization problem for solving the attitude angle parameters α and β of the windsurfing board. The optimization objective function is constructed as follows:

[0035]

[0036] The particle swarm optimization algorithm is used to solve the objective function and obtain the optimization parameters. in Respectively represent the measured values ​​of α, β, and L in mode 3.

[0037] Step 5: Determine the attitude vector of the windsurfing board based on the optimization parameters of step 4

[0038] in, yes The measured value in mode 3, i.e. the measurement result of the windsurfing attitude in mode 3.

[0039] Furthermore, step S4 specifically includes the following steps:

[0040] S401, calculate the ICC (Intraclass Correlation Coefficient) of the measurement results of the four modes. If the ICC coefficient is greater than 0.8, directly take the average of the four measurement results as the final output, otherwise go to step S402 and use the DS evidence theory for further fusion judgment.

[0041] S402. The radar and visible light image confidences, the sailboard segmentation result confidences and the sailboard feature extraction confidences are regarded as three pieces of evidence fused by the DS evidence theory. The radar and visible light image confidences, the sailboard segmentation result confidences and the sailboard feature extraction confidences corresponding to the four modes are normalized to basic probability values ​​respectively. The basic probability distribution function corresponding to the three pieces of evidence in each mode is calculated, and the result of the measurement mode corresponding to the maximum probability distribution function among the four modes is selected as the final output.

[0042] The present invention adopts the above technical solution, and compared with the prior art, its significant technical effects are as follows:

[0043] 1. The present invention realizes all-day and all-weather target perception by combining radar and visible light sensor, greatly improving the reliability of application.

[0044] 2. The present invention makes full use of the detection advantages of radar and visible light sensors to improve the accuracy of target tracking and positioning, thereby improving the quality of perceived information.

[0045] 3. The present invention adopts a radar / visible light sensor collaborative perception method, which can improve the space target perception performance, including image resolution, target component recognition accuracy, target component measurement accuracy and anti-interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a diagram of the radar and visible light joint target positioning structure in an embodiment of the present invention.

[0047] Figure 2 It is a radar imaging and visible light image result diagram in an embodiment of the present invention.

[0048] Figure 3 is a flow chart of identifying and measuring modes in an embodiment of the present invention.

[0049] Figure 4 1 is a target sailboard recognition and feature extraction diagram in recognition and measurement mode 1 in an embodiment of the present invention.

[0050] Figure 5 1 is a target sailboard recognition and feature extraction diagram in recognition and measurement mode 2 in an embodiment of the present invention.

[0051] Figure 61 is a target sailboard recognition and feature extraction diagram in recognition and measurement mode 3 in an embodiment of the present invention.

[0052] Figure 7 1 is a target sailboard recognition and feature extraction diagram in recognition and measurement mode 4 in an embodiment of the present invention.

[0053] Figure 8 It is a decision-level fusion flow chart of target perception results of the present invention. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] The present invention provides a method for space target radar / visible light joint perception information processing, comprising the following steps:

[0056] S1. When the target is at a long-range stage of 100km to 20km, the target is located by radar and visible light, which specifically includes the following steps:

[0057] S101. Radar detects targets based on echoes and provides accurate relative distance and relative speed.

[0058] S102: The visible light camera provides accurate target angle information based on the image.

[0059] S103, using the radar distance information and the visible light angle information, the target is located by radar and visible light joint positioning through time synchronization and space synchronization. The target positioning result under the radar observation coordinates is as follows: Figure 1 As shown in Figure 3, the comparison between the joint positioning point trace and the original set trajectory shows that the error is extremely small and the target trajectory can be accurately reconstructed.

[0060] S2. When the target is at a close range of less than 20 km, radar imaging and visible light camera imaging are performed on the target respectively, which specifically includes the following steps:

[0061] S201, target orbit determination, obtaining the position and velocity of the target in the inertial coordinate system.

[0062] S202: Perform radar imaging feasibility analysis based on the target orbit information in S201 to determine the target imaging time period and imaging quality.

[0063] S203, according to the imaging time period of S202, radar imaging is performed and target parameters related to the target image are continuously saved and output. The imaging result is as follows Figure 2 As shown, Figure 2 (a) is the radar image, Figure 2 (b) is a visible light image.

[0064] S3. Based on step S2, four identification and measurement modes are synchronously executed, wherein mode 1 is radar identification and measurement, mode 2 is radar fusion identification and radar measurement, mode 3 is radar single source identification and radar and visible light fusion measurement, and mode 4 is radar and visible light fusion identification and measurement.

[0065] Among them, mode 1 includes a radar image component recognition module and a radar image component measurement module; mode 2 includes a feature-level fusion component recognition module and a radar image component measurement module; mode 3 includes a radar image component recognition module and a fusion component measurement module; mode 4 includes a feature-level fusion component recognition module and a fusion component measurement module.

[0066] In the radar image component recognition module of mode 1 and mode 3, the Mask-RCNN network is used to recognize the sailboard components of the radar image; in the feature-level fusion component recognition module of mode 2 and mode 4, the radar and visible light surface target image feature-level fusion network based on cooperative learning is used to recognize the target sailboard. This network can use both single-channel radar image input for sailboard recognition and dual-channel radar and visible light images for sailboard recognition. In mode 2, single-channel input radar images are used for sailboard recognition, and in mode 4, dual-channel input radar and visible light images are used for sailboard recognition.

[0067] In this embodiment, taking the process of the target approaching the detection star as an example, when the target is 4.5 km away from the radar sensor, radar imaging and visible light imaging are performed on the target, and then four target recognition and measurement modes are executed synchronously. The flow chart is as follows: Figure 3 As shown. The actual length of the long side of the target sailboard is 1.4m, and the true posture is [1,0,0] T .

[0068] like Figure 3 As shown in (a), mode 1 includes the following sub-steps:

[0069] Step 1: Execute the radar image component recognition module: Use the Mask-RCNN network to identify the sailboard components in the radar image, and output the segmented binary sailboard mask and the confidence of the segmentation result. The component recognition result based on the Mask-RCNN network is as follows: Figure 4 As shown, the target sailboard is accurately identified.

[0070] Step 2: Execute the radar image component measurement module: Use the LSD algorithm to extract the straight line features of the long side of the sailboard in the binary sailboard mask image, output the projection length of the sailboard in the radar range and Doppler directions, and output the confidence of the sailboard feature extraction. The results of the long side feature extraction of the sailboard are as follows: Figure 4 As shown, the four line segments represent the extracted long side information of the windsurfing board.

[0071] Step 3: Combine the radar observation geometry and feature extraction results to establish a three-dimensional hybrid observation equation, and output the analytical relationship between the image features and the size and attitude of the sailboard. The observation equation formula is expressed as follows:

[0072]

[0073] In the formula, i LOS and i d Respectively represent the radar sight direction vector and Doppler direction vector of the current frame, · represents the inner product, represents the attitude vector of the sailboard, α and β represent the pitch angle and azimuth angle of the sailboard in the radar reference coordinate system, r represents the projection length of the sailboard in the radar range direction, and d represents the projection length of the sailboard in the Doppler direction.

[0074] Step 4: Convert the equation in step 3 into the optimization problem of solving the attitude angle parameters α and β of the windsurfing board. The optimization objective function is constructed as follows:

[0075]

[0076] The particle swarm optimization algorithm is used to solve the objective function and obtain the optimization parameters. in and Represent the measured values ​​of α and β in mode 1, respectively.

[0077] Step 5: Determine the attitude vector of the windsurfing board based on the optimization parameters of step 4. The specific formula is:

[0078]

[0079] Based on the attitude vector, the length of the sailboard is further calculated. The specific formula is:

[0080]

[0081] in, Indicates the measured length of the board in mode 1.

[0082] The estimated attitude and length information of mode 1 are shown in Table 1. The measured attitude of the sailboard is (0.9955, -0.0950, -0.0017), the error is 5.4535°, the measured length of the sailboard is 1.3500m, and the error percentage is 3.57%.

[0083] like Figure 3As shown in (b), the feature-level fusion component recognition module in mode 2 uses a single-channel input radar image to identify sailboards, outputs the segmented binary sailboard mask and the confidence of the segmentation result, and the radar image component measurement module is the same as mode 1. The results of sailboard segmentation and feature extraction are shown in Figure 5 As shown in Figure 1, the target sailboard recognition results and feature extraction results are relatively accurate; the attitude estimation information is shown in Table 1. The measured attitude of the sailboard is (0.9970, 0, -0.0772, 0), the error is 4.4300°, the measured length of the sailboard is 1.3539m, and the error percentage is 3.29%.

[0084] like Figure 3 As shown in (c), mode 3 includes the following sub-steps:

[0085] Step 1: Execute the radar image component recognition module: Use the Mask-RCNN network to identify the sailboard components in the radar image, output the segmented binary sailboard mask, and output the confidence of the segmentation result.

[0086] Step 2: Execute the fusion component measurement module: Use the LSD algorithm to extract the projection length of the long side straight line feature of the sailboard in the radar binary sailboard mask image in the distance direction, expressed as r 1 ; Use the LSD algorithm to extract the horizontal projection length of the long side straight line feature of the sailboard in the visible light image I at the same time, expressed as u 1 , the vertical projection length, denoted as v 1 ; Use the LSD algorithm to extract the horizontal projection length of the long side straight line feature of the sailboard in the visible light image II at the same time, expressed as u 2 , the vertical projection length, denoted as v 2 ; At the same time, the confidence of the sailboard feature extraction in each image is output. Among them, the sight direction azimuth angle of visible light image I is 141.6141°, and the pitch angle is 0°; the sight direction azimuth angle of visible light image II is 19.3606°, and the pitch angle is 0°.

[0087] The feature extraction results of radar image and visible light image are shown in Figure 6 As shown, Figure 6 (a) is the result of visible light image I, Figure 6 (b) is the result of visible light image II. Figure 6 (c) is the result of radar image, which is Figure 6 It can be seen that the target sailboard recognition results and feature extraction results are relatively accurate.

[0088] Step 3: Combine the radar observation geometry and feature extraction results to establish a three-dimensional hybrid observation equation, output the analytical relationship between the image features and the size and attitude of the sailboard, and the observation equation formula is expressed as follows;

[0089]

[0090] In the formula, i u,1 and i v,1 Respectively represent the unit vectors in the horizontal and vertical directions of the visible light image I; i u,2 and i v,2 They represent the unit vectors in the horizontal and vertical directions of the visible light image II respectively, and L is the actual length of the sailboard.

[0091] Step 4: Convert the equation in step 3 into an optimization problem for solving the attitude angle parameters α and β of the windsurfing board. The optimization objective function is constructed as follows:

[0092]

[0093] The particle swarm optimization algorithm is used to solve the objective function and obtain the optimization parameters. in Respectively represent the measured values ​​of α, β and L in mode 3.

[0094] Step 5: Determine the attitude vector of the windsurfing board based on the optimization parameters of step 4

[0095] in, yes The measured value in mode 3, i.e. the measurement result of the windsurfing attitude in mode 3.

[0096] The estimated attitude and length information of mode 3 are shown in Table 1. The measured attitude of the sailboard is (0.9980, 0.0636, 0.0030), the error is 3.6511°, the measured length of the sailboard is 1.3812m, and the error percentage is 1.34%.

[0097] like Figure 3 As shown in (d), the feature-level fusion component recognition module of mode 4 uses dual-path input radar and visible light images to identify the sailboard, and outputs the segmented binary sailboard mask and the confidence of the segmentation result. The fusion component measurement module is the same as mode 3. The sailboard attitude vector and length output by mode 4 are respectively and The feature extraction results of radar image and visible light image are shown in Figure 7 As shown, Figure 7 (a) is the result of visible light image I, Figure 7 (b) is the result of visible light image II. Figure 7 (c) is the result of radar image, which is Figure 7It can be seen that the target sailboard recognition results and feature extraction results are relatively accurate. The estimated posture and length information of mode 4 are shown in Table 1. The measured posture of the sailboard is (0.9997, -0.0223, 0.0058), the error is 1.3224°, the measured length of the sailboard is 1.4075m, and the error percentage is 0.54%.

[0098] Table 1 Target sailboard attitude and length estimation

[0099]

[0100] In step S4, the process is as follows Figure 8 As shown, the specific steps include:

[0101] S401, calculate the intra-group correlation coefficient of the measurement results of the four modes, and continuously record the measurement values ​​of the four modes at the nth moment and the previous 10 moments, expressed as X k,n-j , where k = 1, 2, 3, 4, j = 1, 2, ... 10, then the measured mean of modes 1 to 4 is expressed as:

[0102]

[0103] The mean of all measurements is:

[0104]

[0105] The between-group variance is:

[0106]

[0107] The within-group variance is:

[0108]

[0109] ICC is calculated as follows:

[0110]

[0111] S402. If ICC is greater than 0.8, directly take the average of the four measurement results (X 1,n +X 2,n +X 3,n +X 4,n ) / 4 as the final output; otherwise, the process goes to step S403, and the DS evidence theory is used for further fusion judgment. After calculation, the ICC coefficients of the four modes in this embodiment are less than 0.8, and the DS evidence theory is implemented.

[0112] S403. Assign basic probabilities of three pieces of evidence in four modes based on DS evidence theory:

[0113] The confidences of radar and visible light images, the confidences of sailboard segmentation results and the confidences of sailboard feature extraction corresponding to the four modes are normalized to basic probability values. The results are shown in Table 2. The sum of the basic probabilities of each piece of evidence is 1 (the sum of each row in Table 2 is 1).

[0114] Table 2 Basic probability assignment

[0115]

[0116] S404, calculating the basic probability distribution function, specifically:

[0117] The basic probability distribution function m corresponding to the three pieces of evidence 1 (·), m 2 (·) and m 3 (·), the basic probability of mode 1 (A) is calculated according to the DS rule, and the formula is:

[0118]

[0119] Among them, the coefficient 1 / (1-K) is the normalization factor,

[0120] The basic probability of Mode 2 (B) is calculated as follows:

[0121]

[0122] The basic probability of Mode 3 (C) is calculated as follows:

[0123]

[0124] The basic probability of mode 4 (D) is calculated as follows:

[0125]

[0126] S405, determine the fusion output result: After calculation, among the four modes of m(A), m(B), m(C), and m(D), m(D) is the largest, so the measurement result corresponding to output mode 4 is the final output, that is, the windsurfing attitude is (0.9997, -0.0223, 0.0058) T , with a length of 1.4075m.

[0127] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0132] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for space target radar / visible light joint perception information processing, characterized in that: include: S1. When the target is at a long distance of 100km to 20km, the target is located by radar and visible light; S2: When the target is at a close range of less than 20 km, radar imaging and visible light camera imaging are performed on the target respectively; S3, based on step S2, synchronously execute 4 identification and measurement modes, wherein mode 1 is radar identification and measurement, mode 2 is radar fusion identification and radar measurement, mode 3 is radar single source identification and radar, visible light fusion measurement, and mode 4 is radar, visible light fusion identification and measurement; Among them, mode 1 includes a radar image component recognition module and a radar image component measurement module; mode 2 includes a feature-level fusion component recognition module and a radar image component measurement module; mode 3 includes a radar image component recognition module and a fusion component measurement module; mode 4 includes a feature-level fusion component recognition module and a fusion component measurement module; In the radar image component recognition module of mode 1 and mode 3, a convolutional neural network based on mask area is used to recognize sailboard components from radar images; in the feature-level fusion component recognition module of mode 2, a radar and visible light surface target image feature-level fusion network based on cooperative learning is used to recognize sailboards from radar target images, that is, single-path input radar images are used for sailboard recognition; in the feature-level fusion component recognition module of mode 4, a radar and visible light surface target image feature-level fusion network based on cooperative learning is used to recognize sailboards from radar and visible light target images, that is, dual-path input radar and visible light images are used for sailboard recognition; S4. Based on step S3, the measurement results of the four modes are fused at the decision level, and finally the posture measurement result of the target at the current moment is output.

2. The method for space target radar / visible light joint perception information processing according to claim 1, characterized in that: Step S1 specifically includes the following steps: S101, radar detects the target based on the echo and provides the relative distance and relative speed of the target; S102, the visible light camera provides angle information of the target according to the image; S103: Using the distance information of the radar and the angle information of the visible light, the target is jointly located by radar and visible light through time synchronization and space synchronization.

3. The method for space target radar / visible light joint perception information processing according to claim 1, characterized in that: Step S2 specifically includes the following steps: S201, target orbit determination, obtaining orbit information of the position and velocity of the target in an inertial coordinate system; S202, performing radar imaging feasibility analysis according to the target orbit information of step S201, and determining the target imaging time period and imaging quality; S203. According to the imaging time period of step S202, radar imaging is performed and target parameters related to the target image are continuously saved and output.

4. The method for space target radar / visible light joint perception information processing according to claim 1, characterized in that: Mode 1 includes the following sub-steps: Step 1, execute the radar image component recognition module: use the convolutional neural network based on the mask area to recognize the sailboard components of the radar image, and output the segmented binary sailboard mask and the confidence of the segmentation result; Step 2, execute the radar image component measurement module: use the straight line segment detection algorithm to extract the long side straight line features of the sailboard in the binary sailboard mask image, output the projection length of the sailboard in the radar range direction and Doppler direction, and output the confidence of the sailboard feature extraction; Step 3: Combine the radar observation geometry and feature extraction results to establish a three-dimensional hybrid observation equation and output the analytical relationship between the image features and the size and attitude of the sailboard. The specific formula is as follows: In the formula, i LOS and i d Respectively represent the radar sight direction vector and Doppler direction vector of the current frame, · represents the inner product, represents the attitude vector of the sailboard, α and β represent the sailboard pitch angle and azimuth angle in the radar reference coordinate system, r represents the projection length of the sailboard in the radar range direction, and d represents the projection length of the sailboard in the Doppler direction; Step 4: Convert the relational expression in step 3 into the optimization problem of solving the attitude angle parameters α and β of the sailboard. The optimization objective function is constructed as follows: The particle swarm optimization algorithm is used to solve the objective function and obtain the optimization parameters. in and They represent the measured values ​​of α and β in mode 1 respectively; Step 5: Determine the attitude vector of the windsurfing board based on the optimization parameters of step 4. The specific formula is: in, yes The measured value in mode 1, i.e. the measurement result of the windsurfing attitude in mode 1; The length of the sailboard is calculated based on the attitude vector. The specific formula is: in, Indicates the measured length of the board in mode 1.

5. The method for space target radar / visible light joint perception information processing according to claim 4 is characterized in that: The feature-level fusion component recognition module in mode 2 uses a single-channel input radar image to identify the sailboard, and outputs the segmented binary sailboard mask and the confidence of the segmentation result. The radar image component measurement module is the same as mode 1. The sailboard attitude vector and length output by mode 2 are respectively and express.

6. The method for space target radar / visible light joint perception information processing according to claim 4, characterized in that: In step S3, mode 3 includes the following sub-steps: Step 1: In the radar image component recognition module, a convolutional neural network based on the mask area is used to recognize the sailboard components of the radar image, and the segmented binary sailboard mask and the confidence of the segmentation result are output; Step 2, in the execution fusion component measurement module, use the straight line detection algorithm to extract the projection length of the long side straight line feature of the sailboard in the radar binary sailboard mask image in the distance direction, which is expressed as r1; use the straight line detection algorithm to extract the horizontal projection length of the long side straight line feature of the sailboard in the visible light image of the first observation angle at the same time, which is expressed as u1, and the vertical projection length, which is expressed as v1; use the straight line detection algorithm to extract the horizontal projection length of the long side straight line feature of the sailboard in the visible light image of the second observation angle at the same time, which is expressed as u2, and the vertical projection length, which is expressed as v2; and output the confidence of the sailboard feature extraction in each image at the same time; Step 3: Combine the radar observation geometry and feature extraction results to establish a three-dimensional hybrid observation equation, and output the analytical relationship between the image features and the size and attitude of the sailboard. The specific formula is as follows; In the formula, i u，1 and i v，1 Respectively represent the unit vectors in the horizontal and vertical directions of the visible light image I; i u，2 and i v，2 They represent the unit vectors in the horizontal and vertical directions of the visible light image II, respectively, and L is the real length of the sailboard; Step 4: Convert the relational expression in step 3 into the optimization problem of solving the attitude angle parameters α and β of the sailboard. The optimization objective function is constructed as follows: The particle swarm optimization algorithm is used to solve the objective function and obtain the optimization parameters. in Respectively represent the measured values ​​of α, β, and L in mode 3; Step 5: Determine the attitude vector of the windsurfing board based on the optimization parameters of step 4. The specific formula is: in, yes The measured value in mode 3, i.e. the measurement result of the windsurfing attitude in mode 3.

7. The method for space target radar / visible light joint perception information processing according to claim 6, characterized in that: In step S3, the feature-level fusion component recognition module of mode 4 uses the dual-path input radar and visible light images to identify the sailboard, and outputs the segmented binary sailboard mask and the confidence of the segmentation result. The fusion component measurement module is the same as mode 3. The sailboard attitude vector and length output by mode 4 are respectively and express.

8. The method for space target radar / visible light joint perception information processing according to claim 1, characterized in that: In step S4, obtaining the posture measurement result includes the following: The intra-group correlation coefficients of the measurement results of the four modes were calculated and divided into two cases according to the size of the coefficients: (1) If the intraclass correlation coefficient is greater than 0.8, the mean of the four measurement results is directly taken as the final output; (2) If the intraclass correlation coefficient is less than or equal to 0.8, the DS evidence theory is used for further fusion judgment: The radar and visible light image confidences, the confidences of the sailboard segmentation results and the confidences of the sailboard feature extraction are taken as the three evidences for the fusion of DS evidence theory. The radar and visible light image confidences, the confidences of the sailboard segmentation results and the confidences of the sailboard feature extraction corresponding to the four modes are normalized to basic probability values ​​respectively. The basic probability distribution function corresponding to the three evidences in each mode is calculated, and the result of the measurement mode corresponding to the maximum probability distribution function among the four modes is selected as the final output.

Images