Method and system for autonomous recognition of non-cooperative space targets based on jetson tx2

Abstract

The application discloses a method and system for autonomous identification of non-cooperative space targets based on Jetson TX2, which comprises the following steps: pre-processing the image taken by a long-distance camera, filtering out all dark stars in the background, and then extracting four-connected domains through an OpenCV library to obtain the centroid coordinates and brightness information of the point clusters in the image; pre-processing the image taken by a short-distance camera, performing connected domain segmentation on the target by using a graph closure operation of the OpenCV library, obtaining the center coordinates of each connected domain through the connected domain extraction method of the OpenCV library, calculating the line-of-sight angle of the target, and obtaining the identification result; comparing the results obtained from the long-distance camera and the short-distance camera, if the center coordinates of the target are consistent, it is considered that the target has been stably captured, and the identification result obtained from the short-distance camera is output at this time; otherwise, the coordinates and information obtained from the long-distance camera are output. The application can complete the identification and tracking of space targets from far to near and from point to plane, has a large range of action and high identification accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing and artificial intelligence, and particularly relates to a method and system for autonomous identification of non-cooperative space targets based on Jetson TX2. BACKGROUND

[0002] With the development of space technology, the scope of space missions has been greatly expanded, showing diversification, such as on-orbit servicing, on-orbit assembly, space debris removal, formation cooperation, deep space exploration, space confrontation, etc. These tasks put forward higher requirements for the space target situation awareness capability of spacecraft. Due to the limitation of communication delay or limited signal coverage range, the task spacecraft often cannot obtain the data of the ground control center in time, and needs to rely on its own real-time situation awareness of space targets and autonomous approach capability.

[0003] Space target information acquisition is the primary problem to be solved in space situation awareness. In order to adapt to the complex space environment, space-based detection systems based on microsatellites have gradually become the focus of research. Most of the current mainstream satellite target detection systems are based on FPGA+DSP architecture. The shortcomings of this architecture are as follows: (1) the target detection calculation process is mostly floating-point calculation, which will cause large resource occupation and great loss to the algorithm performance when calculated and processed on FPGA; (2) most target detection algorithms are developed based on high-level programming languages such as Python, which are quite different from the Verilog, VHDL and C languages supported by FPGA, bringing a huge workload for algorithm transplantation and conversion. SUMMARY

[0004] The present application provides a method and system for autonomous identification of non-cooperative space targets based on Jetson TX2, which connects one remote camera and one close-up camera through a network cable, and connects a GNC platform through a serial port. According to the two processes of remote point target detection and close-up surface target component segmentation in the approach task, the original images captured by the cameras can be automatically processed, the remote and close-up algorithms can be automatically switched according to the distance, the coordinates, line-of-sight distance and line-of-sight angle information of the target can be automatically calculated, and the GNC platform can be sent through the serial port to complete the identification and tracking in the autonomous approach process of space targets. The application platform is flexible, the operation speed is fast, and the target detection accuracy is high.

[0005] In order to realize the above functions, the application adopts the following technical scheme:

[0006] The method for autonomous identification of non-cooperative space targets based on Jetson TX2 comprises the following steps:

[0007] S1, the camera transmits raw image data to TX2 through socket, TX2 unpacks the data according to the agreed data format and restores the raw data to image.

[0008] S2, the image taken by the long-distance camera is preprocessed to filter out all the dark star points in the background, and then the four-connected domain is extracted through the OpenCV library to obtain the centroid coordinates and brightness information of the point cluster in the image.

[0009] S3, the image taken by the long-distance camera is preprocessed, and the graph closure operation of the OpenCV library is used to segment the target connected domain. The center coordinates of each connected domain are obtained through the connected domain extraction method of the OpenCV library. According to the relative position of the components, the center coordinates of the target main body are obtained, and the coordinates are converted to the inertial coordinate system through the coordinate system conversion matrix. The line of sight angle of the target is calculated, and the recognition result is obtained.

[0010] S4, compare the results obtained in steps S2 and S3. If the target center coordinates are consistent, it is considered that the target has been stably captured, and the recognition result obtained in step S3 is output at this time. Otherwise, output the coordinates and information obtained in step S2.

[0011] S5, the result output in step S4 is transmitted to the GNC platform through the RS422 protocol-based serial port.

[0012] Further, in step S1, TX2 is connected to the two network cameras through Ethernet cable, and the Socket connection between TX2 and the network cameras is established based on the Socket library under Linux. The image data is transmitted according to the agreed data format. The power-on automatic start function of TX2 is completed by modifying the related configuration file of the system.

[0013] Further, in step S2, the centroid coordinates and brightness information of the point cluster include the following contents:

[0014] S201, the long-distance image preprocessing is mainly based on filtering and threshold segmentation, aiming to filter out the background with large area and smoothness, realize the suppression or elimination of noise and background, and reduce the interference of background and noise on target detection as much as possible. At the same time of processing, the star points and candidate targets are preliminarily extracted to facilitate subsequent faster target recognition. Among them, the main purpose of image filtering is to suppress the noise of star map, and the purpose of threshold segmentation is to separate star points and background.

[0015] Among them, the filtering process is carried out by using Gaussian filter, and the segmentation is carried out by using empirical threshold method, and the threshold is defined as:

[0016] T th =E+k f σ E

[0017] wherein T th represents a segmentation threshold, E represents the average value of all pixel gray values of the entire image, k f represents a proportionality coefficient, σ E represents the variance of the image.

[0018] S202, traversing the image, when the gray value of a pixel is greater than the gray value of the segmentation threshold:

[0019] a. If there is a label on the left or above the point, the label is copied;

[0020] b. If the labels on the left and above the point are consistent, the label is copied;

[0021] c. If the labels on the left and above the point are different, the label of the point above is copied, and the corresponding label is input into the equivalence table as an equivalent label;

[0022] d. In other cases, a new label is assigned to the pixel, i.e. label+1, and the new pixel is added to the equivalence table.

[0023] S203, repeating step S202 until all pixels with a gray value greater than the segmentation threshold in the image are scanned.

[0024] S204, merging the pixels with the same label in the equivalence table, and assigning a new label to them to obtain a set of star points and targets with the same label.

[0025] S205, extracting the centroid of each connected domain using the threshold centroid method, and judging the pixel value, the specific formula being:

[0026]

[0027] wherein I'(x, y) represents the centroid pixel value, I(x, y) represents the current pixel value, x represents the current horizontal coordinate, y represents the current vertical coordinate, T s represents a fixed pixel value, and T represents a preset threshold.

[0028] S206, after the judgment of the pixel value is completed, the centroid method is used to calculate the star point position to obtain the centroid coordinates, and the centroid method takes the pixel value as the weight to calculate the first moment of the star point area in the X and Y directions, the specific formula being:

[0029]

[0030] wherein x0 represents the centroid horizontal coordinate, m represents the number of rows, n represents the number of columns, and y0 represents the centroid vertical coordinate.

[0031] After the star points and the target are coarsely extracted, the coordinates of the stars in the star table are projected into the image plane, and each star is subtracted from the coarsely extracted star points, so that the star points are removed, and the target is identified.

[0032] Further, in step S3, the pre-processing image includes the following contents:

[0033] The pre-processing includes but is not limited to gray processing, Gaussian filtering, normalization, gamma enhancement, and image segmentation.

[0034] The image denoising adopts a Gaussian filtering method, that is, a Gaussian distribution is used as a weight matrix, the matrix is used for convolution on the image, each pixel in the output image takes a weighted average value of the surrounding pixels, and a filtered image is obtained, and a specific formula is as follows:

[0035]

[0036] Wherein, G(x,y) represents a two-dimensional Gaussian distribution; sigma represents a standard deviation of the Gaussian distribution, which determines the smoothing effect, and the larger the standard deviation, the more obvious the smoothing effect; I represents a spatial target observation image; I G represents a filtered spatial target observation image; and G represents a Gaussian filtering weight matrix.

[0037] The automatic gamma correction is used for contrast enhancement processing of the spatial target image, so that the average gray scale of the enhanced image approaches the reference value, and the problem that the target features are weakened under uneven illumination can be effectively avoided, and a specific formula is as follows:

[0038]

[0039] Wherein, gamma represents a gamma enhancement index, RefAveBRT represents an average brightness, and X represents an average value of the filtered image.

[0040] Since the star chart has the characteristics of uneven illumination and easy burst noise, a single threshold value cannot take into account the actual change of the image sequence, therefore, an adaptive threshold segmentation method is used for processing the image, and a specific formula is as follows:

[0041]

[0042] Wherein, (x,y) represents a coordinate position of a pixel point; Thresh(x,y) represents a segmentation threshold value of the pixel point; T m represents a sensitivity, which is represented by information entropy; I E (x,y) represents an enhanced image; and r represents a radius of a local window, and the radius size affects the threshold segmentation effect.

[0043] Traverse the image, compare the pixel gray value and the threshold value, when the gray value is lower than the threshold value, the pixel gray value is set to 0, that is, classified as background; when the gray value is higher than the threshold value, the pixel gray value is set to 1, that is, classified as target. The specific formula is:

[0044]

[0045] Wherein, g(x, y) represents the pixel value after binarization, f(x, y) represents the current pixel value.

[0046] Further, in step S3, the line of sight angle of the target includes the following contents:

[0047] The image is processed by using the morphological operation method. According to the principle, the image after threshold segmentation is first subjected to open operation, the outline of the target is smoothed, the narrow neck is disconnected and the slender protrusions such as the connecting tripod between the sailboard and the body are eliminated, and then the closed operation is used to fill the small holes in the image obtained after pretreatment and the broken lines in the outline, so that the segmentation of each component is completed. The center coordinates of the target body are obtained by extracting the connected domain of each component after segmentation by using the OpenCV method.

[0048] The centroid coordinates of the target in the image plane and the pixel size and focal length provided by the camera are used to convert the target from pixel coordinates to camera coordinates according to the projection transformation, and the line of sight angle is calculated. Wherein, the pixel coordinate system O P -uv is located in the image plane, the origin is located in the upper left corner of the imaging plane, the u axis points to the right parallel to the x axis of the plane coordinate system, the v axis points downward perpendicular to the y axis, and the coordinates are in pixel units. The image coordinate system O R -xy is a two-dimensional rectangular coordinate system, whose origin is at the intersection of the camera optical axis (i.e. the Z C axis of the camera coordinate system) and the imaging plane O R , that is, the center point of the image, the x and y axes are parallel to the u and v axes in the imaging plane.

[0049] In the pixel coordinate system, if the target coordinates are (u P ,v P ), without considering the lens distortion, the coordinates in the image coordinate system are:

[0050]

[0051] Wherein, (x R ,y R ) represents the coordinates of the target in the image coordinate system, dx and dy are the sizes of the unit pixels on the x and y axes respectively, and (u0, v0) represents the plane center coordinates of the target in the pixel coordinate system.

[0052] For a camera with focal length f, the target line-of-sight angle is expressed in vector form in the camera coordinate system as:

[0053] (-f,X C ,Y C )

[0054] Wherein, X C represents the horizontal coordinate of the camera coordinate system, X C =x R ; Y C represents the vertical coordinate of the camera coordinate system, Y C =y R .

[0055] After unitizing the vector, it is expressed as:

[0056] i c =[i x ,i y ,i z ]

[0057] Wherein, i c represents a unit vector, i x , i y , i z represent unit vectors on the X, Y, Z axes respectively.

[0058] The horizontal angle and vertical angle of the target in the camera coordinate system are expressed as:

[0059]

[0060] Wherein, alpha represents the horizontal angle, and beta represents the vertical angle.

[0061] Further, in step S5, the Jetson TX2 is connected with the integrated electronic platform through the on-board UART module, the termios library under Linux is called in the software to complete the configuration of the serial port baud rate, data bits, stop bits and parity bits, and the data transmission and analysis with the GNC are completed according to the agreed data packet format.

[0062] Further, the application also provides a system for autonomous identification of non-cooperative space targets based on Jetson TX2, comprising:

[0063] A centroid coordinate and brightness information acquisition module is used for pre-processing the image photographed by the long-distance camera, filtering out all the dark star points in the background, and then extracting four connected domains through the OpenCV library to obtain the centroid coordinates and brightness information of the point clusters in the image.

[0064] The line-of-sight angle obtaining module is configured to preprocess an image captured by the close-range camera, perform connected domain segmentation on the target by using a graph closure operation of an OpenCV library, obtain the center coordinates of each connected domain by using a connected domain extraction method of the OpenCV library, and calculate the line-of-sight angle of the target.

[0065] The recognition result output module is configured to compare the results obtained by the center-of-mass coordinate and brightness information obtaining module and the line-of-sight angle obtaining module, and if the target center coordinates are consistent, it is considered that the target has been stably captured, and in this case, the close-range camera image is processed; otherwise, the long-range processing result is output.

[0066] Further, the present application also provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the method for autonomously recognizing a non-cooperative space target based on Jetson TX2 when executing the computer program.

[0067] Further, the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for autonomously recognizing a non-cooperative space target based on Jetson TX2.

[0068] The present application adopts the above technical scheme, and has the following significant technical effects compared with the prior art.

[0069] The present application can complete the recognition and tracking of a space target from far to near and from point to plane, and output the target center coordinates and angle measurement information, has a large range of action and high recognition accuracy, and can assist a task star to complete the approach task of a non-cooperative space target. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 is the overall implementation flowchart of the present application.

[0071] Figure 2 is the schematic diagram of the embedded system architecture of the present application.

[0072] Figure 3 is the long-range algorithm filtering result graph in the embodiment of the present application.

[0073] Figure 4 is the long-range algorithm global threshold segmentation result graph in the embodiment of the present application.

[0074] Figure 5 is the target recognition result graph of the long-range algorithm in the embodiment of the present application.

[0075] Figure 6 is the close-range algorithm preprocessing result graph in the embodiment of the present application.

[0076] Figure 7 is the morphological operation result graph of the near distance algorithm in the embodiment of the application.

[0077] Figure 8 is the component identification result graph of the near distance algorithm in the embodiment of the application.

[0078] Figure 9 is the double-thread schematic diagram of the application.

[0079] Figure 10 is the far and near distance calculation result graph in the embodiment of the application.

[0080] Figure 11 is the running result graph in the embodiment of the application. DETAILED DESCRIPTION

[0081] In order to make the above objectives, characteristics and advantages of the application more apparent, comprehensible and easier to understand, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0082] The application proposes a method for autonomous identification of non-cooperative space targets based on Jetson TX2, as shown in Figure 1 , including the following steps:

[0083] S1, as shown in Figure 2 , an embedded system architecture is constructed. The application is deployed on a Jetson TX2 embedded module. The TX2 is connected to one remote camera and one near camera through a network cable. The cameras perform image acquisition tasks and transmit images to the TX2 based on the UDP protocol. The TX2 module processes images from the cameras and runs target identification algorithms to obtain target center coordinates and angle measurement information. The TX2 is connected to the GNC through the RS422 serial port, and the target identification results are output to the GNC to assist in completing the approach task. The specific content is as follows:

[0084] S101, in Ubuntu 18.04, set the TX2 power-on automatic start of the space target autonomous identification software described in the patent.

[0085] S1011, modify the / lib / systemd / system / rc-local.service file with root user permission, and add the following content at the end of the file:

[0086] WantedBy=multi-user.target;

[0087] Alias=rc-local.service;

[0088] S1012. Modify the / etc / rc.local file with root user privileges and add the absolute path of the executable file to be started automatically before exit 0.

[0089] S1013. Run the sudo systemctl enable rc-local command in the terminal to make the configuration take effect.

[0090] S102. Connect the two Gigabit Ethernet ports on the TX2 board to the Ethernet port long-range camera and the Ethernet port close-range camera respectively via Ethernet cables, and establish communication for data transmission.

[0091] S1021, using the camera as the server and TX2 as the client, establish a socket connection based on the network service library under Linux;

[0092] S1022. Agree on a transmission data packet format, and transmit the data through the Socket according to the established format.

[0093] S103: The camera transmits the original image data to TX2 via the socket. TX2 unpacks the data according to the agreed data format and restores the original data to an image.

[0094] S2. Preprocess the image taken by the telephoto camera to filter out all the darker stars in the background. Then, use the OpenCV library to extract the four-connected domain to obtain the centroid coordinates and brightness information of the point cluster in the image. The specific content is:

[0095] S201. Long-range image preprocessing primarily involves filtering and threshold segmentation, aiming to remove large, smooth background areas, suppress or eliminate noise and background, and minimize interference from background and noise on target detection. During processing, preliminary extraction of star points and candidate targets is performed to facilitate subsequent, faster target identification. Image filtering primarily aims to suppress noise in the star image, while threshold segmentation aims to separate star points from the background. Since the noise in the star image is primarily high-frequency, a 5×5 Gaussian filter is used in this embodiment. The key to threshold segmentation is selecting an appropriate segmentation threshold. Because the grayscale values ​​of star points differ significantly from those of the target and background, an empirical thresholding method is employed to effectively separate them.

[0096] Among them, Gaussian filter is used for filtering, and empirical threshold method is used for segmentation. The threshold is defined as:

[0097] T th =E+k f σ E

[0098] Among them, T th represents the segmentation threshold, E represents the average grayscale value of all pixels in the entire image, kf denotes a proportionality coefficient, σ E denotes the variance of the image.

[0099] S202, Since the spot energy distribution of the cluster in the image approximates to Gaussian distribution, the connected domain division adopts the four-connected domain division mode. The four-connected domain division is adopted: taking the pixel point with each value as 1 as the center point, if there is a pixel point with value 1 in the upper, lower, left and right four directions, it is considered that the point and the center point are in the same region, that is, the same object.

[0100] S203, Repeat step S202 until all the pixel points with the gray value greater than the segmentation threshold in the image are scanned.

[0101] S204, Merge the pixel points with the same label in the equivalence table, and assign a new label to them, to obtain the set of star points and targets with the same label.

[0102] S205, The centroid method with threshold is used to extract the centroid of each connected domain, and the pixel value is judged, and the specific formula is:

[0103]

[0104] Wherein, I'(x, y) represents the centroid pixel value, I(x, y) represents the current pixel value, x represents the current horizontal coordinate, y represents the current vertical coordinate, T s represents a fixed pixel value, and T represents a preset threshold.

[0105] S206, After judging the pixel value, the centroid method is used to calculate the star point position, and the centroid coordinates are obtained. The centroid method takes the pixel value as the weight to calculate the first moment of the star point area in the X and Y directions, and the specific formula is:

[0106]

[0107] Wherein, x0 represents the centroid horizontal coordinate, m represents the number of rows, n represents the number of columns, and y0 represents the centroid vertical coordinate.

[0108] After the star points and targets are roughly extracted, the star coordinates in the star table are projected into the image plane, and the difference value between each star and the roughly extracted star points is calculated, so that the star points are removed, and the target is identified.

[0109] As can be seen from Figure 3 , (a) of Figure 3 is the original image, Figure 3 (b) is the result of the distant algorithm filtering, and it can be seen that the background noise is obviously suppressed. Figure 4is the global threshold segmentation result of the far distance algorithm, wherein the green box is the target and the blue box is the star. Some stars have small gray values, and only the central pixel is left after threshold segmentation. The star points are effectively separated from the target and the background. The connected domain extraction result is shown in Figure 5 Figure 3B, two connected domains are extracted, connected domain 1 is removed according to the star map information, and connected domain 2 is the target with coordinates (1023, 1047).

[0110] S3, preprocessing the image captured by the close distance camera, using the graph closure operation of the OpenCV library to segment the target, obtaining the center coordinates of each connected domain through the connected domain extraction method of the OpenCV library, obtaining the center coordinates of the target main body according to the relative positions of the components, and converting the coordinates to the inertial coordinate system through the coordinate system conversion matrix, calculating the line of sight angle of the target, and obtaining the recognition result, the specific content is:

[0111] S301, preprocessing includes gray processing, Gaussian filtering, normalization, gamma enhancement, and image segmentation.

[0112] The image denoising adopts a Gaussian filtering method, that is, a Gaussian distribution is used to weight the matrix, the matrix is used to convolve the image, and each pixel in the output image takes the weighted average value of the surrounding pixels to obtain the filtered image. The specific formula is:

[0113]

[0114] Wherein, G(x, y) represents a two-dimensional Gaussian distribution; σ represents the standard deviation of the Gaussian distribution, which determines the smoothing effect, and the larger the standard deviation, the more obvious the smoothing effect; I represents the spatial target observation image; I G represents the filtered spatial target observation image; G represents the Gaussian filtering weight matrix.

[0115] In this embodiment, the window size is 5 and the standard deviation σ = 1.4.

[0116] The automatic gamma correction is used to enhance the contrast of the spatial target image, so that the average gray value of the enhanced image is close to the reference value, which can effectively avoid the problem that the target features are weakened under uneven illumination. The specific formula is:

[0117]

[0118] Wherein, γ represents the gamma enhancement index, RefAveBRT represents the average brightness, and X represents the average value of the filtered image.

[0119] Due to the characteristics of uneven illumination and easy burst noise of the star map, a single threshold cannot take into account the actual changes of the image sequence, therefore, an adaptive threshold segmentation method is used to process the image, and the specific formula is:

[0120]

[0121] where (x, y) represents the coordinate position of the pixel point; Thresh(x, y) represents the segmentation threshold of the pixel point; T m represents the sensitivity, which is represented by information entropy; I E (x, y) represents the enhanced image; r represents the radius of the local window, and the radius size affects the effect of threshold segmentation.

[0122] In this embodiment, the window radius is 5, so as to retain as much target information as possible.

[0123] The image is traversed, and the gray value of the pixel point is compared with the threshold value. When the gray value is lower than the threshold value, the gray value of the pixel point is set to 0, that is, it is classified as background; when the gray value is higher than the threshold value, the pixel gray value is set to 1, that is, it is classified as a target. The specific formula is:

[0124]

[0125] where g(x, y) represents the pixel value after binarization, and f(x, y) represents the current pixel value.

[0126] S302, the image is processed by using a morphological operation method. According to the principle, the image after threshold segmentation is first subjected to an open operation, the outline of the target is smoothed, the narrow neck is disconnected, and the slender protrusions such as the connecting tripod between the sailboard and the body are eliminated, and then a closed operation is used to fill the small holes in the image obtained after preprocessing and the breaks in the outline, so as to complete the segmentation of each part. The center coordinates of the target main body are obtained by extracting the connected domain of each part obtained after segmentation by using the OpenCV method.

[0127] The centroid coordinates of the target in the image plane and the pixel size and focal length provided by the camera are used to convert the target from the pixel coordinates to the camera coordinate system according to the projection transformation, and the line of sight angle is calculated. Wherein, the pixel coordinate system O P -uv is located in the image plane, the origin is located at the upper left corner of the imaging plane, the u axis points to the right parallel to the x axis of the plane coordinate system, the v axis points downward perpendicular to the y axis, and the coordinates are in units of pixels. The image coordinate system O R -xy is a two-dimensional rectangular coordinate system, whose origin is at the intersection O C of the camera optical axis (i.e. the Z R axis of the camera coordinate system) and the imaging plane, that is, the center point of the image, the x and y axes are parallel to the u and v axes in the imaging plane.

[0128] In the pixel coordinate system, if the target coordinates are (u P , vP ), without considering lens distortion, its coordinates in the image coordinate system are:

[0129]

[0130] Among them, (x R ,y R ) represents the coordinates of the target in the image coordinate system, dx and dy are the sizes of the unit pixel on the x-axis and y-axis respectively, and (u0, v0) represents the plane center coordinates of the target in the pixel coordinate system.

[0131] For a camera with a focal length of f, the target sight angle is expressed as a vector in the camera coordinate system:

[0132] (-f,X C ,Y C )

[0133] Among them, X C Indicates the horizontal coordinate of the camera coordinate system, X C =x R ; Y C Indicates the vertical coordinate of the camera coordinate system, Y C =y R .

[0134] The vector is normalized and expressed as:

[0135] i c =[i x ,i y ,i z ]

[0136] Among them, i c represents a unit vector, i x 、i y 、i z Represent the unit vectors on the X, Y, and Z axes respectively.

[0137] The horizontal angle and vertical angle of the target in the camera coordinate system are expressed as:

[0138]

[0139] Here, α represents the horizontal angle and β represents the vertical angle.

[0140] from Figure 6 It can be seen that Figure 6 (a) is the grayscale processing result. Figure 6 (b) is the filtering result diagram, Figure 6 (c) is the enhanced result graph, Figure 6(d) is the adaptive threshold segmentation result figure, it can be seen that the filtered image gray scale changes more smoothly, and the Gaussian filter can well remove the noise of the target surface; the enhanced image is overall brightened, the main body and the sailboard are clearly visible, the clear target edge is retained, and no extra noise is generated; the threshold segmentation makes the satellite target be basically separated from the background, and the details of the sailboard and the main body are basically retained.

[0141] Figure 7 is the result of the morphological operation of the near-distance algorithm, it can be seen that the internal cavities of the sailboard and the main body are completely eliminated, the structure outside the sailboard and the main body is completely removed, and the geometric features of the target are basically retained. Figure 8 As shown in

[0142] S4, as shown in Figure 9 , compared with the results obtained in steps S2 and S3, if the target center coordinates are consistent, it is considered that the target has been stably captured, and the recognition result obtained in step S3 is output at this time; otherwise, the coordinates and information obtained in step S2 are output.

[0143] In this embodiment, the simulation image verification shows that the edge condition of the near-distance algorithm for component segmentation is about 350 one-dimensional pixels of the target. Therefore, the one-dimensional pixel of the connected domain greater than 350 is selected as the switching condition of the near-distance algorithm, and also as the trigger condition of the near-distance camera thread enablement.

[0144] In the switching stage, the target center coordinates obtained by the far-distance algorithm are (1032, 944), and the target center coordinates obtained by the near-distance algorithm are (1031, 944), the calculation results of which are within the error range, and are considered to be successfully captured, and the algorithm switching is performed.

[0145] S5, the results output in step S4 are transmitted to the GNC platform through the serial port based on the RS422 protocol, and the specific content is:

[0146] S501, the on-board UART module of TX2 is connected to the receiving module through the Dupont line.

[0147] S502, find the serial port device by running ls / dev / ttyTHS* on the terminal, and enable it.

[0148] S503, configure the serial port parameters such as baud rate, data bits, stop bits and parity check through the Linux system call and the termios library.

[0149] S504, agree to the data packet format, the required data is transmitted in the agreed format through the serial port.

[0150] Figure 10 (a) is the remote angle measurement information output, the remote algorithm can output the target main body elevation angle, azimuth angle and line of sight distance information; Figure 10 (b) is the near angle measurement information output, the near algorithm can output the target line of sight distance, target left and right sailboard line of sight angle and body line of sight angle information.

[0151] Figure 11 is the algorithm running result graph, the algorithm can process image information on TX2, obtain target center angle measurement information, correctly switch the remote algorithm and the near algorithm, and keep tracking and information output of the target center during the approach from far to near.

[0152] The embodiment of the present application also provides a system for autonomously identifying a non-cooperative space target based on a Jetson TX2, which comprises a centroid coordinate and brightness information acquisition module, a line of sight angle acquisition module, an identification result output module and a computer program capable of running on a processor.

[0153] The embodiment of the present application also provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor.

[0154] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program.

[0155] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In one embodiment, the application can be implemented in software and / or firmware. In addition, those skilled in the art will further appreciate that the application can be implemented as a method, apparatus, or computer program product. Therefore, embodiments of the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a "circuit" or "module." Furthermore, embodiments of the application can take the form of a computer program product on a computer-readable storage medium having computer program code embodied in the storage medium. The computer program code can cause a computer, processor, or other programmable data processing apparatus to effect the steps in the embodiments of the application as set forth in the description below.

[0156] The embodiments of methods, apparatuses (systems) and computer program products of the application are described herein with reference to flowchart and / or block diagrams of the methods, apparatuses (systems), and computer program products according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0157] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagram block or blocks. Figure 1 Figure 1

[0159] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they learn of the basic inventive concepts. Therefore, the appended claims are intended to cover all such variations and modifications as fall within the scope of the application. ​​​​​​

[0160] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for autonomous recognition of non-cooperative space targets based on Jetson TX2, characterized in that, Comprise: S1, read the image captured by the camera; S2, the image taken by the long distance camera is preprocessed, and the four connected domains are extracted through the OpenCV library to obtain the centroid coordinates and brightness information of the point cluster in the image; S3, the image taken by the close distance camera is preprocessed, the graphological closing operation of the OpenCV library is used to segment the target, the center coordinates of each connected domain are obtained through the connected domain extraction method of the OpenCV library, the line of sight angle of the target is calculated, and the recognition result is obtained; Specifically: The preprocessed image is opened, and then closed to segment each component, the connected domain extraction method of OpenCV is used to extract each component after segmentation, and the center coordinates of the target body are obtained; The line-of-sight angle of the target is calculated according to the projection transformation from the pixel coordinate to the camera coordinate system by using the centroid coordinate of the target under the image plane and the pixel size, focal length of the camera, wherein the pixel coordinate system O P -uv is located in the image plane, the origin is located in the upper left corner of the imaging plane, the u axis points to the right parallel to the x axis of the plane coordinate system, the v axis points downward perpendicular to the y axis, and the coordinates are in units of pixels pixel; the image coordinate system O R -xy is a two-dimensional rectangular coordinate system, the origin is at the intersection of the camera optical axis and the imaging plane O R , that is, the center point of the image, the x and y axes are parallel to the u and v axes in the imaging plane respectively. In the pixel coordinate system, set the target coordinate as (u P ,v P ), then its coordinate in the image coordinate system is: where (x R ,y R ) represents the coordinates of the target in the image coordinate system, dx and dy are the sizes of a unit pixel on the x-axis and y-axis respectively, and (u0, v0) represents the center coordinates of the target in the pixel coordinate system. For the camera with focal length f, the target line of sight angle is expressed in vector form in the camera coordinate system as: (-f,X C ,Y C ) wherein X C represents the horizontal coordinate of the camera coordinate system, X C = x R ; Y C represents the vertical coordinate of the camera coordinate system, Y C = y R ; After unitizing the vector, it is expressed as: i c = [i x , i y , i z ] wherein, i c represents a unit vector, i x , i y , i z represent unit vectors on X, Y, Z axes, respectively; The horizontal angle and vertical angle of the target in the camera coordinate system are expressed as: Wherein, α represents the horizontal angle, and β represents the vertical angle; S4, compare the results obtained in steps S2 and S3, if the target center coordinates are consistent, it is considered that the target has been stably captured, and the recognition result obtained in step S3 is output at this time; Otherwise, output the coordinates and information obtained in step S2.

2. The method for autonomous identification of non-cooperative space targets based on Jetson TX2 according to claim 1, characterized in that, In step S2, the preprocessed image includes the following contents: The preprocessing includes but is not limited to filtering and threshold segmentation; Among them, the Gaussian filter is used for filtering processing, and the empirical threshold method is used for segmentation, and the threshold is defined as: T th = E + k f σ E where T th represents the segmentation threshold, E represents the average value of all pixel gray values of the whole image, k f represents the proportionality coefficient, σ E represents the variance of the image.

3. The method for autonomous identification of non-cooperative space targets based on Jetson TX2 according to claim 2, characterized in that, In step S2, the four connected domains include the following contents: (1) traverse the image, when the gray value of the pixel point is greater than the gray value of the segmentation threshold: a. If there is a mark on the left or above the point, the mark is copied; b. If the marks on the left and above the point are consistent, the mark is copied; c. If the marks on the left and above the point are different, the mark of the point above is copied, and the corresponding mark is input into the equivalence table as the equivalent mark; d. In other cases, a new mark is assigned to the pixel point, and the new pixel point is added to the equivalence table; (2) repeat step (1) until all pixel points with gray value greater than the segmentation threshold in the image are scanned; (3) merge the pixel points with the same mark in the equivalence table, assign a new mark to them, and obtain the set of star points and targets with the same label.

4. The method for autonomous identification of non-cooperative space targets based on Jetson TX2 according to claim 3, characterized in that, In step S2, the centroid coordinates and brightness information of the point cluster include the following contents: (1) the centroid of each connected domain is extracted by using the threshold centroid method, and the pixel value is judged, and the specific formula is: wherein I'(x, y) represents the centroid pixel value, I(x, y) represents the current pixel value, x represents the current horizontal coordinate, y represents the current vertical coordinate, T s represents a fixed pixel value, and T represents a preset threshold value; (2) the star point position is calculated by using the centroid method, and the centroid coordinates are obtained, and the specific formula is: Wherein, x0 represents the centroid horizontal coordinate, m represents the number of rows, n represents the number of columns, and y0 represents the centroid vertical coordinate; The gray value information of the coordinate point is read by using the OpenCV library, which is the brightness information; (3) after roughly extracting the star points and targets, the coordinates of the stars in the star table are projected into the image plane, and the difference between each star and the roughly extracted star point is calculated, the star point is removed, and the target is recognized.

5. The method for autonomous identification of non-cooperative space targets based on Jetson TX2 according to claim 1, characterized in that, In step S3, the preprocessed image includes the following contents: The preprocessing includes but is not limited to gray processing, Gaussian filtering, normalization, gamma enhancement and image segmentation; Wherein, the image is convolved by using Gaussian filter to obtain the filtered image, and the specific formula is: where G(x, y) represents a two-dimensional Gaussian distribution, σ represents a standard deviation of the Gaussian distribution, I represents a spatial target observation image, I G represents a filtered spatial target observation image, and G represents a Gaussian filter weight matrix. The spatial target image is processed by using automatic gamma correction for contrast enhancement, and the specific formula is: Wherein, γ represents the gamma enhancement index, RefAveBRT represents the average brightness, and X represents the average value of the filtered image. The image is processed by using adaptive threshold segmentation method, and the specific formula is: where (x, y) represents the coordinate position of the pixel point; Thresh(x, y) represents the segmentation threshold of the pixel point; T m represents the sensitivity; I E (x, y) represents the enhanced image; r represents the radius of the local window; The image is traversed, the gray value of the pixel point is compared with the threshold value, and the pixel point is classified, and the specific formula is: Wherein, g(x, y) represents the binary pixel value, and f(x, y) represents the current pixel value.

6. A system for autonomous recognition of non-cooperative space objects based on Jetson TX2, characterized in that, It comprises: The centroid coordinate and brightness information acquisition module is used for pre-processing the image shot by the long-distance camera, filtering out all the dark star points in the background, and then extracting the four-connected domain through the OpenCV library to obtain the centroid coordinate and brightness information of the point cluster in the image; The line-of-sight angle acquisition module is used for pre-processing the image shot by the short-distance camera, performing connected domain segmentation on the target by using the graph closure operation of the OpenCV library, obtaining the center coordinates of each connected domain by using the connected domain extraction method of the OpenCV library, and calculating the line-of-sight angle of the target; Specifically: The pre-processed image is subjected to open operation and then closed operation to segment each component, the connected domain extraction method of the OpenCV library is used to extract each component after segmentation, and the center coordinates of the target main body are obtained; The view angle of the target is calculated according to the projection transformation from the pixel coordinate to the camera coordinate system by using the centroid coordinate of the target on the image plane and the pixel size, focal length of the camera, wherein the pixel coordinate system O P -uv is located on the image plane, the origin is located at the upper left corner of the imaging plane, the u axis points to the right parallel to the x axis of the plane coordinate system, and the v axis points downward perpendicular to the y axis, and the coordinates are in units of pixels pixel; the image coordinate system O R -xy is a two-dimensional rectangular coordinate system, the origin is at the intersection of the camera optical axis and the imaging plane O R , that is, the center point of the image, the x and y axes are parallel to the u and v axes in the imaging plane respectively. In the pixel coordinate system, set the target coordinate as (u P ,v P ), then its coordinate in the image coordinate system is: where (x R ,y R ) represents the coordinates of the target in the image coordinate system, dx and dy are the sizes of a unit pixel on the x-axis and y-axis respectively, and (u0, v0) represents the planar center coordinates of the target in the pixel coordinate system. For a camera with a focal length of f, the target line-of-sight angle is expressed in vector form in the camera coordinate system as: (-f,X C ,Y C ) wherein X C represents the horizontal coordinate of the camera coordinate system, X C = x R ; Y C represents the vertical coordinate of the camera coordinate system, Y C = y R ; After unitizing the vector, it is expressed as: i c = [i x , i y , i z ] wherein, i c represents a unit vector, i x , i y , i z represents a unit vector on the X, Y, Z axis, respectively; The horizontal angle and vertical angle of the target in the camera coordinate system are expressed as: Wherein, α represents the horizontal angle, and β represents the vertical angle; The result output module is used for comparing the results obtained by the centroid coordinate and brightness information acquisition module and the line-of-sight angle acquisition module, if the target center coordinates are consistent, it is considered that the target has been stably captured, and the short-distance camera image processing is performed at this time; Otherwise, the long-distance processing result is output.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium being characterized by, The computer program is run by the processor to execute the method of any one of claims 1 to 5.

Images