A method for extracting spatial targets in complex scenes by combining camera imaging models

Abstract

This invention discloses a method for extracting spatial targets in complex scenes by combining a camera imaging model. The method includes: acquiring satellite camera images; determining the Earth region in the image based on satellite attitude and orbit information and the camera imaging model; dividing the image into a deep-space region and an Earth region; and performing image target detection based on the divided image to determine the target region and target location information. This invention's method for extracting spatial targets in complex scenes by combining a camera imaging model determines the presence of the Earth in the camera's field of view, divides the image into an Earth background region and a deep-space background region, and performs target detection and extraction based on the divided regions. This enables accurate detection and tracking of spatial targets in complex scenes, resulting in better spatial target detection performance.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and more specifically to a method for extracting spatial targets in complex scenes by combining camera imaging models. Background Technology

[0002] Space-based space target surveillance is an important development direction in the field of space target surveillance, playing a crucial role in areas such as space situational awareness and space debris removal. In actual space target surveillance, satellite-borne cameras need to possess certain search capabilities, and their fields of view are generally large. The Earth may enter the camera's field of view, resulting in a complex scene with backgrounds including deep space and the Earth. The different radiometric statistical characteristics of deep space and Earth backgrounds lead to background inhomogeneity. However, existing space target extraction methods typically only consider target extraction against a deep space background, resulting in poor detection and extraction accuracy for space targets in complex scenarios such as deep space and Earth backgrounds, making effective extraction of space targets difficult. Summary of the Invention

[0003] To address some or all of the technical problems existing in the prior art, the present invention provides a method for extracting spatial targets in complex scenes by combining a camera imaging model.

[0004] The technical solution of the present invention is as follows:

[0005] A method for extracting spatial targets in complex scenes by combining a camera imaging model is provided, the method comprising:

[0006] Acquire satellite camera images;

[0007] Based on satellite attitude and orbit information and camera imaging models, determine the Earth region in the image;

[0008] The image is divided into deep space and Earth regions;

[0009] Based on the image after region segmentation, image target detection is performed to determine the target region and target location information.

[0010] In some possible implementations, the Earth region in the image is determined based on satellite attitude and orbit information and camera imaging model, including:

[0011] Based on the satellite attitude and orbit information, determine the unit vector of the satellite camera pointing to the center of the Earth and the unit vector of the satellite camera pointing to the edge of the Earth in the geocentric inertial coordinate system.

[0012] Based on the camera imaging model, the unit vector in the geocentric inertial coordinate system is transformed into coordinates in the image coordinate system to determine the image coordinates of the Earth's edge point and the Earth's center.

[0013] Based on the image size, the image coordinates of the Earth's edge points and the Earth's center, determine the region of Earth in the image.

[0014] In some possible implementations, the unit vector pointing from the satellite camera to the Earth's edge is determined using the following formula:

[0015]

[0016] Where γ represents the unit vector of the satellite camera pointing to the edge of the Earth in the geocentric inertial coordinate system, k1 and k2 represent the scaling factors, α represents the unit vector of the satellite camera pointing to the Earth's center in the geocentric inertial coordinate system, β represents the direction of the optical axis of the satellite camera in the geocentric inertial coordinate system, R represents the semi-major axis of the Earth ellipsoid, h0 represents the height of the Earth's edge obstruction caused by the atmosphere, and L represents the distance from the satellite camera to the Earth's center.

[0017] In some possible implementations, the Earth region in the image is determined in the following way:

[0018] Using the image coordinates of the Earth's center as the center and the distance from the Earth's center to the edge of the Earth as the radius, draw a circle and take the overlapping part of the circular area and the image as the Earth region in the image.

[0019] In some possible implementations, image target detection is performed based on the segmented image to determine the target region and target location information, including:

[0020] Statistical information for deep space and Earth regions is determined, including image mean and standard deviation;

[0021] The thresholds corresponding to the deep space region and the Earth region are determined based on statistical parameters. The deep space region and the Earth region of the image are then segmented according to the corresponding thresholds to obtain a binary image.

[0022] Based on the obtained binary image, the bright areas are marked with connected components through morphological processing to extract the target region;

[0023] Calculate the target centroid position based on the extracted target region.

[0024] In some possible implementations, the threshold corresponding to the deep space region is determined using the following formula:

[0025] th1=μ1+q1σ1

[0026] The threshold corresponding to a region on Earth is determined using the following formula:

[0027] th2=μ2+q2σ2

[0028] Where th1 represents the threshold corresponding to the deep space region, μ1 and σ1 represent the image mean and standard deviation of the deep space region, respectively, th2 represents the threshold corresponding to the Earth region, μ2 and σ2 represent the image mean and standard deviation of the Earth region, respectively, and q1 and q2 represent the coefficients for controlling the false alarm rate.

[0029] In some possible implementations, the method further includes:

[0030] Continue acquiring satellite camera images. Based on the determined target area and target location information, use a space target tracking algorithm to track the target in the satellite camera images and generate the target trajectory in the image coordinate system.

[0031] In some possible implementations, the method further includes:

[0032] After continuously tracking the target in a preset number of frames, the target detection is performed again for the next frame to determine the target region and target location information. Based on the newly determined target region and target location information, continuous target tracking is performed on the subsequent preset number of frames.

[0033] In some possible implementations, the method further includes:

[0034] Calculate the target miss distance based on the determined target location information, and adjust the satellite camera optical axis according to the miss distance so that the satellite camera optical axis points to the target.

[0035] In some possible implementations, the method further includes:

[0036] After acquiring satellite camera images, the images are preprocessed using median filtering.

[0037] The main advantages of the technical solution of this invention are as follows:

[0038] The complex scene spatial target extraction method of the present invention, which combines camera imaging model, determines the presence of the Earth in the camera's field of view by using the camera imaging model, divides the image into Earth background region and deep space background region, and sets different thresholds for different regions to perform target detection and extraction. This method can achieve accurate detection and tracking of spatial targets in complex scenes and has better spatial target detection effect. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a method for extracting spatial targets in complex scenes by combining a camera imaging model according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of a satellite camera imaging a space target according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram illustrating the principle of a method for determining Earth regions in an image according to an embodiment of the present invention.

[0043] Figure 4 This is a schematic diagram of an original image provided in Example 1 of the present invention;

[0044] Figure 5 for Figure 4 A schematic diagram showing the corresponding Earth region identification results;

[0045] Figure 6 for Figure 5 Corresponding Earth region mask map;

[0046] Figure 7 This is a schematic diagram of the target extraction results in three consecutive frames of images provided in Example 1 of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] The technical solutions provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] refer to Figure 1 An embodiment of the present invention provides a method for extracting spatial targets in complex scenes by combining a camera imaging model. The method includes the following steps S1-S4:

[0050] Step S1: Acquire satellite camera images.

[0051] Specifically, the satellite camera's optical axis is directed according to the initially given guidance information, and the satellite camera is controlled to perform imaging and acquire images.

[0052] The guidance information is predetermined based on the possible area where the space target is located, so that the optical axis of the satellite camera is pointed to the approximate area where the space target is located.

[0053] Step S2: Based on the satellite attitude and orbit information and the camera imaging model, determine the Earth region in the image.

[0054] In one embodiment of the present invention, determining the Earth region in an image based on satellite attitude and orbit information and a camera imaging model further includes the following steps S21-S23:

[0055] Step S21: Based on the satellite attitude and orbit information, determine the unit vector of the satellite camera pointing to the Earth's center and the unit vector of the satellite camera pointing to the Earth's edge in the geocentric inertial coordinate system.

[0056] refer to Figure 2 The settings are as follows: In the geocentric inertial (ECI) coordinate system, the positions of the satellite camera, the Earth's center, and the target are A, O, and C, respectively. The unit vector pointing from the satellite camera to the Earth's center is α, and the unit vector pointing from the satellite camera's optical axis is β. The point on the edge of the Earth in the plane containing α and β is B, and the unit vector pointing from the satellite camera to the point on the edge of the Earth is γ.

[0057] Based on the satellite's orbit and attitude data, the unit vector pointing from the satellite camera to the Earth's center, α, and the unit vector pointing from the satellite camera's optical axis, β, can be directly calculated. Furthermore, since α and β are coplanar with γ, γ can be solved using α and β.

[0058] Specifically, let γ = k1α + k2β. Since |α| = |β| = |γ| = 1, then we have

[0059] Where k1 and k2 represent the set proportional coefficients.

[0060] Furthermore, let R represent the semi-major axis of the Earth's ellipsoid, h0 represent the height of atmospheric obstruction at the Earth's edge, and L represent the distance from the satellite camera to the Earth's center. Then, the angle θ between α and γ satisfies the following formula:

[0061]

[0062]

[0063] By combining the formulas and formula We can obtain: k1 = cosθ - k2αβ.

[0064] Therefore, in one embodiment of the present invention, the unit vector pointing from the satellite camera to the edge of the Earth can be determined using the following formula:

[0065]

[0066] Step S22: Based on the camera imaging model, the unit vector in the geocentric inertial coordinate system is transformed into coordinates in the image coordinate system to determine the image coordinates of the Earth's edge point and the Earth's center.

[0067] Specifically, the camera imaging model can be described as follows:

[0068]

[0069] Where m represents the proportionality coefficient. Let (x, y) represent the transformation matrices from the orbital coordinate system to the geocentric inertial coordinate system, from the satellite body coordinate system to the orbital coordinate system, and from the camera coordinate system to the satellite body coordinate system, respectively. Let (x, y) represent the image coordinates, and ψ v (x,y) and ψ a (x,y) represent the optical axis pointing angles perpendicular to the satellite orbit direction and along the satellite orbit direction, respectively, and τ and δ represent the right ascension and declination of the corresponding spatial vectors.

[0070] Through camera imaging models, the transformation from vectors in the geocentric inertial coordinate system to image coordinates can be achieved. Specifically, if the unit vector r in the geocentric inertial coordinate system is known... ECI Then the coordinates in the camera coordinate system are Then the coordinates in the camera coordinate system are converted to coordinates in the image coordinate system.

[0071] Based on the predetermined camera imaging model, the unit vector α of the satellite camera pointing to the Earth's center and the unit vector γ of the satellite camera pointing to the Earth's edge point, determined in step S22 above, can be transformed into coordinates in the image coordinate system. The resulting coordinates in the image coordinate system are the image coordinates of the Earth's edge point and the Earth's center.

[0072] Step S23: Determine the Earth region in the image based on the image size, the image coordinates of the Earth's edge points and the Earth's center.

[0073] refer to Figure 3 In one embodiment of the present invention, the Earth region in the image is determined in the following manner:

[0074] Using the image coordinates of the Earth's center O as the center and the distance from the Earth's center O to the Earth's adjacent point B as the radius, draw a circle and take the overlapping part of the circular area and the image as the Earth region in the image.

[0075] The region of the image in the image coordinate system is determined by the image coordinate position of the image center o and the image size.

[0076] Step S3: Divide the image into deep space region and Earth region.

[0077] Specifically, based on the Earth region in the image determined in step S2 above, the acquired satellite camera image is divided into two regions: the deep space region and the Earth region.

[0078] Step S4: Based on the image after region division, perform image target detection to determine the target region and target location information.

[0079] In one embodiment of the present invention, image target detection is performed based on the image after region division to determine the target region and target location information, further including the following steps S41-S44:

[0080] Step S41: Determine statistical information for deep space and Earth regions.

[0081] Specifically, based on the divided deep space region and Earth region, the image mean and standard deviation of the deep space region are calculated according to the image gray value, and the image mean and standard deviation of the Earth region are calculated to obtain statistical information including the image mean and standard deviation.

[0082] Step S42: Determine the thresholds corresponding to the deep space region and the Earth region based on statistical parameters, and perform threshold segmentation on the deep space region and the Earth region of the image according to the corresponding thresholds to obtain a binary image.

[0083] Considering the complex background and uneven intensity distribution of space imaging, directly using a global threshold to segment the image may lead to target detection and extraction failure. Since the grayscale values ​​of the Earth background and targets are higher than those of the deep space background, the targets may be within either the deep space or Earth background. Therefore, in one embodiment of this invention, different thresholds are used for image segmentation of the divided deep space region and Earth region.

[0084] Specifically, in one embodiment of the present invention, the threshold corresponding to the deep space region is set as follows:

[0085] th1=μ1+q1σ1

[0086] The threshold for the corresponding Earth region is set as follows:

[0087] th2=μ2+q2σ2

[0088] Where th1 represents the threshold corresponding to the deep space region, μ1 and σ1 represent the image mean and standard deviation of the deep space region, respectively, th2 represents the threshold corresponding to the Earth region, μ2 and σ2 represent the image mean and standard deviation of the Earth region, respectively, q1 and q2 represent the coefficients for controlling the false alarm rate, and q1 and q2 are set according to the actual situation.

[0089] Furthermore, for the deep space region in the image, image segmentation is performed using the following formula:

[0090]

[0091] For the Earth region in the image, image segmentation is performed using the following formula:

[0092]

[0093] Where f1(x,y) represents the gray value at coordinate (x,y) in the deep space region, B1(x,y) represents the gray value at coordinate (x,y) in the segmented deep space region, f2(x,y) represents the gray value at coordinate (x,y) in the Earth region, and B2(x,y) represents the gray value at coordinate (x,y) in the segmented Earth region.

[0094] By performing thresholding on the image, a corresponding binary image can be obtained.

[0095] Step S43: Based on the obtained binary image, the bright spot region is marked with connected components through morphological processing to extract the target region.

[0096] Specifically, based on the acquired binary image, morphological processing is used to mark the bright areas in the binary image as connected components, and the acquired connected components are used as the target regions.

[0097] Furthermore, to reduce false alarms, in one embodiment of the present invention, when extracting the target region, connected regions that are too large or too small are eliminated based on pixel size. That is, when extracting the target region, connected regions whose size meets preset requirements are selected as the target region. The preset region size requirement is set according to the actual situation.

[0098] Step S44: Calculate the target centroid position based on the extracted target region.

[0099] In one embodiment of the present invention, after the target region is extracted, the target centroid position is calculated based on the determined target region. The center of the target region can be selected as the target centroid.

[0100] Furthermore, in one embodiment of the present invention, the method further includes the following steps:

[0101] Step S5: Continue to acquire satellite camera images. Based on the determined target area and target location information, use a space target tracking algorithm to track the target in the satellite camera images and generate the target trajectory in the image coordinate system.

[0102] By using the target region and target location information extracted from the detection, and employing a space target tracking algorithm to track the target in subsequent multi-frame satellite camera images, it is possible to achieve fast and real-time target tracking.

[0103] In one embodiment of the present invention, if the determined targets include multiple targets, a space target tracking algorithm can be used to track each space target separately.

[0104] Among them, the spatial target tracking algorithm can adopt the kernel correlation filter (KCF) algorithm. The KCF algorithm uses cyclic offset for dense sampling, which can solve the problem of insufficient samples, and has better real-time performance through fast solution in the discrete Fourier domain.

[0105] Furthermore, in one embodiment of the present invention, step S5 may further include:

[0106] After continuously tracking the target in a preset number of frames, the target detection is performed again for the next frame to determine the target region and target location information. Based on the newly determined target region and target location information, continuous target tracking is performed on the subsequent preset number of frames.

[0107] By re-performing image target detection at preset intervals of a certain number of frames, it can adapt to complex situations such as existing targets leaving the camera's field of view and new targets entering the camera's field of view, thereby improving the accuracy of target detection and extraction.

[0108] Furthermore, in one embodiment of the present invention, both step S4 and step S5 may further include:

[0109] Calculate the target miss distance based on the determined target location information, and adjust the satellite camera optical axis according to the miss distance so that the satellite camera optical axis points to the target.

[0110] Specifically, during the detection and tracking of space targets, the miss distance of the target is calculated in real time based on the determined target position information, and the optical axis of the satellite camera is adjusted in real time according to the miss distance so that the optical axis of the satellite camera points to the target.

[0111] Here, the target miss distance represents the distance difference between the target's centroid and the center point of the image.

[0112] Furthermore, in one embodiment of the present invention, the method further includes:

[0113] After acquiring satellite camera images, the images are preprocessed using median filtering.

[0114] After space image acquisition, imaging noise can significantly impact the stability and accuracy of target detection and tracking. Therefore, median filtering of satellite camera images after acquisition effectively eliminates "salt and pepper" noise, ensuring the accuracy of subsequent target detection and tracking.

[0115] An embodiment of the present invention provides a method for extracting spatial targets in complex scenes by combining a camera imaging model. By utilizing the camera imaging model to determine the presence of the Earth in the camera's field of view, the image is divided into an Earth background region and a deep space background region. Different thresholds are set for different regions to perform target detection and extraction. This method can achieve accurate detection and tracking of spatial targets in complex scenes and has a better spatial target detection effect.

[0116] The following examples illustrate the beneficial effects of a complex scene spatial target extraction method combining a camera imaging model provided by an embodiment of the present invention:

[0117] In Example 1, a space target monitoring image is generated using photoelectric simulation software. Based on the generated space target monitoring image and its corresponding camera imaging model and attitude data, the complex scene space target extraction method combined with the camera imaging model provided in an embodiment of the present invention is used for target detection and tracking.

[0118] refer to Figures 4-6 The complex scene spatial target extraction method combined with camera imaging model provided in one embodiment of the present invention can accurately distinguish between the Earth background and the spatial background, so as to facilitate subsequent target detection and tracking processing.

[0119] refer to Figure 7 The method for extracting spatial targets in complex scenes by combining a camera imaging model, provided by one embodiment of the present invention, can achieve continuous and accurate detection and tracking of spatial targets in complex scenes.

[0120] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Additionally, the terms "front," "back," "left," "right," "upper," and "lower" in this document refer to the placement shown in the accompanying drawings.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting spatial targets of a complex scene in combination with a camera imaging model, characterized in that, The method comprises: acquiring a satellite camera image; determining an earth region in the image according to satellite attitude and orbit information and a camera imaging model; dividing the image into a deep space region and an earth region; performing image target detection based on the divided image to determine a target region and target position information; determining an earth region in the image according to satellite attitude and orbit information and a camera imaging model, comprising: determining a unit vector of a satellite camera pointing to the center of the earth and a unit vector of the satellite camera pointing to an earth limb point in a geocentric inertial coordinate system according to satellite attitude and orbit information; converting the unit vectors in the geocentric inertial coordinate system into coordinates in an image coordinate system to determine image coordinates of the earth limb point and the center of the earth according to a camera imaging model; determining an earth region in the image according to the image size, the image coordinates of the earth limb point and the center of the earth; the unit vector of the satellite camera pointing to the earth limb point is determined by the following formula: ； wherein, denotes the unit vector of the satellite camera pointing to the Earth limb point in the geocentric inertial coordinate system, and denotes the proportionality coefficient, denotes the unit vector of the satellite camera pointing to the Earth center in the geocentric inertial coordinate system, denotes the satellite camera optical axis pointing in the geocentric inertial coordinate system, denotes the Earth ellipsoid semi-major axis, denotes the Earth limb obscuration height caused by the atmosphere, denotes the satellite camera distance to the Earth center; the conversion of the unit vectors in the geocentric inertial coordinate system into coordinates in the image coordinate system is determined by the following formula: ； wherein, represents a proportionality coefficient, , , respectively represent a transformation matrix from the satellite body coordinate system to the camera coordinate system, a transformation matrix from the orbital coordinate system to the satellite body coordinate system, a transformation matrix from the geocentric inertial coordinate system to the orbital coordinate system, represents a unit vector in the geocentric inertial coordinate system; the earth region in the image is determined in the following manner: taking the image coordinate position of the center of the earth as the center and the distance from the center of the earth to the earth limb point as the radius to make a circle, and taking the overlapping part of the circular region and the image as the earth region in the image.

2. The method for extracting spatial targets of a complex scene based on a camera imaging model according to claim 1, wherein, performing image target detection based on the divided image to determine a target region and target position information, comprising: determining statistical information of the deep space region and the earth region, the statistical information including image mean and standard deviation; determining threshold values corresponding to the deep space region and the earth region according to the statistical parameters, and performing threshold segmentation on the deep space region and the earth region of the image according to the corresponding threshold values to obtain a binary image; performing connected component labeling on the bright spot region through morphological processing according to the obtained binary image to extract a target region; calculating the target centroid position according to the extracted target region.

3. The method for complex scene spatial target extraction based on camera imaging model according to claim 2, characterized in that, the threshold value corresponding to the deep space region is determined by the following formula: ； the threshold value corresponding to the earth region is determined by the following formula: ； wherein, denotes a threshold value corresponding to the deep space region, and denote the image mean and standard deviation of the deep space region, respectively, denotes a threshold value corresponding to the earth region, and denote the image mean and standard deviation of the earth region, respectively, and denote a coefficient controlling the false alarm rate.

4. The method for spatial target extraction of complex scenes coupled with camera imaging model according to claim 1, characterized in that, The method further comprises: continuing to acquire satellite camera images, and performing target tracking on the satellite camera images using a space target tracking algorithm based on the determined target region and target position information to generate a target trajectory in an image coordinate system.

5. The method for complex scene spatial target extraction based on camera imaging model according to claim 4, characterized in that, The method further comprises: after continuously tracking a preset number of images, re-performing image target detection to determine a target region and target position information for the next image, and continuously tracking a preset number of subsequent images based on the re-determined target region and target position information.

6. The method for extracting spatial targets of complex scenes based on camera imaging model according to any one of claims 1-5, characterized in that, The method further comprises: calculating the miss distance of the target according to the determined target position information, and adjusting the optical axis of the satellite camera to point to the target according to the miss distance.

7. The method for complex scene spatial target extraction based on camera imaging model according to claim 1, characterized in that, The method further comprises: after acquiring the satellite camera image, performing median filtering preprocessing on the image.

Images