Satellite video eigendecomposition method, system, storage medium and device based on moving target energy constraint

Abstract

Satellite video intrinsic decomposition method, system, storage medium and device based on moving target energy constraint. The present invention relates to a satellite video intrinsic decomposition method, system, storage medium and device. The purpose of the present invention is to solve the problem of light change and moving target energy dispersion in satellite video. The method includes the following process: step 1: constructing a satellite video intrinsic decomposition model; step 2: constructing a reflectivity time domain constraint item based on satellite video background information; step 3: constructing a moving target energy constraint item based on satellite video moving target information; step 4: constructing a reflectivity local constraint item based on satellite video reflection prior; step 5: optimizing and solving the satellite video reflectivity component based on the reflectivity time domain constraint item, the moving target energy constraint item and the reflectivity local constraint item. The present invention belongs to the field of remote sensing image processing.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing image processing, and relates to eigendecomposition of satellite video, and specifically to a method, system, storage medium and equipment for eigendecomposition of satellite video based on energy constraints of moving targets. Background Art

[0002] In recent years, with the successful launch of video satellites, dynamic observation of the earth's surface has become possible, which has attracted more and more attention and made target tracking applications possible. Video satellites can provide rich data for regional dynamic change monitoring, such as enemy reconnaissance, wide-area monitoring, disaster monitoring, and traffic flow estimation. However, video satellites use staring imaging mode. During the imaging process, they modify their attitude angles to achieve staring imaging of a specific area. This physical process will cause obvious light changes in satellite videos, which will have a negative impact on satellite video target tracking applications.

[0003] At present, the existing satellite video intrinsic decomposition methods mainly focus on the continuous characteristics of the video to ensure the stability of the reflectivity component in the time domain. They do not consider the application of moving targets in satellite videos. The target energy in the reflectivity component is dispersed, which is of limited help to subsequent applications. Therefore, the satellite video intrinsic decomposition algorithm for moving targets is still a problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of light variation and energy dispersion of moving targets in satellite video, and to propose a satellite video intrinsic decomposition method, system, storage medium and device based on moving target energy constraint.

[0005] The satellite video intrinsic decomposition method based on moving target energy constraint includes the following steps:

[0006] Step 1: Construct satellite video intrinsic decomposition model;

[0007] Step 2: Construct reflectivity time domain constraint items based on satellite video background information;

[0008] Step 3: Construct the moving target energy constraint item based on the moving target information of the satellite video;

[0009] Step 4: Based on the satellite video reflectivity prior, construct a local constraint on reflectivity;

[0010] Step 5: Based on the reflectivity time domain constraint, the moving target energy constraint and the reflectivity local constraint, optimize and solve the satellite video reflectivity component.

[0011] The satellite video eigendecomposition system based on moving target energy constraint is used to execute the satellite video eigendecomposition method based on moving target energy constraint.

[0012] A storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement a satellite video intrinsic decomposition system based on moving target energy constraints.

[0013] The satellite video intrinsic decomposition device based on moving target energy constraint comprises a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to realize the satellite video intrinsic decomposition system based on moving target energy constraint.

[0014] The beneficial effects of the present invention are:

[0015] In order to solve the problem of light changes in satellite videos, the present invention utilizes moving targets and background information to perform intrinsic video decomposition on satellite videos, with the aim of obtaining the satellite video reflectivity component, removing the illumination changes while ensuring the energy concentration of the moving target, achieving target enhancement, and improving the application performance of satellite video target tracking.

[0016] The present invention proposes a satellite video intrinsic decomposition method based on moving target energy constraint, which utilizes the background and moving target characteristics of satellite video to effectively solve the illumination change in satellite video, maintain the time domain continuity of reflectivity component, realize moving target enhancement, and improve the application performance of satellite video target tracking.

[0017] The present invention discloses a satellite video intrinsic decomposition method based on moving target energy constraint, which jointly utilizes the background and moving target information in the satellite video, constructs the reflectivity time domain constraint and the moving target energy constraint, and completes the decomposition of the reflectivity component without moving target energy loss from the satellite video. The implementation process is as follows: (1) inputting satellite video data, constructing the reflectivity time domain constraint term for the background information; (2) constructing the moving target energy constraint term for the moving target information; (3) constructing the reflectivity local constraint term based on the reflection prior; (4) optimizing and solving the satellite video reflectivity component. The present invention can extract the satellite video reflectivity component, remove the influence of illumination change, and does not lose the target energy. The method is simple and easy to implement.

[0018] In order to solve the problem of complex illumination changes in videos caused by the gaze imaging process of satellite videos, the present invention jointly utilizes background information and moving target information in satellite videos to construct a satellite video intrinsic decomposition model, reduce the impact of satellite video illumination changes, maintain the continuity of reflectivity components between frames, concentrate target energy, improve target resolution, and contribute to the improvement of the accuracy of subsequent satellite video target tracking and other applications.

[0019] In order to verify the performance of the algorithm proposed in the present invention, experiments were conducted on video data captured by Jilin-1 video. The experimental results verified the effectiveness of the satellite video eigendecomposition method based on moving target energy constraint proposed in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the implementation of the present invention;

[0021] Figure 2 This is the original satellite video data of Jilin-1;

[0022] Figure 3 It is the reflectivity component map of the original Jilin-1 satellite video;

[0023] Figure 4 It is a local target map of Jilin-1 original satellite video data;

[0024] Figure 5 It is the reflectivity component map of local targets in the original Jilin-1 satellite video data. DETAILED DESCRIPTION

[0025] Specific implementation method 1: In this implementation method, a satellite video intrinsic decomposition method based on moving target energy constraint includes the following process:

[0026] Step 1: Construct satellite video intrinsic decomposition model;

[0027] Step 2: Construct reflectivity time domain constraint items based on satellite video background information;

[0028] Step 3: Construct the moving target energy constraint item based on the moving target information of the satellite video;

[0029] Step 4: Based on the satellite video reflectivity prior, construct a local constraint on reflectivity;

[0030] Step 5: Based on the reflectivity time domain constraint, the moving target energy constraint and the reflectivity local constraint, optimize and solve the satellite video reflectivity component.

[0031] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the satellite video intrinsic decomposition model is constructed in step 1; the specific process is:

[0032] Generally speaking, image intrinsic decomposition decomposes the image into two intrinsic components, namely the brightness component and the reflectivity component related only to the material itself:

[0033] I=ρ×L

[0034] Among them, I, ρ, L represent the original image, reflectivity component and brightness component respectively, and × represents pixel-by-pixel corresponding multiplication;

[0035] Satellite video has the characteristics of static background, a small number of dynamic targets and complex illumination changes. Combining the characteristics of satellite video, a satellite video intrinsic decomposition model is proposed:

[0036] I=ρ(b)×L(b)+ρ(m)×L(m)

[0037] The satellite video intrinsic decomposition model is divided into two parts: the decomposition of background pixels and moving target pixels;

[0038] Where b and m represent background pixels and moving target pixels respectively; ρ(b) represents the reflectance component of satellite video background pixels; L(b) represents the brightness component of satellite video background pixels; ρ(m) represents the reflectance component of satellite video moving target pixels; L(m) represents the brightness component of satellite video moving target pixels; I represents the original satellite video image; × represents pixel-by-pixel corresponding multiplication.

[0039] The other steps and parameters are the same as those in the first embodiment.

[0040] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that, in step 2, a reflectivity time domain constraint item is constructed for the satellite video background information; the specific process is:

[0041] First, consider the background information. Satellite video has the characteristic of unchanged background and the proportion of background pixels is very large. Therefore, based on the assumption that the background pixels of two adjacent frames in the satellite video have the same reflectivity, the difference in background pixel reflectivity is minimized to maintain the temporal consistency of the satellite video reflectivity component, and the following relationship can be obtained:

[0042]

[0043] Among them, E t (ρ t ∣ρ t-1 ) represents the reflectivity time domain constraint term, ρ t (b), ρ t-1 (b) represent the reflectivity components of the satellite video background pixel b at time t and time t-1 respectively, and B represents the set of satellite video background pixels.

[0044] The image intrinsic decomposition model I = ρ × L is expressed in the logarithmic domain, and I = ρ + L is obtained. For the convenience of representation, I, ρ, and L are still used to represent the original image, reflectivity component, and brightness component in the logarithmic domain. In the logarithmic domain, multiplication operations are expressed as addition operations. Using the above equation, ρ t (b) = I t (b)-L t (b) Substituting into the reflectivity time domain constraint, we get:

[0045]

[0046] Among them, I t (b) represents the background pixel of the satellite video at time t, L t (b) represents the brightness and darkness components of the background pixels in the satellite video at time t, E t (L t ∣I t ,ρ t-1 ) indicates that in the known I t and ρ t-1 Under the condition of t .

[0047] The other steps and parameters are the same as those in the first or second embodiment.

[0048] Specific implementation method 4: This implementation method is different from any one of the specific implementation methods 1 to 3 in that in step 3, a moving target energy constraint item is constructed for the moving target information of the satellite video; the specific process is:

[0049] Satellite video imaging can realize dynamic observation of a certain area or target, such as moving vehicles, ships and aircraft. Although intrinsic decomposition can remove the influence of illumination, it will disperse the energy of the moving target. However, for the intrinsic decomposition of satellite video, it is expected to eliminate the illumination change while also hoping that the energy of the moving target will not be dispersed, but concentrated in the reflectivity component to improve the contrast and distinction between the target and the background. Therefore, a moving target energy constraint is proposed. This constraint linearly represents the light and dark components of the moving target pixel with its surrounding background. The purpose is to make the light and dark components of the moving target pixel similar to the background, and the target signal energy can be concentrated in the reflectivity component. Therefore, the following relationship is obtained:

[0050]

[0051]

[0052] Among them, E m (L t ) represents the energy constraint term of the moving target, L t (m) represents the light and dark components of the moving target pixel m in the satellite video at time t, L t (n) represents the light and dark components of the background pixels n around the moving target pixel m in the satellite video at time t, m represents the moving target pixel in the satellite video, M represents the set of moving target pixels in the satellite video, n represents the background pixels around the moving target pixel m in the satellite video, and B mrepresents the set of background pixels around the moving target pixel m in the satellite video (the set of background pixels in a 5×5 area centered on the moving target pixel m in the satellite video), D(m,n) represents the correlation measure between pixels m and n, and d(m,n) represents the distance between pixels m and n.

[0053] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0054] Specific implementation method 5: This implementation method is different from any one of the specific implementation methods 1 to 4 in that, in step 4, a reflectivity local constraint item is constructed based on the satellite video reflectivity prior; the specific process is as follows:

[0055] For the reflectivity component, the same material has the same reflectivity, so the reflectivity component has a strong correlation in the local area. Based on the optic nerve (Retinex) theory, the reflectivity component and the light and dark component can be distinguished by the image gradient. When the image gradient is large, it can be considered to be caused by the reflectivity difference. When the image gradient is small, it can be considered to be caused by the light and dark component. Therefore, the local constraint term of the reflectivity is constructed, as shown in the following relationship:

[0056]

[0057] Among them, E l (ρ t ,L t ) represents the local constraint term of reflectivity, L t (p) represents the brightness and darkness components of pixel p in the satellite video at time t, L t (q) represents the brightness and darkness component of pixel q in the satellite video at time t, ρ t (p) represents the reflectivity component of pixel p in the satellite video at time t, ρ t (q) represents the reflectivity component of pixel q at time t in the satellite video, α (p,q) represents the weight, p and q refer to adjacent pixel pairs, and N represents the set of all adjacent pixel pairs;

[0058] The reflectivity component and the brightness component are determined by α (p,q) To balance, α (p,q) Determined by calculating the chromaticity difference:

[0059]

[0060] in, represents the chromaticity of pixel p, represents the chromaticity of pixel q, and t′ represents the chromaticity threshold;

[0061] The chromaticity threshold t′ is a basic parameter for determining whether the chromaticity change in the image comes from the reflectance or the light and dark components. t=0.003 is set, and the parameter t′ uses the gradient information of the chromaticity to decompose the image.

[0062] Using the eigendecomposition model I=ρ+L in the logarithmic domain, ρ t =I t -L t Substitution get:

[0063]

[0064]

[0065]

[0066]

[0067] Among them, ΔL t Indicates L t (p) and L t (q), ΔI t Indicates I t (p) and I t (q) difference, ΔL t =L t (p)-L t (q), ΔI t =I t (p)-I t (q), I t (p) represents the pixel size, I t (q) represents the pixel size q, and ΔI t Can be considered as a constant;

[0068] E l (ρ t ,L t |I t ) indicates that in the known I t Under the condition of t , L t ; E l (L t |I t ) indicates that in the known I t Under the condition of t .

[0069] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0070] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that, in step 4, the satellite video reflectivity component is optimized and solved based on the reflectivity time domain constraint term, the moving target energy constraint term and the reflectivity local constraint term; the specific process is:

[0071] Arranging the above three cost functions, the total cost function can be obtained as follows:

[0072] E=E l +βE t +λE m

[0073]

[0074]

[0075] Among them, E l 、E t and E m They represent the local reflectivity constraint, the temporal reflectivity constraint and the moving target energy constraint respectively; β and λ are the weights of the latter two constraints, respectively, and are set to 1000; I t (m) represents the size of the moving target pixel m in the satellite video at time t;

[0076] E(L t ∣I t ,ρ t-1 ) indicates that in the known I t , t-1 Under the condition of t ;

[0077] Solution Medium t The process is:

[0078]

[0079] It is about L t The quadratic function form is E(L t ∣I t ,ρ t-1 ) is converted into a quadratic function matrix form, as shown below:

[0080]

[0081] The above formula is the matrix form of the optimization function;

[0082] Among them, A and b are respectively t ∣I t ,ρ t-1 ) is composed of the coefficients of the quadratic term and the coefficients of the linear term; Indicates L t The transpose of represents the transpose of b, c represents the constant term; L t Including L t (p) and L t (q);

[0083] Since L in the formula t (p) and L t (q) Subject to spatial constraints, the matrix A is a positive definite symmetric matrix;

[0084] Since the quadratic function matrix Where A is a positive definite symmetric matrix. According to the convex optimization algorithm, the quadratic function matrix It has a global minimum value. The minimum value of the optimization function has the form of an analytical solution. In the entire optimization process, the global optimal solution can be obtained, avoiding falling into the local value problem. The quadratic function matrix The analytical solution is:

[0085] AL t =b

[0086] The obtained L t To satisfy L t ;

[0087] According to the eigendecomposition representation in the logarithmic domain, through the light and dark components L t , the reflectivity component ρ can be obtained t :

[0088] ρ t =I t -L t

[0089] ρ t represents the reflectivity component of the moving target pixel in the satellite video at time t; L t I represents the brightness and darkness components of the moving target pixel in the satellite video at time t; t Represents the original satellite video image at time t.

[0090] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0091] Specific implementation method seven: This implementation method is a satellite video intrinsic decomposition system based on moving target energy constraints, and the system is used to execute a satellite video intrinsic decomposition method based on moving target energy constraints.

[0092] Specific implementation eight: This implementation is a storage medium, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement the satellite video intrinsic decomposition system based on energy constraints of moving targets.

[0093] It should be understood that any method described in the present invention may be provided as a computer program product, software or computerized method, which may include a non-transitory machine-readable medium having instructions stored thereon, and the instructions may be used to program a computer system or other electronic device. The storage medium may include, but is not limited to, magnetic storage media, optical storage media; magneto-optical storage media include: read-only memory ROM, random access memory RAM, erasable programmable memory (e.g., EPROM and EEPROM) and flash memory layer; or other types of media suitable for storing electronic instructions.

[0094] Specific implementation nine: This implementation is a satellite video intrinsic decomposition device based on moving target energy constraint, the device includes a processor and a memory, it should be understood that including any device including a processor and a memory described in the present invention, the device may also include other units and modules that perform display, interaction, processing, control, etc. and other functions through signals or instructions;

[0095] At least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the satellite video intrinsic decomposition system based on moving target energy constraint.

[0096] Experiment designed by the present invention:

[0097] The data used in the experiment is a set of Jilin-1 satellite video image data. Figure 2 It is the original video data of Jilin-1 satellite; Figure 3 is the reflectivity component map of the satellite video data; Figure 4 It is a local target map of Jilin-1 original satellite video data;

[0098] Figure 5 is the reflectivity component diagram of the local target; Table 1 compares the target tracking accuracy of the intrinsic decomposition result of the method of the present invention with the original data. From the comparison results, it can be seen that the reflectivity component generated by the method proposed by the present invention can effectively improve the application of satellite video target tracking.

[0099] Table 1 Comparison of target tracking accuracy between the intrinsic decomposition results of the method of the present invention and the original data

[0100]

[0101] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A satellite video intrinsic decomposition method based on moving target energy constraints, characterized by: The method comprises the following steps: Step 1: Construct satellite video intrinsic decomposition model; Step 2: Construct reflectivity time domain constraint items based on satellite video background information; Step 3: Construct the moving target energy constraint item based on the moving target information of the satellite video; Step 4: Based on the satellite video reflectivity prior, construct a local constraint on reflectivity; Step 5: Based on the reflectivity time domain constraint, the moving target energy constraint and the reflectivity local constraint, optimize and solve the satellite video reflectivity component; In step 1, a satellite video intrinsic decomposition model is constructed; the specific process is as follows: Satellite video intrinsic decomposition model: I=ρ(b)×L(b)+ρ(m)×L(m) The satellite video intrinsic decomposition model is divided into two parts: the decomposition of background pixels and moving target pixels; Where b and m represent background pixels and moving target pixels respectively; ρ(b) represents the satellite video background pixel reflectance component; L(b) represents the brightness and darkness components of the background pixels in the satellite video; ρ(m) represents the reflectivity component of the moving target pixel in the satellite video; L(m) represents the brightness and darkness components of the moving target pixel in the satellite video; I represents the original satellite video image; × represents pixel-by-pixel corresponding multiplication; In step 2, a reflectivity time domain constraint item is constructed for the satellite video background information; the specific process is as follows: Among them, E t (ρ t ∣ρ t-1 ) represents the reflectivity time domain constraint term, ρ t (b), ρ t-1 (b) represents the reflectivity component of the satellite video background pixel b at time t and time t-1, respectively, and B represents the set of satellite video background pixels; Using the eigendecomposition model I = ρ + L in the logarithmic domain, ρ t (b) = I t (b)-L t (b) Substituting into the reflectivity time domain constraint, we get: Among them, I t (b) represents the background pixel of the satellite video at time t, L t (b) represents the brightness and darkness components of the background pixels in the satellite video at time t, E t (L t ∣I t ,ρ t-1 ) indicates that in the known I t and ρ t-1 Under the condition of t , ρ represents the reflectivity component, and L represents the brightness component; In step 3, a moving target energy constraint item is constructed for the moving target information of the satellite video; the specific process is: Among them, E m (L t ) represents the energy constraint term of the moving target, L t (m) represents the light and dark components of the moving target pixel m in the satellite video at time t, L t (n) represents the light and dark components of the background pixels n around the moving target pixel m in the satellite video at time t, m represents the moving target pixel in the satellite video, M represents the set of moving target pixels in the satellite video, n represents the background pixels around the moving target pixel m in the satellite video, and B m represents the set of background pixels around the moving target pixel m in the satellite video, D(m,n) represents the correlation measure between pixels m and n, and d(m,n) represents the distance between pixels m and n; In step 3, based on the reflection prior, a local constraint term of reflectivity is constructed; the specific process is as follows: Among them, E l (ρ t ,L t ) represents the local constraint term of reflectivity, L t (p) represents the brightness and darkness components of pixel p in the satellite video at time t, L t (q) represents the brightness and darkness component of pixel q in the satellite video at time t, ρ t (p) represents the reflectivity component of pixel p in the satellite video at time t, ρ t (q) represents the reflectivity component of pixel q at time t in the satellite video, α (p,q) represents the weight, p and q refer to adjacent pixel pairs, and N represents the set of all adjacent pixel pairs; The reflectivity component and the brightness component are determined by α (p,q) To balance, α (p,q) Determined by calculating the chromaticity difference: in, represents the chromaticity of pixel p, represents the chromaticity of pixel q, and t′ represents the chromaticity threshold; Using the eigendecomposition model I = ρ + L in the logarithmic domain, ρ t =I t -L t Substitution Where, ΔL t Indicates L t (p) and L t (q), ΔI t Indicates I t (p) and I t (q) difference, ΔL t =L t (p)-L t (q), ΔI t =I t (p)-I t (q), I t (p) represents the pixel size, I t (q) represents the pixel size q, and ΔI t is a constant; E l (ρ t ,L t |I t ) indicates that in the known I t Under the condition of t , L t ; E l (L t |I t ) indicates that in the known I t Under the condition of t ; In step 4, based on the reflectivity time domain constraint term, the moving target energy constraint term and the reflectivity local constraint term, the satellite video reflectivity component is optimized and solved; the specific process is: Arranging the above three cost functions, the total cost function can be obtained as follows: E=E l +βE t +λE m Among them, E l 、E t and E m They represent the local reflectivity constraint, the temporal reflectivity constraint and the moving target energy constraint respectively; β and λ are the weights of the latter two constraints respectively; I t (m) represents the size of the moving target pixel m in the satellite video at time t; E(L t ∣I t ,ρ t-1 ) indicates that in the known I t , t-1 Under the condition of t ; Solution Medium t The process is: E(L t ∣I t ,ρ t-1 ) is converted into a quadratic function matrix form, as shown below: Among them, A and b are respectively t ∣I t ,ρ t-1 ) is composed of the coefficients of the quadratic term and the coefficients of the linear term; Indicates L t The transpose of represents the transpose of b, c represents the constant term; L t Including L t (p) and L t (q); Since L in the formula t (p) and L t (q) Subject to spatial constraints, the matrix A is a positive definite symmetric matrix; Since the quadratic function matrix Where A is a positive definite symmetric matrix. According to the convex optimization algorithm, the quadratic function matrix The analytical solution is: <h2 style=";text-align:left;direction:ltr">AL<h2 style=";text-align:left;direction:ltr"> t <h2 style=";text-align:left;direction:ltr"> (b) The obtained L t To satisfy L t ; Through the light and dark component L t , the reflectivity component ρ can be obtained t : r t =I t -L t ρ t represents the reflectivity component of the moving target pixel in the satellite video at time t; L t I represents the brightness and darkness components of the moving target pixel in the satellite video at time t; t Represents the original satellite video image at time t.

2. Satellite video intrinsic decomposition system based on moving target energy constraint, characterized by: The system is used to execute the satellite video eigendecomposition method based on moving target energy constraint as described in claim 1.

3. A storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the satellite video intrinsic decomposition system based on moving target energy constraint as claimed in claim 2.

4. Satellite video intrinsic decomposition device based on moving target energy constraint, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the satellite video intrinsic decomposition system based on moving target energy constraint as claimed in claim 2.

Images