Graph structure construction method based on multi-feature fusion

Abstract

The invention relates to the technical field of data processing, in particular to a graph structure construction method based on multi-feature fusion, which comprises the following steps of: collecting a multi-view image data set, generating a high-quality multi-view data set, and extracting basic feature representation of each sample for each view in the high-quality multi-view data set; and according to the generated basic feature representation, introducing an illumination condition to predict a future occurrence condition, and setting a time change-based reward function and a constraint condition to generate a high-quality graph structure. According to the method, the graph structure is stable and flexible, a clustering task and other tasks depending on the relationship between samples can be effectively supported, the capability of the model for future prediction is enhanced, and it is ensured that the model can provide accurate output in the continuously changing environment.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a graph structure construction method based on multi-feature fusion. Background Art

[0002] With the advent of the big data era, the amount of data has exploded, and how to effectively extract valuable information from massive data has become an important research topic. Especially in the fields of image processing, social network analysis, bioinformatics, etc., data often presents complex nonlinear relationships and high-dimensional characteristics. Traditional machine learning methods encounter bottlenecks when processing such complex data, especially when facing multi-view or multi-modal data, where the information of a single view or modality is not enough to fully describe the essential characteristics of the object.

[0003] In order to solve the above problems, researchers have proposed a variety of graph-based learning methods, aiming to capture the intrinsic connections between data by building graph structures. However, most existing graph learning methods focus on static features, that is, they assume that all samples remain unchanged throughout their life cycle, ignoring the impact of sample features changing over time. For example, in video surveillance systems, the appearance of target objects is affected by factors such as lighting and posture. Therefore, a graph structure construction method based on multi-feature fusion is designed. Summary of the invention

[0004] The purpose of the present invention is to provide a graph structure construction method based on multi-feature fusion to solve the problem that most of the graph learning methods proposed in the above background technology mainly focus on static features, that is, they assume that all samples remain unchanged throughout their life cycle and ignore the impact of sample features changing over time.

[0005] To achieve the above object, the present invention provides a method for constructing a graph structure based on multi-feature fusion, comprising the following steps:

[0006] S1. Collect a multi-view image dataset, clean, transform and standardize the image dataset to generate a high-quality multi-view dataset, and extract the basic feature representation of each sample for each view in the high-quality multi-view dataset;

[0007] S2. Based on the generated basic feature representation, an initial graph structure that reflects the relationship between samples is constructed for each view. The edge weights in the graph structure are adjusted by introducing lighting conditions to predict future situations. In the process of introducing lighting conditions to predict future situations, the influence of time differences on lighting conditions is considered for optimization.

[0008] S3. After adjusting the edge weights of the graph structure, set a time-based reward function and constraints to generate a high-quality graph structure. In the process of setting the reward function, introduce the influencing factors of drastic changes in sample feature similarity for optimization.

[0009] As a further improvement of the present technical solution, the multi-view image data set in S1 includes image samples from different viewing angles from 0 degrees to 180 degrees, and is used to provide different viewing angle information of the same image.

[0010] As a further improvement of the technical solution, in S2, an initial graph structure reflecting the relationship between samples is constructed for each view based on the generated basic feature representation, specifically:

[0011] S21. Based on the basic feature representation, the similarity between two samples is obtained using high-order statistics, and an initial graph structure is constructed based on the similarity. Each node of the graph structure represents an image sample, and the edge represents the similarity between two image samples.

[0012] S22. Introduce lighting conditions to predict future situations, adjust the weights of edges in the graph structure based on changes in lighting prediction, and then generate new edge weights.

[0013] As a further improvement of the technical solution, in S21, the similarity between the two samples is obtained according to the basic feature representation, as follows:

[0014]

[0015] Among them, sim(X i ,X j ) is the sample X i and X j Similarity between them; d is the dimension of the feature vector; k = 1, 2, ..., d; μ k (X i ) is the sample X i The mean value in the kth feature dimension; μ k (X j ) is the sample X j The mean value in the kth feature dimension; σ k (X i ) is the sample X i The standard deviation on the jth feature dimension; σ k (X j ) is the sample X j The standard deviation in the kth feature dimension; p is the adjustment parameter of the mean; q is the adjustment parameter of the standard deviation.

[0016] As a further improvement of the technical solution, the specific steps of S22 are as follows:

[0017] The difference in light intensity and light direction between the two samples is obtained, the change in light conditions is quantified, and then the degree of influence of the change in light conditions on the similarity is calculated; based on historical light data, the light conditions are predicted through time series analysis, and the predicted light conditions are combined with the actual light conditions to obtain the optimized light conditions; according to the adjusted light conditions, the difference in light intensity and light direction between the two samples is recalculated to obtain the adjusted weights.

[0018] As a further improvement of the technical solution, the S22 is specifically:

[0019] Differences in lighting conditions:

[0020] ΔL ij =|L i -L j ∣+α 0 ·|θ i -θ j ∣;

[0021] Among them, ΔL ij is the difference in illumination conditions between sample i and sample j; L i is the illumination condition of sample i; L j is the illumination condition of sample j; α 0 is the adjustment coefficient; θ i is the incident angle of the light source on sample i; θ j is the incident angle of the light source of sample j;

[0022] The impact of lighting conditions on similarity:

[0023]

[0024] in, is the edge weight after introducing the lighting condition; is the initial edge weight; λ is the adjustment coefficient; L i is the light intensity of sample i; L j is the light intensity of sample j;

[0025] Lighting conditions synthesis:

[0026]

[0027] Among them, L′ i The adjusted lighting condition for sample i; is the lighting condition predicted for sample i; L′ j The adjusted lighting condition for sample j; is the lighting condition predicted for sample j; α 1 is the weight coefficient;

[0028] Recompute the difference between the two samples:

[0029] ΔL′ ij =|L′ i -L′ j ∣+α 0 ·|θ′ i -θ′ j ∣;

[0030] Among them, ΔL′ ij is the difference in lighting conditions between sample i and sample j after adjustment;

[0031] Adjust edge weights based on predicted lighting:

[0032]

[0033] in, is the edge weight after introducing the predicted lighting conditions.

[0034] As a further improvement of the technical solution, in S2, in the process of introducing the lighting conditions to predict the future situation, the influence of the time difference on the lighting conditions is considered and optimized, and the optimization is specifically as follows:

[0035] Time-weighted average of lighting conditions:

[0036] It is known that the illumination conditions of sample i at different time points are L i (t 1 ),L i (t 2 ),…,L i (t n ), the prediction time is t pred , the predicted lighting conditions are calculated by weighted averaging:

[0037]

[0038] in, For time t pred Predicted lighting conditions; β 1 ,β 2 ,…,β n is the weight coefficient, which is assigned according to the credibility or importance of the lighting conditions at each time point;

[0039] Time decay factor adjustment:

[0040] γ(t i ,t j )=exp(-β·|t i -t j |);

[0041] Among them, γ(t i ,t j ) is the time attenuation factor; β is the attenuation coefficient; t i is the time of sample i; t j is the time of sample j;

[0042] Differences in lighting conditions that introduce time differences:

[0043]

[0044] Where, ΔL″ ij (t i ,t j ) is sample i at time t i When and sample j at time t j Differences in light intensity at different times; is sample i at time t i The predicted lighting conditions at is sample j at time t j The predicted lighting conditions at θ′ i (t i ) is sample i at time t i The direction of the light source at time θ′ j (t j ) is sample i at time t i The direction of the light source at ;

[0045] The final optimized edge weights are:

[0046]

[0047] in, is the optimized edge weight; is the initial sample i at time t i When and sample j at time t j The edge weight at .

[0048] As a further improvement of the technical solution, in S3, a reward function based on time changes is set, specifically:

[0049]

[0050] Among them, R time is a time-varying reward function.

[0051] As a further improvement of the technical solution, in S3, the influence factor of the drastic change of sample feature similarity is introduced in the process of setting the reward function for optimization, and the optimization is as follows:

[0052]

[0053] in, is the optimized reward function based on time variation; γ 0 is the influence coefficient of illumination difference on reward function; δ is the adjustment factor; sim(X i (t i ),X j (t j )) is the similarity between the features of sample i and sample j after evolution over time;

[0054]

[0055] Among them, R total is the overall reward function; S ij (t) is the similarity score between sample i and sample j at time t; λ(t) is the time decay factor.

[0056] As a further improvement of the present technical solution, in S3, the constraints include graph sparsity constraints, similarity consistency constraints and connectivity constraints.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. In this graph structure construction method based on multi-feature fusion, the influence of time differences on lighting conditions is considered and optimized in the process of introducing lighting conditions to predict future situations. The optimized edge weights not only reflect the physical distance and feature similarity between samples, but also incorporate lighting conditions and their changing characteristics over time, making the graph structure not only stable but also flexible, and able to effectively support clustering tasks and other tasks that rely on the relationship between samples.

[0059] 2. In this graph structure construction method based on multi-feature fusion, the influencing factors of drastic changes in sample feature similarity are introduced for optimization in the process of setting the reward function, which helps to avoid the model's excessive reliance on static features and pay more attention to dynamic changes, especially for application scenarios where features are prone to change with the environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flowchart of the overall process of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 As shown, a method for constructing a graph structure based on multi-feature fusion is provided, comprising the following steps:

[0063] S1. Collect a multi-view image dataset, clean, transform and standardize the image dataset to generate a high-quality multi-view dataset, and extract the basic feature representation of each sample for each view in the high-quality multi-view dataset;

[0064] Using a multi-view image dataset containing different perspectives can ensure that the information received by the model is more comprehensive and diverse, avoiding the limitations and biases that may be caused by a single perspective. For example, in face recognition tasks, if you only rely on frontal photos, the recognition accuracy may be affected by posture changes or occlusions; but if you can obtain side or even back angle information, it will help the model better capture individual features and maintain high accuracy even under non-ideal conditions.

[0065] The basic feature representation extracted based on high-quality multi-view datasets can not only enhance the model's understanding of the target object, but also provide a more solid foundation for modeling the relationship between nodes in the subsequent graph structure learning stage, ultimately achieving a more accurate and stable clustering effect. At the same time, this also provides an effective means to solve problems such as posture changes and differences in lighting conditions.

[0066] The multi-view image dataset in S1 includes image samples from different viewing angles from 0 to 180 degrees, which are used to provide different viewing angle information of the same image. It not only enriches the basic feature representation of each sample, but also greatly promotes the subsequent graph structure learning and feature fusion process. Specifically, by introducing information from multiple viewing angles, the model's ability to understand complex characteristics such as object shape and texture can be significantly improved, thereby improving the quality and robustness of clustering results.

[0067] S2. Based on the generated basic feature representation, an initial graph structure that reflects the relationship between samples is constructed for each view. The edge weights in the graph structure are adjusted by introducing lighting conditions to predict future situations. In the process of introducing lighting conditions to predict future situations, the influence of time differences on lighting conditions is considered for optimization.

[0068] In S2, based on the generated basic feature representation, an initial graph structure reflecting the relationship between samples is constructed for each view, specifically:

[0069] S21. Based on the basic feature representation, the similarity between two samples is obtained using high-order statistics, and an initial graph structure is constructed based on the similarity. Each node of the graph structure represents an image sample, and the edge represents the similarity between two image samples.

[0070] In S21, the similarity between two samples is obtained based on the basic feature representation, as follows:

[0071]

[0072] Among them, sim(X i ,X j ) is the sample X i and X j Similarity between them; d is the dimension of the feature vector; k = 1, 2, ..., d; μ k (X i ) is the sample X i The mean value in the kth feature dimension; μ k (X j ) is the sample X j The mean value in the kth feature dimension; σ k (X i ) is the sample X i The standard deviation in the kth feature dimension; σ k (X j ) is the sample X j μ is the standard deviation of the kth feature dimension; p is the adjustment parameter for the mean; q is the adjustment parameter for the standard deviation; the choice of these two parameters can be adjusted according to the specific distribution of the data or the needs of the problem. For example, when the distribution of some features in the data is more concentrated, the difference in standard deviation may be more important; in other cases, the difference in mean may be more significant. k (X i )-μ k (X j ) reflects the difference in the central tendency of two samples on a certain feature. If the mean difference of two samples on this feature dimension is large, their similarity on this dimension is low; σ k (X i )-σ k (X j ) reflects the difference in the degree of dispersion of two samples in a certain feature dimension. If the standard deviation of samples in a certain feature dimension is very different, it means that they have different degrees of variation in this feature, which in turn affects the similarity.

[0073] Traditional similarity calculation methods (such as Euclidean distance, cosine similarity, etc.) usually rely only on the comparison of a single feature, which may ignore the potential complex relationship between samples in multiple dimensions. And often rely only on the distance measurement in the feature space, unable to fully consider the statistical differences between features, especially when the sample distribution has significant differences in some dimensions. By introducing the comparison of mean and standard deviation, this method can more robustly reflect the differences between samples, especially in the presence of outliers or uneven data distribution, and consider the mean difference and standard deviation difference of each dimension at the same time, not only measuring the central tendency (mean) of the sample in each feature dimension, but also capturing the breadth or dispersion (standard deviation) of the feature distribution.

[0074] S22, introduce lighting conditions to predict future situations, adjust the weights of edges in the graph structure based on changes in lighting prediction, and then generate new edge weights;

[0075] For example, in some application scenarios, if two samples become more similar in the future, the strength of the connection between them can be appropriately increased now. This forward-looking design makes the final graph structure closer to actual changes and enhances the model's ability to adapt to future events.

[0076] The specific steps of S22 are as follows:

[0077] Obtain the difference in light intensity and light direction between the two samples, quantify the change in light conditions, and then calculate the degree of influence of the change in light conditions on the similarity; predict light conditions through time series analysis based on historical light data, combine the predicted light conditions with the actual light conditions to obtain optimized light conditions; recalculate the difference in light intensity and light direction between the two samples based on the adjusted light conditions, and then obtain the adjusted weights;

[0078] S22 is specifically:

[0079] Differences in lighting conditions:

[0080] ΔL ij =|L i -L j ∣+α 0 ·|θ i -θ j ∣;

[0081] Among them, ΔL ij is the difference in illumination conditions between sample i and sample j; L i is the illumination condition of sample i; L j is the illumination condition of sample j; α 0 is the adjustment coefficient used to control the contribution of the light source angle to the illumination difference; θ iis the incident angle of the light source on sample i; θ j is the incident angle of the light source of sample j;

[0082] The impact of lighting conditions on similarity:

[0083]

[0084] in, is the edge weight after introducing the lighting condition; is the initial edge weight, that is, the edge weight without considering the lighting conditions; λ is the adjustment coefficient, which indicates the impact of the difference in lighting conditions on the similarity; L i is the light intensity of sample i; L j is the light intensity of sample j; taking the light conditions into consideration, this formula allows the model to dynamically adjust the correlation strength between samples, thereby better adapting to image features in different environments. This step helps to improve the quality of the graph structure, making it more in line with the actual situation, thereby improving the clustering effect and the robustness of the model.

[0085] Among them, the lighting conditions of sample i and sample j are specifically:

[0086] Extracting light intensity is based on the average brightness method, calculating the average brightness of each image. If the image data is an RGB image, then:

[0087]

[0088] Among them, R ik is the red component of each pixel in the image; G ik is the green component of each pixel in the image; B ik is the blue component of each pixel in the image; N is the number of pixels in the image;

[0089] If the image data is a grayscale image, you only need to calculate the average pixel value of the image;

[0090] Determining the direction of the light source is based on the illumination estimation model. By estimating the angle between the surface normal and the light source, the direction of the light source irradiating the image surface is inferred. This usually requires relying on the depth information of the image or three-dimensional reconstruction.

[0091] Lighting conditions synthesis:

[0092]

[0093] Among them, L′ i The adjusted lighting condition for sample i; is the lighting condition predicted for sample i; L′ j The adjusted lighting condition for sample j; is the lighting condition predicted for sample j; α1 is the weight coefficient that controls the fusion ratio of actual lighting conditions and predicted lighting conditions. This value can be set according to different application scenarios, usually determined by the reliability of the prediction; the synthesis of lighting conditions provides corrected lighting information for further optimization of the graph structure, so that the model can foresee the future trend of lighting changes to a certain extent. This forward-looking design not only enhances the model's adaptability to future scenarios, but also improves its ability to respond to emergencies, such as rapidly changing weather or lighting environments.

[0094] Recompute the difference between the two samples:

[0095] ΔL′ ij =|L′ i -L′ j ∣+α 0 ·|θ′ i -θ′ j ∣;

[0096] Among them, ΔL′ ij is the difference in lighting conditions between sample i and sample j after adjustment; the purpose of recalculating the lighting condition difference is to ensure that the graph structure can reflect the latest lighting status in a timely manner and avoid misjudgment caused by sudden changes in lighting.

[0097] Adjust edge weights based on predicted lighting:

[0098]

[0099] in, The edge weights after introducing the predicted lighting conditions;

[0100] Assume that we have an image dataset containing multiple views, and we have built an initial graph structure for each view. Now, we want to introduce lighting conditions to further optimize this graph. First, we collect the actual lighting conditions L of all samples. i and L j Then, the lighting influence factor is calculated according to the above formula, and the weight of each edge is adjusted accordingly. Finally, we will get a graph structure corrected by lighting conditions, in which the edge weights more accurately reflect the similarity relationship between nodes;

[0101] By fine-tuning the differences in lighting conditions, the resulting edge weights are closer to real-world situations, which helps build a graph structure that is both stable and flexible. Such a graph structure not only improves the performance of clustering tasks, but also lays a solid foundation for other tasks that rely on relationships between samples (such as classification, recommendation systems, etc.).

[0102] In S2, when introducing lighting conditions to predict future situations, the influence of time differences on lighting conditions is considered for optimization. The optimization is as follows:

[0103] Time-weighted average of lighting conditions:

[0104] When we make illumination predictions at different time points, we can combine historical illumination conditions with predicted illumination conditions by weighted averaging. Introducing illumination predictions in the time dimension helps the model better adapt to changes in illumination over time and improves the accuracy of estimating future illumination conditions.

[0105] It is known that the illumination conditions of sample i at different time points are L i (t 1 ),L i (t 2 ),…,L i (t n ), the prediction time is t pred , the predicted lighting conditions are calculated by weighted averaging:

[0106]

[0107] in, For time t pred Predicted lighting conditions; β 1 ,β 2 ,…,β n is the weight coefficient, which is assigned according to the credibility or importance of the lighting conditions at each time point;

[0108] In order to make the model adapt to the changes in light over time, we can introduce a time decay factor when calculating the light impact factor, so that the longer the time interval, the smaller the impact of light differences on similarity. This can avoid unnecessary light difference effects over a long time span. The time decay factor is adjusted as follows:

[0109] γ(t i ,t j )=exp(-β·|t i -t j |);

[0110] Among them, γ(t i ,t j ) is the time decay factor, which controls the influence of time difference on similarity; β is the decay coefficient, which controls the influence intensity of time difference on similarity; t i is the time of sample i; t jis the time of sample j; the role of the time decay factor is to reduce the impact of unnecessary illumination differences over a long time span on the similarity evaluation, so that the model pays more attention to the most recent data. This is very important for maintaining the sensitivity and timeliness of the model, especially when dealing with rapidly changing environments.

[0111] Differences in lighting conditions with time differences:

[0112]

[0113] Where, ΔL″ ij (t i ,t j ) is sample i at time t i When and sample j at time t j Differences in light intensity at different times; is sample i at time t i The predicted lighting conditions at is sample j at time t j The predicted lighting conditions at θ′ i (t i ) is sample i at time t i The direction of the light source at time θ′ j (t j ) is sample i at time t i The direction of the light source at ;

[0114] By introducing time differences, the formula can more accurately capture the changing trend of lighting conditions over time and avoid misjudgments caused by static analysis. This step enhances the model's ability to understand dynamic environments, enabling it to make more reasonable judgments when lighting conditions are complex and changing.

[0115] The final optimized edge weights are:

[0116]

[0117] in, is the optimized edge weight; is the initial sample i at time t i When and sample j at time t j The edge weight at time (if there is no time influence, the edge weight is fixed);

[0118] The optimized edge weights not only reflect the physical distance and feature similarity between samples, but also incorporate the lighting conditions and their changing characteristics over time. This design makes the graph structure not only stable but also flexible, and can effectively support clustering tasks and other tasks that rely on the relationship between samples (such as classification, recommendation systems, etc.). In addition, this fine-tuning also helps improve the performance of the model in complex lighting environments, ensuring its long-term stability and accuracy.

[0119] S3. After adjusting the edge weights of the graph structure, set a time-based reward function and constraints to generate a high-quality graph structure. In the process of setting the reward function, the influence of the drastic changes in sample feature similarity is introduced for optimization; in order to further improve the quality of the graph structure, a time-based reward function is set in this step, and the influence of the drastic changes in sample feature similarity is introduced as the optimization target. The reward function encourages those sample relationships that remain stable over time, while punishing those abnormal fluctuations caused by external factors (such as light, weather, etc.). In addition, some necessary constraints are imposed, such as limiting the maximum and minimum edge weight ranges, to ensure the rationality of the graph structure.

[0120] In S3, a reward function based on time changes is set, specifically:

[0121] Considering that the illumination and similarity of nodes change over time, the reward function should also consider the impact of changes in time factors on similarity. Based on time series data, we can design a reward function based on time differences to reward graph structures that have less changes and stronger stability in the time dimension:

[0122]

[0123] Among them, R time is a reward function based on time variation;

[0124] This formula defines a reward function R based on the time difference time , which is used to measure the change of the relationship between nodes in the graph structure over time. Specifically, it is calculated by the exponential decay factor exp(-β·|t i -t j |) to quantify the effect of time interval on similarity, where β is a parameter that controls the intensity of the effect of time difference; at the same time, the difference in lighting conditions |L i (t i )-L j (t j )| to reflect the degree of illumination change, ensuring that node pairs with more stable illumination conditions can obtain higher rewards.

[0125] In S3, the factors affecting the drastic changes in sample feature similarity are introduced in the process of setting the reward function for optimization. After optimization, the specific results are:

[0126] Considering that the similarity and lighting conditions of samples i and j change over time, the time difference will affect the relationship between them. In the reward function, it is hoped that the change of edge weights will be penalized by the time difference, that is, if two samples change significantly in time, the edge weight between them should be reduced accordingly.

[0127]

[0128] in, is the optimized reward function based on time variation; γ 0 is the influence coefficient of illumination difference on the reward function; δ is the adjustment factor used to control the influence of sample feature similarity change on the reward function; sim(X i (t i ),X j (t j )) is the similarity between the features of sample i and sample j after evolution over time;

[0129] Optimized reward function We further consider the changes in sample feature similarity over time. In addition to the original time decay term and illumination difference term, a new term is added to evaluate the impact of changes in sample feature similarity; the introduction of factors affecting drastic changes in sample feature similarity is to more accurately capture the trend of the evolution of the relationship between samples over time. This approach helps avoid the model's over-reliance on static features, and instead pays more attention to dynamic changes, especially for application scenarios where features are prone to change with the environment (for example, object recognition under weather conditions), enhancing the model's temporal adaptability and enabling it to better cope with complex and changing situations in the real world.

[0130]

[0131] Among them, R total is the overall reward function; S ij (t) is the similarity score between sample i and sample j at time t; λ(t) is the time decay factor, which indicates the influence of the overall stability and structural changes of the graph over time;

[0132] Overall reward function R totalIt can not only accurately evaluate the stability and consistency between samples within the graph structure, but also effectively guide the model to focus on those sample connections that remain relatively stable over a long period of time. This design is particularly suitable for applications that require long-term tracking of objects or maintenance of specific patterns, such as behavioral analysis in video surveillance or user interaction pattern mining in social networks. In addition, it also promotes the effect of multi-feature fusion and improves the expressiveness and robustness of the model when dealing with complex tasks.

[0133] In S3, the constraints include graph sparsity constraints, similarity consistency constraints, and connectivity constraints;

[0134] In order to avoid too many edge connections in the graph, a sparsity constraint of the graph is introduced. A maximum edge limit or expected edge density can be set to control the complexity of the graph structure. In order to ensure that the sample connections in the graph meet the expectations of similarity, a similarity consistency constraint is designed, requiring that the edge weights in the graph should be consistent with the actual similarity between samples. For example, if the similarity between sample (node) i and sample (node) j in the feature space is high, the edge weight between them should be large. In order to ensure the connectivity of the graph, ensure that each node is connected to at least one other node through an edge, so the connectivity constraint of the graph is introduced. For example, it can be ensured that the generated graph is connected, or at least has one connected subgraph.

[0135] Through the design of the reward function and the introduction of constraints, the resulting graph structure will have the following characteristics:

[0136] High-quality edge connections: The edge weights between nodes with high similarity are large, and the lighting differences and time factors are also effectively considered;

[0137] Graph sparsity: avoid graph structures being too dense, control the number of edges, and improve computational efficiency;

[0138] Graph connectivity: Ensure that the graph has reasonable connectivity and avoid isolated nodes;

[0139] Stability: In a time-varying environment, the generated graph can adapt to changes in illumination and dynamic changes in similarity. The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected.

Claims

1. A graph structure construction method based on multi-feature fusion, characterized in that: The steps include: S1. Collect a multi-view image dataset, clean, transform and standardize the image dataset to generate a high-quality multi-view dataset, and extract the basic feature representation of each sample for each view in the high-quality multi-view dataset; S2. Based on the generated basic feature representation, an initial graph structure that reflects the relationship between samples is constructed for each view. The edge weights in the graph structure are adjusted by introducing lighting conditions to predict future situations. In the process of introducing lighting conditions to predict future situations, the influence of time differences on lighting conditions is considered for optimization. S3. After adjusting the edge weights of the graph structure, set a time-based reward function and constraints to generate a high-quality graph structure. In the process of setting the reward function, introduce the influencing factors of drastic changes in sample feature similarity for optimization.

2. The method for constructing a graph structure based on multi-feature fusion according to claim 1, characterized in that: The multi-view image dataset in S1 includes image samples from different viewing angles ranging from 0 degrees to 180 degrees, and is used to provide different viewing angle information of the same image.

3. The method for constructing a graph structure based on multi-feature fusion according to claim 2, characterized in that: In S2, based on the generated basic feature representation, an initial graph structure reflecting the relationship between samples is constructed for each view, specifically: S21. Based on the basic feature representation, the similarity between two samples is obtained using high-order statistics, and an initial graph structure is constructed based on the similarity. Each node of the graph structure represents an image sample, and the edge represents the similarity between two image samples. S22. Introduce lighting conditions to predict future situations, adjust the weights of edges in the graph structure based on changes in lighting prediction, and then generate new edge weights.

4. The method for constructing a graph structure based on multi-feature fusion according to claim 3, characterized in that: In S21, the similarity between the two samples is obtained according to the basic feature representation, as follows: Among them, sim(X i , X j ) is the sample X i and X j Similarity between them; d is the dimension of the feature vector; k = 1, 2, ..., d; μ k (X i ) is the sample X i The mean value in the kth feature dimension; μ k (X j ) is the sample X j The mean value in the kth feature dimension; σ k (X i ) is the sample X i The standard deviation in the kth feature dimension; σ k (X j ) is the sample X j The standard deviation in the kth feature dimension; p is the adjustment parameter of the mean; q is the adjustment parameter of the standard deviation.

5. The method for constructing a graph structure based on multi-feature fusion according to claim 4, characterized in that: The specific steps of S22 are as follows: Obtain the difference in light intensity and light direction between two samples, quantify the change in light conditions, and then calculate the degree of influence of the change in light conditions on similarity; predict light conditions through time series analysis based on historical light data, combine the predicted light conditions with the actual light conditions, and obtain optimized light conditions; According to the adjusted lighting conditions, the difference in lighting intensity and lighting direction between the two samples is recalculated to obtain the adjusted weights.

6. The method for constructing a graph structure based on multi-feature fusion according to claim 5, characterized in that: The S22 is specifically: Differences in lighting conditions: ΔL ij =|L i -L j |+α0·|θ i -θ j |; Where, ΔL ij is the difference in illumination conditions between sample i and sample j; L i is the illumination condition of sample i; L j is the illumination condition of sample j; α0 is the adjustment coefficient; θ i is the incident angle of the light source on sample i; θ j is the incident angle of the light source of sample j; The impact of lighting conditions on similarity: in, is the edge weight after introducing the lighting condition; is the initial edge weight; λ is the adjustment coefficient; L i is the light intensity of sample i; L j is the light intensity of sample j; Lighting conditions synthesis: Among them, L′ i The adjusted lighting condition for sample i; is the lighting condition predicted for sample i; L′ j The adjusted lighting condition for sample j; is the lighting condition predicted for sample j; α1 is the weight coefficient; Recompute the difference between the two samples: ΔL′ ij =|L′ i -L′ j |+α0·|θ′ i -θ′ j |; Among them, ΔL′ ij is the difference in lighting conditions between sample i and sample j after adjustment; Adjust edge weights based on predicted lighting: in, is the edge weight after introducing the predicted lighting conditions.

7. The method for constructing a graph structure based on multi-feature fusion according to claim 6, characterized in that: In S2, when introducing the illumination condition to predict the future situation, the influence of the time difference on the illumination condition is considered to be optimized, and the optimization is as follows: Time-weighted average of lighting conditions: It is known that the illumination conditions of sample i at different time points are L i (t1), L i (t2), ..., L i (t n ), the prediction time is t pred , the predicted lighting conditions are calculated by weighted averaging: in, For time t pred Predicted lighting conditions; β1, β2, ..., β n is the weight coefficient, which is assigned according to the credibility or importance of the lighting conditions at each time point; Time decay factor adjustment: γ(t i ,t j )=exp(-β·|t i -t j |); Among them, γ(t i , t j ) is the time decay factor; β is the decay coefficient; t i is the time of sample i; t j is the time of sample j; Differences in lighting conditions with time differences: Where, ΔL ij (t i , t j ) is sample i at time t i When and sample j at time t j Differences in light intensity at different times; is sample i at time t i The predicted lighting conditions at is sample j at time t j The predicted lighting conditions at θ′ i (t i ) is sample i at time t i The direction of the light source at time θ′ j (t j ) is sample i at time t i The direction of the light source at ; The final optimized edge weights are: in, is the optimized edge weight; is the initial sample i at time t i When and sample j at time t j The edge weight at .

8. The method for constructing a graph structure based on multi-feature fusion according to claim 7, characterized in that: In S3, a reward function based on time variation is set, specifically: Among them, R time is a time-varying reward function.

9. The method for constructing a graph structure based on multi-feature fusion according to claim 8, characterized in that: In S3, the factors affecting the drastic changes in sample feature similarity are introduced in the process of setting the reward function for optimization. After optimization, the specific results are: in, is the optimized reward function based on time change; γ0 is the influence coefficient of illumination difference on the reward function; δ is the adjustment factor; sim(X i (t i ), X j (t j )) is the similarity between the features of sample i and sample j after evolution over time; Among them, R total is the overall reward function; S ij (t) is the similarity score between sample i and sample j at time t; λ(t) is the time decay factor.

10. The method for constructing a graph structure based on multi-feature fusion according to claim 9, characterized in that: In S3, the constraints include graph sparsity constraints, similarity consistency constraints and connectivity constraints.

Images