Space-based and air-based target cooperative identification method and device

Abstract

The application provides a space-based and air-based target cooperative identification method and device, relates to the technical field of image recognition, and comprises the following steps: inputting a first image collected by space-based collection and a second image collected by air-based collection into a target identification model, determining a first evidence intensity vector corresponding to the first image and a second evidence intensity vector corresponding to the second image through the target identification model; determining a first subjective opinion based on the first evidence intensity vector and a second subjective opinion based on the second evidence intensity vector, wherein the first subjective opinion comprises a first trust degree vector and a first single-source uncertainty, and the second subjective opinion comprises a second trust degree vector and a second single-source uncertainty; when it is determined that there is no target association error based on the first trust degree vector and the second trust degree vector, determining a target category based on the first trust degree vector, the second trust degree vector, the first single-source uncertainty and the second single-source uncertainty. The application can improve the reliability of target cooperative identification.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and apparatus for collaborative target recognition based on space-based and air-based systems. Background Technology

[0002] With the rapid development of aerospace technology, space-based and airborne multi-source collaborative remote sensing has become a key technology for Earth observation and target identification. By coordinating space-based and airborne platforms such as satellites and UAVs, an integrated three-dimensional observation network can be constructed, enabling all-weather, all-day, and highly timely monitoring of key areas, with wide applications in multiple fields.

[0003] Space-based and air-based multi-source collaborative target identification enhances the ability to identify ground targets by fusing remote sensing data from different platforms. Compared with a single data source, multi-source information fusion can comprehensively utilize the advantages of each source data, thereby obtaining more refined and accurate identification results. It has become a core component of intelligent interpretation of space-based and air-based remote sensing data.

[0004] However, due to the significant heterogeneity of remote sensing data acquired by space-based and air-based platforms, there are certain difficulties in information fusion. Moreover, in actual processing, situations such as target association errors may occur in the front-end stages, that is, images of different targets are mistakenly paired as the same target. All of the above factors will lead to low reliability of the recognition results when space-based multi-source collaborative target recognition is performed. Summary of the Invention

[0005] This invention provides a method and apparatus for collaborative target identification based on space-based and air-based systems, which addresses the shortcomings of low reliability of identification results in existing technologies for multi-source collaborative target identification based on space-based systems, thereby improving the reliability of collaborative identification.

[0006] This invention provides a target collaborative identification method based on space-based and air-based systems, comprising: The first image acquired by space-based acquisition and the second image acquired by air-based acquisition are input into the target recognition model. The target recognition model determines the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image. Each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category, and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category. A first subjective opinion is determined based on the first evidence strength vector, and a second subjective opinion is determined based on the second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty. The second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector is used to characterize the degree of certainty that the target in the first image belongs to the corresponding category. Each element in the second confidence vector is used to characterize the degree of certainty that the target in the second image belongs to the corresponding category. If it is determined that there is no target association error based on the first confidence vector and the second confidence vector, the target category of the collaborative recognition output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty.

[0007] According to the present invention, a space-based and air-based target collaborative recognition method is provided, wherein determining the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model includes: The first depth feature is extracted from the first image and the second depth feature is extracted from the second image through the feature extraction network in the target recognition model. The first deep feature is mapped to the first evidence strength vector through the evidence mapping network in the target recognition model, and the second deep feature is mapped to the second evidence strength vector.

[0008] According to the present invention, a space-based and air-based target collaborative identification method is provided, wherein determining a first subjective opinion based on a first evidence strength vector includes: Based on the first evidence strength vector, a first total evidence is determined, which is used to characterize the degree of confidence of the space-based system in the classification results; The first trust vector is determined based on the first evidence strength vector and the first total amount of evidence; The first single-source uncertainty is determined based on the total amount of the first piece of evidence and the preset total number of categories.

[0009] According to the present invention, a target collaborative identification method based on space-based and air-based systems, wherein determining that there is no target association error based on the first confidence vector and the second confidence vector includes: For all distinct category indices p and q, the element corresponding to category index p in the first confidence vector is multiplied by the element corresponding to category index q in the second confidence vector to obtain multiple product terms; Summing the multiple product terms yields the conflict coefficient; If the conflict coefficient is less than a preset threshold, it is determined that there is no target association error.

[0010] According to the present invention, a space-based and air-based target collaborative identification method is provided, wherein the target category output by the target identification model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty, including: Based on the first trust vector, the second trust vector, the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient, a fused trust vector is determined; The fused uncertainty is determined based on the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient. The product vector is obtained by determining the product between each element in the fusion uncertainty and the corresponding element in the preset basis. The sum of each element in the fusion trust vector and each element in the product vector is determined to obtain the decision probability vector, and each element in the decision probability vector is used to characterize the fusion decision probability of the corresponding category. The target category is determined based on the fusion decision probability of each category.

[0011] According to the present invention, a target collaborative identification method based on space-based and air-based systems is provided, the method further comprising: Based on the elements corresponding to the target category in the first confidence vector, the elements corresponding to the target category in the second confidence vector, the elements corresponding to the target category in the base rate, the first single-source uncertainty, and the second single-source uncertainty, the first marginal benefit of the space-based system and the second marginal benefit of the air-based system are determined. Based on the first marginal benefit and the second marginal benefit, the contribution of the space-based system to the target category and the contribution of the air-based system to the target category are determined respectively.

[0012] According to the present invention, a target collaborative identification method based on space-based and air-based systems is provided, the method further comprising: If a target association error is determined based on the first confidence vector and the second confidence vector, a first decision probability vector is determined based on the first confidence vector and the first single-source uncertainty, and a second decision probability vector is determined based on the second confidence vector and the second single-source uncertainty. The target recognition model outputs the first decision probability vector and the second decision probability vector.

[0013] According to the present invention, a target collaborative identification method based on space-based and air-based systems is provided, wherein the target identification model is trained in the following manner: A training dataset is constructed, which includes multiple sets of training samples. Each set of training samples includes a first sample image from the space-based system, positive sample images from the air-based system that are correctly associated with the target in the first sample image, and negative sample images that are incorrectly associated with the target in the first sample image. The positive sample pairs and negative sample pairs in each group of training samples are respectively input into the initial target recognition model to obtain the single-source decision probability of positive sample pairs, the fusion decision probability of positive sample pairs and the single-source decision probability of negative sample pairs output by the initial target recognition model for each category. The positive sample pair includes the first sample image and the positive sample image, and the negative sample pair includes the first sample image and the negative sample image. Based on the single-source decision probability of the positive sample pair and the fusion decision probability of the positive sample pair, a first classification loss is determined, and based on the single-source decision probability of the negative sample pair, a second classification loss is determined. Based on the first classification loss and the second classification loss, the model parameters of the initial target recognition model are updated to obtain the target recognition model.

[0014] According to the present invention, a space-based and air-based target collaborative identification method is provided, wherein updating the model parameters of the initial target identification model based on the first classification loss and the second classification loss to obtain the target identification model includes: Based on the positive sample pairs, the positive sample conflict coefficient is determined, and based on the negative sample pairs, the negative sample conflict coefficient is determined. The conflict loss is determined based on the preset positive sample loss weight coefficient, the preset negative sample loss weight coefficient, the positive sample conflict coefficient, and the negative sample conflict coefficient. The total loss is determined based on the first classification loss, the second classification loss, and the conflict loss; The model parameters of the initial target recognition model are updated based on the total loss to obtain the target recognition model.

[0015] The present invention also provides a target collaborative identification device based on space-based and air-based systems, comprising: The determination module is used to input the first image acquired by space-based acquisition and the second image acquired by air-based acquisition into the target recognition model, and determine the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model. Each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category, and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category. The determining module is further configured to determine a first subjective opinion based on the first evidence strength vector and a second subjective opinion based on the second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty. The second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector is used to characterize the degree of certainty that the target in the first image belongs to the corresponding category. Each element in the second confidence vector is used to characterize the degree of certainty that the target in the second image belongs to the corresponding category. The output module is used to obtain the target category of the collaborative recognition output by the target recognition model based on the first trust vector, the second trust vector, the first single-source uncertainty, and the second single-source uncertainty, when it is determined that there is no target association error based on the first trust vector and the second trust vector.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the target collaborative identification method based on space-based and air-based methods described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target collaborative identification method based on space-based and air-based methods as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the target collaborative identification method based on space-based and air-based methods as described above.

[0019] The present invention provides a target collaborative identification method and apparatus based on space-based and air-based systems. By inputting a first image acquired by space-based systems and a second image acquired by air-based systems into a target identification model, the target identification model determines a first evidence strength vector corresponding to the first image and a second evidence strength vector corresponding to the second image. Based on the first evidence strength vector, a first subjective opinion is determined, and based on the second evidence strength vector, a second subjective opinion is determined. The first subjective opinion includes a first confidence vector and a first single-source uncertainty; the second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector represents the degree of certainty that the target in the first image belongs to the corresponding category, and each element in the second confidence vector represents the degree of certainty that the target in the second image belongs to the corresponding category. If, based on the first and second confidence vectors, it is determined that there is no target association error, the target category output by the target identification model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty. Because the conversion process between evidence strength vector and subjective opinion can provide a unified evidentiary representation of data from different imaging mechanisms and resolutions, and explicitly measure the reliability of judgments from each data source through single-source uncertainty, even if the data from different sources are heterogeneous, information complementarity and fusion can be adaptively achieved based on the quantified confidence vector and uncertainty, thereby improving the effectiveness of collaborative utilization of heterogeneous source data. Furthermore, since it can effectively detect and reject potential target association errors in the pre-processing stages, avoiding the impact of erroneous data association on the fusion results, it can improve the robustness and reliability of recognition results in space-based multi-source collaborative target identification. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts illustrating the target collaborative identification method based on space-based and air-based systems provided in this embodiment of the invention.

[0022] Figure 2 This is the second flowchart illustrating the target collaborative identification method based on space-based and air-based systems provided in this embodiment of the invention.

[0023] Figure 3 This is a schematic diagram of the structure of a space-based and air-based target collaborative identification device provided in an embodiment of the present invention.

[0024] Figure 4This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] Space-based platforms (satellites) have the advantages of wide-area coverage and periodic surveys, while air-based platforms (UAVs, aircraft) have the advantages of high mobility and high-resolution close-range detailed survey capabilities. The collaboration of the two can form a three-dimensional observation mode of "wide-area discovery - close-range detailed survey". Therefore, building a multi-source collaborative remote sensing system integrating space and air has become an important development direction for improving the timeliness and intelligence of remote sensing observation.

[0027] Target identification based on collaborative multi-source remote sensing data from the sky can achieve more refined and accurate results compared to single-source target identification by fusing information from multiple sources, making it a core component of sky-based remote sensing data interpretation. However, due to the significant heterogeneity of sky-based multi-source remote sensing data, such as differences between optical and synthetic aperture radar (SAR) imaging modes, and multi-scale variations in target size / orientation, and the possibility of errors such as target association errors in the front-end processes (mistakenly pairing images of different targets as the same target), the reliability of target identification results in sky-based multi-source collaborative target identification is relatively low.

[0028] In view of the above-mentioned problems, this invention proposes a target collaborative identification method based on space-based and air-based systems. This method, based on deep networks, explicitly transforms images collected from heterogeneous data sources in space-based and air-based systems into belief assignments in evidence theory, and measures the degree of conflict and uncertainty between data sources online. Even if the data from different data sources are heterogeneous, it can effectively accommodate and quantify the quality differences of evidence from each data source, achieving robust information fusion and compensation. Furthermore, based on the calculated degree of conflict between data sources, it can effectively detect and reject potential target association errors in the pre-processing stage, avoiding the impact of erroneous data association on the fusion results. This improves the reliability of the identification results in space-based multi-source collaborative target identification.

[0029] The following is combined with Figure 1 and Figure 2The target collaborative identification method based on space-based and airborne systems provided in this invention is described below. This invention is applicable to various space-based collaborative sensing scenarios where high accuracy, reliability, and scene adaptability of target identification are required.

[0030] The subject executing this method can be a terminal device, computer, server, server cluster, or specially designed space-based and air-based target collaborative identification device, or a space-based and air-based target collaborative identification device installed in the electronic device. The space-based and air-based target collaborative identification device can be implemented by software, hardware, or a combination of both.

[0031] Figure 1 This is one of the flowcharts illustrating the space-based and air-based target collaborative identification method provided in this embodiment of the invention, such as... Figure 1 As shown, the method includes: Step 101: Input the first image acquired by space-based acquisition and the second image acquired by air-based acquisition into the target recognition model, and determine the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model.

[0032] In this vector, each element of the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category, and each element of the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category.

[0033] Specifically, space-based can be satellites, while air-based can be drones, aircraft, etc.

[0034] The target recognition model is an end-to-end trainable model built on a deep neural network. It can extract discriminative features from the input first and second images and output structured first and second evidence strength vectors. The target recognition model is obtained through supervised training on a large number of labeled image samples.

[0035] The length of the first evidence strength vector is the same as the number of all categories to be identified. Each element in the first evidence strength vector represents the degree of support or strength of evidence that the target in the first image belongs to a specific category. A higher value indicates that the target recognition model has found more features in the first image that support the target belonging to that category.

[0036] Similarly, the length of the second evidence strength vector is the same as the number of all categories to be identified. Each element in the second evidence strength vector represents the degree of support or strength of evidence that the target in the second image belongs to a specific category. A higher value indicates that the target recognition model has found more features in the second image to support the target's classification as that category.

[0037] By determining the first evidence strength vector and the second evidence strength vector, it is possible to display and quantify the strength and degree of support of images collected from each data source for each category of evidence.

[0038] Step 102: Determine the first subjective opinion based on the first evidence strength vector, and determine the second subjective opinion based on the second evidence strength vector.

[0039] The first subjective opinion includes a first confidence vector and a first single-source uncertainty, and the second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector is used to characterize the degree of confidence that the target in the first image belongs to the corresponding category, and each element in the second confidence vector is used to characterize the degree of confidence that the target in the second image belongs to the corresponding category.

[0040] Specifically, the first confidence vector has the same dimension as the first evidence strength vector, and its length is equal to the number of all categories to be identified. Each element value in the first confidence vector represents the degree of confidence that the target in the first image belongs to a certain category. The sum of the confidence of all categories is usually less than or equal to 1.

[0041] The first single-source uncertainty is a single scalar value used to quantify the overall uncertainty or lack of understanding that exists when target recognition is based solely on information from a single first image.

[0042] The second confidence vector has the same dimension as the second evidence strength vector, and its length is equal to the number of all categories to be identified. Each element value in the second confidence vector represents the degree of confidence that the target in the second image belongs to a certain category. The sum of the confidence of all categories is usually less than or equal to 1.

[0043] The second single-source uncertainty is also a separate scalar value used to quantify the overall uncertainty or lack of understanding that exists when target recognition is based solely on information from a single second image.

[0044] By converting the first evidence strength vector into the first subjective opinion and the second evidence strength vector into the second subjective opinion, the images collected from each data source can be normalized into a belief structure with clear semantics. The first and second confidence vectors can represent the support strength distribution for each category, while the first and second single-source uncertainties can characterize the uncertainty in target recognition caused by image quality or model cognitive limitations.

[0045] Step 103: If it is determined that there is no target association error based on the first confidence vector and the second confidence vector, the target category of the collaborative recognition output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty and the second single-source uncertainty.

[0046] Specifically, the degree of conflict between the first and second confidence vectors can be calculated. For example, the conflict can be determined by comparing the differences in confidence distribution across categories. If the degree of conflict is below a preset threshold, it is determined that no target association error has occurred in the previous processing stage, meaning that the first and second images describe the same target. In this case, multi-source information will be fused, and target recognition will be performed based on the fused information.

[0047] If no target association error is found, the first confidence vector and the second confidence vector are synthesized, and the first single-source uncertainty and the second single-source uncertainty are used as key parameters in the fusion calculation to finally generate a fused confidence vector and a more accurate overall uncertainty.

[0048] Furthermore, based on the fused trust vector and the overall uncertainty, the fusion decision probability corresponding to each category can be determined, and the final target category after collaborative recognition can be determined based on this fusion decision probability. For example, the category corresponding to the highest fusion decision probability can be determined as the target category. Finally, the target category is output through the target recognition model.

[0049] The target collaborative recognition method based on space-based and air-based systems provided in this invention involves inputting a first image acquired by space-based systems and a second image acquired by air-based systems into a target recognition model. The target recognition model determines a first evidence strength vector corresponding to the first image and a second evidence strength vector corresponding to the second image. Based on the first evidence strength vector, a first subjective opinion is determined, and based on the second evidence strength vector, a second subjective opinion is determined. The first subjective opinion includes a first confidence vector and a first single-source uncertainty; the second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector represents the degree of certainty that the target in the first image belongs to the corresponding category, and each element in the second confidence vector represents the degree of certainty that the target in the second image belongs to the corresponding category. If no target association error is determined based on the first and second confidence vectors, the target category output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty. Because the conversion process between evidence strength vector and subjective opinion can provide a unified evidentiary representation of data from different imaging mechanisms and resolutions, and explicitly measure the reliability of judgments from each data source through single-source uncertainty, even if the data from different sources are heterogeneous, information complementarity and fusion can be adaptively achieved based on the quantified confidence vector and uncertainty, thereby improving the effectiveness of collaborative utilization of heterogeneous source data. Furthermore, since it can effectively detect and reject potential target association errors in the pre-processing stages, avoiding the impact of erroneous data association on the fusion results, it can improve the robustness and reliability of recognition results in space-based multi-source collaborative target identification.

[0050] For example, based on the above embodiments, when determining the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model, the first depth feature can be extracted from the first image and the second depth feature can be extracted from the second image through the feature extraction network in the target recognition model. Then, the first depth feature can be mapped to the first evidence strength vector and the second depth feature can be mapped to the second evidence strength vector through the evidence mapping network in the target recognition model.

[0051] Specifically, the target association results determined by the target search algorithm for space-based and air-based systems can be used to... In the input target recognition model, where, This represents the i-th first image from the space-based system. Let L represent the i-th second image from the empty base, and L represent the total number of first and second images.

[0052] The target recognition model constructs modal-independent feature extraction networks for each data source, which can be used to identify the target. From the first image Extracting the first depth feature And through feature extraction network From the second image Extracting second depth features The feature extraction network can be a backbone network based on joint spatial and frequency domain learning to enhance the robust representation capability of heterogeneous remote sensing data.

[0053] Furthermore, the first and second deep features can be input into the evidence mapping network for transformation. Based on evidence learning theory, the Dirichlet distribution is a priori of the K-class probability distribution; therefore, the evidence mapping network can be used for transformation. and Mapping the first and second depth features to Dirichlet distribution parameters and Dirichlet distribution parameters As the first evidence strength vector, and using the Dirichlet distribution parameters As the second evidence strength vector. and As shown in formulas (1) and (2) respectively: (1) (2) Where K represents the total number of types to be identified. This represents the strength of evidence that the target in the first image belongs to the k-th category. This represents the strength of evidence that the target in the second image belongs to the k-th category. The softplus activation function ensures that all elements of the output Dirichlet distribution parameter are greater than or equal to 1.

[0054] In this embodiment, by extracting the first depth feature and the second depth feature, and mapping the first depth feature and the second depth feature to the first evidence strength vector and the second evidence strength vector, the original remote sensing image data from different imaging mechanisms and with significant heterogeneity can be uniformly transformed into evidence expressions with clear mathematical meaning and statistical interpretation. This can display and quantify the confidence of each data source in the classification results, and provide a foundation for subsequent fusion of multi-source information.

[0055] For example, based on the above embodiments, when determining the first subjective opinion based on the first evidence strength vector, it can be done in the following way: Based on the first evidence strength vector, the first total evidence is determined. The first total evidence is used to characterize the degree of confidence of the space-based system in the classification results. Based on the first evidence strength vector and the first total evidence, the first confidence vector is determined. Based on the first total evidence and the preset total number of categories, the first single-source uncertainty is determined.

[0056] Specifically, the total amount of first evidence can be obtained by summing all elements in the first evidence strength vector. , that is Through the first total amount of evidence It can represent the sum of evidence collected from the first image that supports all categories, and its magnitude reflects the overall confidence of the space-based system in the classification results.

[0057] In addition, the first trust vector can be determined according to formulas (3) and (4). and the first single-source uncertainty : (3) (4) Among them, the first trust vector This can also be understood as the basic probability allocation for each category. , express The k-th element in the array, where K represents the total number of categories to be identified.

[0058] Based on the first trust vector and the first single-source uncertainty The first subjective opinion can be determined. ,in, Generally, a high-quality, well-defined data source will produce a larger total amount of evidence and lower uncertainty, and vice versa.

[0059] Similarly, based on the second evidence strength vector, according to the formula... Determine the total amount of second evidence And in the same way as in formulas (3) and (4), the second trust vector is determined. Second single-source uncertainty Thus, the second subjective opinion corresponding to the empty base is determined. .

[0060] In this embodiment, by converting the evidence strength vector into subjective opinion, the original evidence strengths with different dimensions and distributions output by each data source can be standardized into a unified belief expression that includes a clear confidence distribution and uncertainty scalar. This provides a key input for subsequent fusion decision-making that can quantify and distinguish the reliability of the judgments made by each data source.

[0061] For example, based on the above embodiments, when determining whether there is a target association error based on the first trust vector and the second trust vector, it can be done in the following way: For all distinct category indices p and q, the element corresponding to category index p in the first confidence vector is multiplied by the element corresponding to category index q in the second confidence vector to obtain multiple product terms. The multiple product terms are summed to obtain the conflict coefficient. If the conflict coefficient is less than a preset threshold, it is determined that there is no target association error.

[0062] Specifically, the conflict coefficient can be determined according to formula (5). : (5) in, Represents the first trust vector The p-th element in Represents the second trust vector The q-th element in.

[0063] Conflict coefficient This indicates the degree of conflict between the first and second images, when the conflict coefficient... When the value is less than a preset threshold, it indicates that there is no target association error between the first image and the second image, meaning that the target in the first image and the target in the second image are the same target. When the conflict coefficient... When the value is greater than or equal to the preset threshold, it indicates that there is a serious conflict between the first image and the second image, such as a target association error, that is, the target in the first image and the target in the second image are not the same target.

[0064] By using the conflict coefficient and preset threshold, it is possible to determine whether there are target association errors. Only when it is determined that there are no target association errors will subsequent target recognition be performed. This can automatically reject scenarios with association errors and serious data source conflicts, avoid the propagation of erroneous decisions, and improve the robustness of the system.

[0065] For example, based on the above embodiments, when obtaining the target category output by the target recognition model based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty, it can be done in the following way: Based on the first confidence vector, the second confidence vector, the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient, the fusion confidence vector is determined. Based on the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient, the fusion uncertainty is determined. Based on the fusion confidence vector and the fusion uncertainty, the fusion decision probability of each category is determined. Based on the fusion decision probability of each category, the target category is determined.

[0066] Specifically, subjective opinions from various data sources can be merged based on Dempster's fusion rules to obtain a merged subjective opinion. ,in, , Represents the fused trust vector. This represents the uncertainty of fusion, and the subjective opinion after fusion. It can characterize collaborative classification opinions and uncertainties based on data from different data sources.

[0067] and It can be determined according to formulas (6) and (7): (6) (7) in, express The k-th element in Represents the first trust vector The k-th element in Represents the second trust vector The k-th element in.

[0068] Based on the determined fusion trust vector and fusion uncertainty This allows us to determine the fusion decision probability for each category, and then determine the final target category based on the fusion decision probability for each category. For example, the category corresponding to the highest fusion decision probability can be determined as the target category.

[0069] In one possible implementation, when determining the fusion decision probability of each category based on the fusion confidence vector and the fusion uncertainty, the product between each element in the fusion uncertainty and the corresponding element in the preset base rate can be determined to obtain the product vector. The sum of each element in the fusion confidence vector and each element in the product vector can be determined to obtain the decision probability vector. Each element in the decision probability vector is used to characterize the fusion decision probability of the corresponding category.

[0070] Specifically, in order to fully utilize uncertainty in decision-making and improve decision stability, a base rate can be defined. The base rate can be a uniform distribution or a learnable parameter. The decision probability vector is obtained by distributing the fusion uncertainty according to the base rate. The decision probability vector can be determined according to the following formula (8): (8) in, The decision probability vector The k-th element in the expression represents the fusion decision probability of category k.

[0071] In this embodiment, the fusion decision probability of each category is determined by fusing the confidence vector and the fusion uncertainty. Based on these fusion decision probabilities, the target category is determined. By considering the uncertainty and confidence levels of multiple data sources, the accuracy of the target category is improved. Furthermore, defining a base rate provides a priori basis for the reasonable allocation of uncertainty. Including the fusion uncertainty in the decision-making process enhances the stability of the final determined target category.

[0072] Furthermore, based on the above embodiments, when determining the target category by fusing decision probabilities, in order to intuitively quantify the contribution of each data source, the marginal contribution of each information source to the final decision can be derived based on the Shapley contribution value, thereby achieving reliable traceability of the category identification results.

[0073] For example, the first marginal benefit of space-based infrastructure and the second marginal benefit of space-based infrastructure can be determined based on the elements corresponding to the target category in the first confidence vector, the elements corresponding to the target category in the second confidence vector, the elements corresponding to the target category in the base rate, the first single-source uncertainty, and the second single-source uncertainty. Based on the first marginal benefit and the second marginal benefit, the contribution of space-based infrastructure to the target category and the contribution of space-based infrastructure to the target category can be determined respectively.

[0074] Specifically, in order to determine the contribution of each data source to the final identification result and thus improve the credibility of the identification process, in this embodiment, the marginal benefit of a data source to the identification result can be defined as the data source contribution based on Shapley contribution. The derivation process is as follows: First, define the data source using formula (9). ,category The decision score is used as a utility function: (9) in, as well as To subset of data source The opinions obtained by combining formulas (6) and (7)

[0075] For a data source m∈{S,A}, its influence on the identification result (e.g., the identification result is category k) is defined as the Shapley value, as shown in formula (10): (10) In formula (10), the first term corresponds to the marginal benefit of introducing data source m alone, and the second term represents the marginal benefit of introducing data source m under the condition that data source m is already present.

[0076] Based on formulas (9) and (10), the closed-form solution can be determined using formulas (11) and (12): (11) (12) in, This represents the marginal benefit of space-based [technology / equipment]. This represents the marginal return of the short position. Indicates base rate The k-th element in.

[0077] In practical applications, the target category can be determined using the following formula (13). : (13) Therefore, in order to save system resources, we can trace only the data sources to determine the target category. The contribution at that time. In practical applications, it can be adopted... By replacing k in formulas (11) and (12), the first marginal benefit of the space-based system and the second marginal benefit of the air-based system can be determined.

[0078] Furthermore, the contributions of space-based and air-based data sources to the target category can be determined based on the first and second marginal benefits. In one possible implementation, the marginal benefits of each data source can be normalized to determine the contribution of each data source. Specifically, the contribution of space-based data sources to the target category is determined by summing the first and second marginal benefits and then using the ratio of the first marginal benefit to the sum. The contribution of air-based data sources to the target category is determined by using the ratio of the second marginal benefit to the sum.

[0079] In this embodiment, by determining the first marginal benefit of space-based and the second marginal benefit of air-based, and based on the first and second marginal benefits, the contribution of space-based and air-based to the target category are determined respectively. This allows for the traceability of the contribution of the final identification result, clarifies the decision weight of each data source, and enhances the interpretability and decision credibility of the collaborative identification result.

[0080] For example, based on the above embodiments, when it is determined that there is a target association error based on the first confidence vector and the second confidence vector, a first decision probability vector is determined based on the first confidence vector and the first single-source uncertainty, and a second decision probability vector is determined based on the second confidence vector and the second single-source uncertainty. The first decision probability vector and the second decision probability vector are then output through the target recognition model.

[0081] Specifically, based on the conflict coefficient determined according to formula (5) When the value is greater than or equal to a preset threshold, it indicates a target association error, meaning the first image and the second image describe different targets. In this case, the first decision probability vector can be determined based on the following formulas (14) and (15). Second Decision Probability vector: (14) (15) in, The first decision probability vector The k-th element in the matrix represents the probability that the target category determined from the first image is k. The second decision probability vector The k-th element in the array represents the probability that the target category determined from the second image is k.

[0082] After determining the first and second decision probability vectors under a single data source, since a target association error has occurred, no fusion recognition will be performed. The target recognition model can directly output the first and second decision probability vectors, which can effectively avoid decision misguidance caused by forced fusion due to previous association errors. This avoids outputting fusion results based on erroneous premises and with questionable credibility, thus improving the accuracy of target category recognition under abnormal conditions.

[0083] For example, based on the foregoing embodiments, the aforementioned target recognition model is trained in the following manner: A training dataset is constructed, comprising multiple sets of training samples. Each set includes a first sample image from a space-based system, positive sample images correctly associated with the target in the first sample image from an air-based system, and negative sample images incorrectly associated with the target in the first sample image. The positive and negative sample pairs from each set of training samples are input into an initial target recognition model to obtain the single-source decision probability, fusion decision probability, and single-source decision probability of the positive sample pair for each category. A positive sample pair includes the first sample image and a positive sample image, and a negative sample pair includes both the first sample image and a negative sample image. Based on the single-source decision probability and fusion decision probability of the positive sample pair, a first classification loss is determined, and based on the single-source decision probability of the negative sample pair, a second classification loss is determined. Based on the first and second classification losses, the model parameters of the initial target recognition model are updated to obtain the target recognition model.

[0084] Specifically, a training dataset with target association errors can be constructed. This dataset includes multiple training samples, for example, a training batch may contain G training samples, where each training sample is a triplet of target slices. ,in, This represents the first sample image from space-based [technology / system]. This represents a positive sample image from which the base case and the target in the first sample image are correctly associated. This represents a negative sample image from which the empty base is incorrectly associated with the target in the first sample image.

[0085] in addition, The corresponding one-hot vector class labels are respectively ,in, , .

[0086] After constructing the training dataset, the positive sample pairs are sequentially... and negative sample pairs In the initial target recognition model, the single-source decision probability of positive samples for each category can be determined according to the method in formula (14). According to the method in formula (8), the fusion decision probability of positive sample pairs can be determined. And in accordance with the method in formula (15), the single-source decision probability of negative samples for each category can be determined. .

[0087] Furthermore, a first classification loss can be determined based on the single-source decision probability of positive samples and the fusion decision probability of positive samples, and a second classification loss can be determined based on the single-source decision probability of negative samples. In one possible implementation, the first classification loss can be determined based on the single-source decision probability of positive samples for each category, the category label of the first sample image, the category label of the positive sample image, and a preset fusion decision loss weight; the second classification loss can be determined based on the single-source decision probability of negative samples, the category label of the first sample image, and the category label of the negative sample image.

[0088] Specifically, the first category loss can be determined according to formulas (16) and (17). Second category loss : in, This represents the category label when the target in the first sample image belongs to category k. This represents the category label when the target in a positive sample image belongs to category k. Indicates the weight of the loss in the fusion decision. This represents the category label when the target in the negative sample image belongs to category k.

[0089] Formulas (16) and (17) can guide the initial target recognition model to update its model parameters, learn good classification task representations, and improve the robustness of the trained target recognition model.

[0090] For example, when updating the model parameters of the initial target recognition model based on the first classification loss and the second classification loss to obtain the target recognition model, the positive sample conflict coefficient can be determined based on the positive sample pairs, and the negative sample conflict coefficient can be determined based on the negative sample pairs. The conflict loss can be determined based on the preset positive sample loss weight coefficient, the preset negative sample loss weight coefficient, the positive sample conflict coefficient, and the negative sample conflict coefficient. The total loss can be determined based on the first classification loss, the second classification loss, and the conflict loss. Thus, the model parameters of the initial target recognition model can be updated based on the total loss to obtain the target recognition model.

[0091] Specifically, in order to supervise the neural network to reasonably estimate the conflict coefficient, a conflict loss can be designed. The positive sample conflict coefficient can be determined based on positive sample pairs, as shown in formula (5). Based on the negative sample pairs, the negative sample conflict coefficient is determined according to the method shown in formula (5). .

[0092] Furthermore, the conflict loss can be determined according to formula (18). : (18) in, This represents the weighting coefficient for the positive sample loss. This represents the weighting coefficient for negative sample loss.

[0093] The total loss can be determined according to formula (19). : (19) After determining the total loss Subsequently, based on the total loss The model parameters of the initial target recognition model are updated, and the above process is repeated until the model converges or the number of iterations reaches the preset number. The final model is then determined as the target recognition model.

[0094] In this embodiment, by designing a conflict loss and a multi-task loss function that includes a first classification loss and a second classification loss, end-to-end training of the model can be achieved based on training data with association errors.

[0095] Figure 2 This is the second flowchart illustrating the space-based and air-based target collaborative identification method provided in this embodiment of the invention. Figure 2 As shown, space-based targets This represents the i-th first image from space-based, and the space-based target. This represents the i-th second image from the empty base. First image. Through the feature extraction network in the target recognition model Extract the first deep features and pass them through the evidence mapping network. The first depth feature is mapped to the first evidence strength vector. The second image... Through feature extraction network Extract the second deep features and pass them through the evidence mapping network. The second deep feature is mapped to a second evidence strength vector. The target recognition model can be a neural network.

[0096] Furthermore, a single-source opinion can be determined based on a first evidence strength vector and a second evidence strength vector, such as determining a first subjective opinion using the first evidence strength vector. And determine the second subjective opinion through the second evidence strength vector. .

[0097] The conflict coefficient can be determined based on the first trust vector in the first subjective opinion and the second trust vector in the second subjective opinion. If the conflict coefficient If the value is greater than or equal to a preset threshold, it indicates a conflict, meaning there is a target association error. If the conflict coefficient... If the value is less than the preset threshold, it means there is no conflict, that is, there is no target association error.

[0098] In the event of conflict, single-source identification can be performed using a target recognition model to determine the decision probability for each source, including the space-based decision probability. And empty base decision probability And output the space-based decision probability through the target recognition model. And empty base decision probability .

[0099] In the absence of conflict, the first subjective opinion will be considered. Second subjective opinion By integrating the results, we can obtain the subjective opinions derived from the integration. And based on subjective opinions Determine the fused decision probability vector .

[0100] In addition, the contribution of each data source to the final identification result can be traced to determine the marginal benefits of space-based systems. Marginal returns of open base This allows us to determine the contribution of space-based systems to the target category and the contribution of air-based systems to the target category.

[0101] During the training phase of the target recognition model, if a target association error is identified through sample images, a space-based decision probability can be used. And empty base decision probability Determine the space-based classification loss and the air-based classification loss respectively, and train the model based on these two losses respectively.

[0102] If it is determined from the sample images that there are no errors in target association, the fusion classification loss can be determined, including the first classification loss and the second classification loss described in the previous embodiments. A conflict loss can also be determined based on the conflict coefficient. Therefore, based on the first classification loss, the second classification loss, and the conflict loss, a total loss is determined. The target recognition model is then trained based on this total loss. The fusion classification loss can be used to supervise the accuracy of the fusion result, while the conflict loss can be used to supervise the model's ability to distinguish between correctly associated and incorrect samples.

[0103] In this embodiment, based on deep evidence learning theory, a deep network is used to explicitly transform the feature outputs of heterogeneous data sources from space-based and air-based systems into belief assignments in evidence theory, and to measure the degree of conflict and uncertainty between data sources online. Furthermore, an adaptive evidence fusion rule is constructed, assigning higher decision weights to high-quality, highly consistent data sources, and making an intelligent "refusal to identify" judgment when data source conflicts or insufficient evidence occur. Moreover, based on the contribution of each data source to the decision result derived from Shapley contribution derivation, the decision weights of space-based / air-based data sources are intuitively given, enabling attribution and tracing of the data source for the final decision confidence level, thereby improving the interpretability of the algorithm's decision and adapting to the high reliability and autonomy requirements of air-based edge processing.

[0104] Furthermore, in this embodiment, target category fusion identification can be performed by combining space-based and air-based data, which can improve decision-making performance. In multi-source fusion scenarios, the robustness of the system can be improved in uncertain scenarios such as target association errors and data source quality degradation. Moreover, it does not rely on spatial registration or temporal synchronization of space-based and air-based data, and can be adapted to multimodal remote sensing data such as optical, SAR, and multispectral data, making it highly versatile and applicable to a wide range of scenarios.

[0105] The target collaborative identification device based on space-based and air-based systems provided by the present invention is described below. The target collaborative identification device based on space-based and air-based systems described below can be referred to in correspondence with the target collaborative identification method based on space-based and air-based systems described above.

[0106] Figure 3 This is a schematic diagram of the structure of the space-based and air-based target collaborative identification device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the space-based and air-based target collaborative identification device 300 includes: The determination module 11 is used to input the first image acquired by space-based acquisition and the second image acquired by air-based acquisition into the target recognition model, and determine the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model. Each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category, and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category. The determining module 11 is further configured to determine a first subjective opinion based on the first evidence strength vector and a second subjective opinion based on the second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty. The second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector is used to characterize the degree of certainty that the target in the first image belongs to the corresponding category. Each element in the second confidence vector is used to characterize the degree of certainty that the target in the second image belongs to the corresponding category. Output module 12 is used to obtain the target category of the collaborative recognition output by the target recognition model based on the first trust vector, the second trust vector, the first single-source uncertainty, and the second single-source uncertainty, when it is determined that there is no target association error based on the first trust vector and the second trust vector.

[0107] In one example embodiment, the determining module 11 is specifically used for: The first depth feature is extracted from the first image and the second depth feature is extracted from the second image through the feature extraction network in the target recognition model. The first deep feature is mapped to the first evidence strength vector through the evidence mapping network in the target recognition model, and the second deep feature is mapped to the second evidence strength vector.

[0108] In one example embodiment, the determining module 11 is specifically used for: Based on the first evidence strength vector, a first total evidence is determined, which is used to characterize the degree of confidence of the space-based system in the classification results; The first trust vector is determined based on the first evidence strength vector and the first total amount of evidence; The first single-source uncertainty is determined based on the total amount of the first piece of evidence and the preset total number of categories.

[0109] In one example embodiment, the determining module 11 is specifically used for: For all distinct category indices p and q, the element corresponding to category index p in the first confidence vector is multiplied by the element corresponding to category index q in the second confidence vector to obtain multiple product terms; Summing the multiple product terms yields the conflict coefficient; If the conflict coefficient is less than a preset threshold, it is determined that there is no target association error.

[0110] In one example embodiment, the output module 12 is specifically used for: Based on the first trust vector, the second trust vector, the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient, a fused trust vector is determined; The fused uncertainty is determined based on the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient. The product vector is obtained by determining the product between each element in the fusion uncertainty and the corresponding element in the preset basis. The sum of each element in the fusion trust vector and each element in the product vector is determined to obtain the decision probability vector, and each element in the decision probability vector is used to characterize the fusion decision probability of the corresponding category. The target category is determined based on the fusion decision probability of each category.

[0111] In one example embodiment, the determining module 11 is further configured to: Based on the elements corresponding to the target category in the first confidence vector, the elements corresponding to the target category in the second confidence vector, the elements corresponding to the target category in the base rate, the first single-source uncertainty, and the second single-source uncertainty, the first marginal benefit of the space-based system and the second marginal benefit of the air-based system are determined. Based on the first marginal benefit and the second marginal benefit, the contribution of the space-based system to the target category and the contribution of the air-based system to the target category are determined respectively.

[0112] In one example embodiment, the determining module 11 is further configured to, when determining that there is a target association error based on the first confidence vector and the second confidence vector, determine a first decision probability vector based on the first confidence vector and the first single-source uncertainty, and determine a second decision probability vector based on the second confidence vector and the second single-source uncertainty; Output module 12 is also used to output the first decision probability vector and the second decision probability vector through the target recognition model.

[0113] In one example embodiment, the target recognition model is trained in the following manner: A training dataset is constructed, which includes multiple sets of training samples. Each set of training samples includes a first sample image from the space-based system, positive sample images from the air-based system that are correctly associated with the target in the first sample image, and negative sample images that are incorrectly associated with the target in the first sample image. The positive sample pairs and negative sample pairs in each group of training samples are respectively input into the initial target recognition model to obtain the single-source decision probability of positive sample pairs, the fusion decision probability of positive sample pairs and the single-source decision probability of negative sample pairs output by the initial target recognition model for each category. The positive sample pair includes the first sample image and the positive sample image, and the negative sample pair includes the first sample image and the negative sample image. Based on the single-source decision probability of the positive sample pair and the fusion decision probability of the positive sample pair, a first classification loss is determined, and based on the single-source decision probability of the negative sample pair, a second classification loss is determined. Based on the first classification loss and the second classification loss, the model parameters of the initial target recognition model are updated to obtain the target recognition model.

[0114] In one example embodiment, the device further includes an update module, wherein: The determining module 11 is further configured to determine the positive sample conflict coefficient based on the positive sample pair, and to determine the negative sample conflict coefficient based on the negative sample pair; The determining module 11 is also used to determine the conflict loss based on the preset positive sample loss weight coefficient, the preset negative sample loss weight coefficient, the positive sample conflict coefficient and the negative sample conflict coefficient; The determination module 11 is further configured to determine the total loss based on the first classification loss, the second classification loss, and the conflict loss. An update module is used to update the model parameters of the initial target recognition model based on the total loss, so as to obtain the target recognition model.

[0115] The apparatus of this embodiment can be used in any of the methods in the side embodiment of the target collaborative identification method based on space-based and air-based systems. Its specific implementation process and technical effects are similar to those in the side embodiment of the target collaborative identification method based on space-based and air-based systems. For details, please refer to the detailed description in the side embodiment of the target collaborative identification method based on space-based and air-based systems, which will not be repeated here.

[0116] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a target collaborative identification method based on space-based and air-based systems. This method includes: inputting a first image acquired by space-based systems and a second image acquired by air-based systems into a target identification model; determining a first evidence strength vector corresponding to the first image and a second evidence strength vector corresponding to the second image through the target identification model; each element in the first evidence strength vector characterizes the evidence strength that the target in the first image belongs to a corresponding category; each element in the second evidence strength vector characterizes the evidence strength that the target in the second image belongs to a corresponding category; determining a first subjective opinion based on the first evidence strength vector; and determining a second subjective opinion based on the second evidence strength vector. The second subjective opinion comprises a first confidence vector and a first single-source uncertainty, and a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector represents the degree of certainty that the target in the first image belongs to the corresponding category, and each element in the second confidence vector represents the degree of certainty that the target in the second image belongs to the corresponding category. If, based on the first confidence vector and the second confidence vector, it is determined that there is no target association error, the target category output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty.

[0117] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target collaborative recognition method based on space-based and air-based methods provided by the above methods. This method includes: inputting a first image acquired by space-based acquisition and a second image acquired by air-based acquisition into a target recognition model; determining a first evidence strength vector corresponding to the first image and a second evidence strength vector corresponding to the second image through the target recognition model; each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category; and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category. A first subjective opinion is determined based on a first evidence strength vector, and a second subjective opinion is determined based on a second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty, and the second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector represents the degree of certainty that the target in the first image belongs to the corresponding category, and each element in the second confidence vector represents the degree of certainty that the target in the second image belongs to the corresponding category. If it is determined that there is no target association error based on the first confidence vector and the second confidence vector, the target category of the collaborative recognition output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty.

[0119] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the target collaborative recognition method based on space-based and air-based methods provided above. This method includes: inputting a first image acquired by space-based acquisition and a second image acquired by air-based acquisition into a target recognition model; determining a first evidence strength vector corresponding to the first image and a second evidence strength vector corresponding to the second image through the target recognition model; each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to a corresponding category; and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to a corresponding category; and determining a first subjective... The first subjective opinion is determined based on the second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty. The second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector represents the degree of certainty that the target in the first image belongs to the corresponding category. Each element in the second confidence vector represents the degree of certainty that the target in the second image belongs to the corresponding category. If it is determined that there is no target association error based on the first confidence vector and the second confidence vector, the target category of the collaborative recognition output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty.

[0120] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target collaborative identification method based on space-based and air-based systems, characterized in that, include: The first image acquired by space-based acquisition and the second image acquired by air-based acquisition are input into the target recognition model. The target recognition model determines the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image. Each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category, and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category. A first subjective opinion is determined based on the first evidence strength vector, and a second subjective opinion is determined based on the second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty. The second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector is used to characterize the degree of certainty that the target in the first image belongs to the corresponding category. Each element in the second confidence vector is used to characterize the degree of certainty that the target in the second image belongs to the corresponding category. If it is determined that there is no target association error based on the first confidence vector and the second confidence vector, the target category of the collaborative recognition output by the target recognition model is obtained based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty.

2. The target collaborative identification method based on space-based and air-based systems according to claim 1, characterized in that, The step of determining the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model includes: The first depth feature is extracted from the first image and the second depth feature is extracted from the second image through the feature extraction network in the target recognition model. The first deep feature is mapped to the first evidence strength vector through the evidence mapping network in the target recognition model, and the second deep feature is mapped to the second evidence strength vector.

3. The target collaborative identification method based on space-based and air-based systems according to claim 1, characterized in that, The determination of the first subjective opinion based on the first evidence strength vector includes: Based on the first evidence strength vector, a first total evidence is determined, which is used to characterize the degree of confidence of the space-based system in the classification results; The first trust vector is determined based on the first evidence strength vector and the first total amount of evidence; The first single-source uncertainty is determined based on the total amount of the first piece of evidence and the preset total number of categories.

4. The target collaborative identification method based on space-based and air-based systems according to claim 1, characterized in that, The step of determining that there is no target association error based on the first trust vector and the second trust vector includes: For all distinct category indices p and q, the element corresponding to category index p in the first confidence vector is multiplied by the element corresponding to category index q in the second confidence vector to obtain multiple product terms; Summing the multiple product terms yields the conflict coefficient; If the conflict coefficient is less than a preset threshold, it is determined that there is no target association error.

5. The target collaborative identification method based on space-based and air-based systems according to claim 4, characterized in that, The step of obtaining the target category output by the target recognition model based on the first confidence vector, the second confidence vector, the first single-source uncertainty, and the second single-source uncertainty includes: Based on the first trust vector, the second trust vector, the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient, a fused trust vector is determined; The fused uncertainty is determined based on the first single-source uncertainty, the second single-source uncertainty, and the conflict coefficient. The product vector is obtained by determining the product between each element in the fusion uncertainty and the corresponding element in the preset basis. The sum of each element in the fusion trust vector and each element in the product vector is determined to obtain the decision probability vector, and each element in the decision probability vector is used to characterize the fusion decision probability of the corresponding category. The target category is determined based on the fusion decision probability of each category.

6. The target collaborative identification method based on space-based and air-based systems according to claim 5, characterized in that, The method further includes: Based on the elements corresponding to the target category in the first confidence vector, the elements corresponding to the target category in the second confidence vector, the elements corresponding to the target category in the base rate, the first single-source uncertainty, and the second single-source uncertainty, the first marginal benefit of the space-based system and the second marginal benefit of the space-based system are determined. Based on the first marginal benefit and the second marginal benefit, the contribution of the space-based system to the target category and the contribution of the air-based system to the target category are determined respectively.

7. The target collaborative identification method based on space-based and air-based systems according to claim 1, characterized in that, The method further includes: If a target association error is determined based on the first confidence vector and the second confidence vector, a first decision probability vector is determined based on the first confidence vector and the first single-source uncertainty, and a second decision probability vector is determined based on the second confidence vector and the second single-source uncertainty. The target recognition model outputs the first decision probability vector and the second decision probability vector.

8. The target collaborative identification method based on space-based and air-based systems according to any one of claims 1-7, characterized in that, The target recognition model was trained in the following manner: A training dataset is constructed, which includes multiple sets of training samples. Each set of training samples includes a first sample image from the space-based system, positive sample images from the air-based system that are correctly associated with the target in the first sample image, and negative sample images that are incorrectly associated with the target in the first sample image. The positive sample pairs and negative sample pairs in each group of training samples are respectively input into the initial target recognition model to obtain the single-source decision probability of positive sample pairs, the fusion decision probability of positive sample pairs and the single-source decision probability of negative sample pairs output by the initial target recognition model for each category. The positive sample pair includes the first sample image and the positive sample image, and the negative sample pair includes the first sample image and the negative sample image. Based on the single-source decision probability of the positive sample pair and the fusion decision probability of the positive sample pair, a first classification loss is determined, and based on the single-source decision probability of the negative sample pair, a second classification loss is determined. Based on the first classification loss and the second classification loss, the model parameters of the initial target recognition model are updated to obtain the target recognition model.

9. The target collaborative identification method based on space-based and air-based systems according to claim 8, characterized in that, The step of updating the model parameters of the initial target recognition model based on the first classification loss and the second classification loss to obtain the target recognition model includes: Based on the positive sample pairs, the positive sample conflict coefficient is determined, and based on the negative sample pairs, the negative sample conflict coefficient is determined. The conflict loss is determined based on the preset positive sample loss weight coefficient, the preset negative sample loss weight coefficient, the positive sample conflict coefficient, and the negative sample conflict coefficient. The total loss is determined based on the first classification loss, the second classification loss, and the conflict loss; The model parameters of the initial target recognition model are updated based on the total loss to obtain the target recognition model.

10. A target collaborative identification device based on space-based and air-based systems, characterized in that, include: The determination module is used to input the first image acquired by space-based acquisition and the second image acquired by air-based acquisition into the target recognition model, and determine the first evidence strength vector corresponding to the first image and the second evidence strength vector corresponding to the second image through the target recognition model. Each element in the first evidence strength vector is used to characterize the evidence strength that the target in the first image belongs to the corresponding category, and each element in the second evidence strength vector is used to characterize the evidence strength that the target in the second image belongs to the corresponding category. The determining module is further configured to determine a first subjective opinion based on the first evidence strength vector and a second subjective opinion based on the second evidence strength vector. The first subjective opinion includes a first confidence vector and a first single-source uncertainty. The second subjective opinion includes a second confidence vector and a second single-source uncertainty. Each element in the first confidence vector is used to characterize the degree of certainty that the target in the first image belongs to the corresponding category. Each element in the second confidence vector is used to characterize the degree of certainty that the target in the second image belongs to the corresponding category. The output module is used to obtain the target category of the collaborative recognition output by the target recognition model based on the first trust vector, the second trust vector, the first single-source uncertainty, and the second single-source uncertainty, when it is determined that there is no target association error based on the first trust vector and the second trust vector.

Images