A fusion of track and image collaborative target recognition method and system

Abstract

The application relates to the technical field of target identification, and discloses a kind of fusion track and image collaborative target identification method and system, the method comprises: synchronously obtaining track data and image data, and track data and image data are time-space registered, and collaborative data is obtained;Based on collaborative data, track data is time-series correlation fitting, and target track is obtained;Based on collaborative data, image data is multi-scale feature fusion, and image fusion atlas is obtained;Target track and image fusion atlas are bidirectional intercommunication correlation, and track atlas pair is obtained;The behavior semantics of track atlas pair is interpreted, and behavior intention is obtained, and the situation representation between behavior intention is evaluated, and situation representation is obtained;Based on situation representation, track atlas pair is structured analysis, and target identification result is obtained;The application can improve the accuracy of collaborative target identification.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and in particular to a collaborative target recognition method and system that integrates dot patterns and images. Background Technology

[0002] In existing target recognition technologies, the processing of point data and image data is often separate, lacking an effective spatiotemporal coordination mechanism. The inherent correlation between the two types of data is not fully explored, resulting in fragmented data information. The temporal correlation of point data is difficult to effectively combine with the visual features of image data, making the depiction of the continuity of target trajectory insufficiently accurate. Image feature extraction is also limited to a single scale, failing to comprehensively reflect the multidimensional information of the target, thus hindering subsequent recognition work.

[0003] Meanwhile, existing technologies lack in-depth behavioral semantic interpretation and situational assessment processes after data processing, making it difficult to accurately extract the target's behavioral intentions and relationships from fused data. Insufficient structured analysis capabilities result in superficial target recognition results that fail to meet the demands for accuracy and comprehensiveness. Therefore, improving the accuracy and comprehensiveness of target recognition has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a collaborative target recognition method and system that integrates dot patterns and images to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a collaborative target recognition method that integrates dot patterns and images, comprising: S1. Simultaneously acquire point data and image data from the sensor, and perform spatiotemporal registration on the point data and the image data to obtain the collaborative data of the sensor; S2. Based on the collaborative data, perform time-series correlation fitting on the point data to obtain the target trajectory of the sensor; S3. Based on the collaborative data, perform multi-scale feature fusion on the image data to obtain the image fusion map of the sensor; S4. The target trajectory is bidirectionally interconnected with the image fusion map to obtain the trajectory map pair of the sensor; S5. Perform behavioral semantic interpretation on the track map pairs to obtain the behavioral intentions of the sensors, and evaluate the situational representations between the behavioral intentions to obtain the situational representations of the sensors. S6. Based on the situational characterization, the track map is structured and analyzed to obtain the target recognition result of the sensor.

[0006] In a preferred embodiment, the step of synchronously acquiring point data and image data from the sensor, and performing spatiotemporal registration on the point data and the image data to obtain the sensor's collaborative data includes: Simultaneously acquire point data and image data from the sensor; The dot data and the image data are time-aligned to obtain the intermediate dot data and intermediate image data of the sensor; Mapping the intermediate dot data to the coordinate system of the intermediate image data yields the spatial alignment data of the dot data. The spatial alignment data is fused and encapsulated with the intermediate image data to obtain the collaborative data of the sensor.

[0007] In a preferred embodiment, the step of performing time-series correlation fitting on the point data based on the collaborative data to obtain the target trajectory of the sensor includes: Based on the collaborative data, spatiotemporal relationship identification is performed on the point data to obtain the potential correlation of the point data; Based on the potential correlation, the point data is linked in a regular manner to obtain continuous segments of the point data; The continuous segments are smoothed and fitted to obtain the target trajectory of the point data.

[0008] In a preferred embodiment, the step of performing multi-scale feature fusion on the image data based on the collaborative data to obtain the image fusion map of the sensor includes: Based on the collaborative data, the image data is deconstructed hierarchically to obtain a multi-scale data subset of the image data; Cross-scale feature interaction is performed on the multi-scale data subset to obtain heterogeneous features of the image data; The heterogeneous features are topologically integrated to obtain the image fusion map of the sensor.

[0009] In a preferred embodiment, the step of bidirectionally linking the target track with the image fusion map to obtain the sensor track map pair includes: Mapping the track points in the target track to the image fusion map yields the initial associated point set of the sensor; Based on the initial set of associated points, neighborhood features are extracted from the image fusion map; Based on the neighborhood features, the target trajectory is semantically enhanced to obtain the enhanced trajectory of the target trajectory; Based on the enhanced trajectory, the correlation attenuation adjustment is performed on the image fusion map to obtain the focused map of the image fusion map; The enhanced track and the focused map are mapped and matched to obtain the track map pair of the sensor.

[0010] In a preferred embodiment, the step of adjusting the correlation attenuation of the image fusion map based on the enhanced track to obtain the focus map of the image fusion map includes: The spatial position, instantaneous velocity, and time information of the enhanced trajectory are obtained; Based on the spatial location and the time information, the correlation between the pixel location in the image fusion map and the track point in the enhanced track is analyzed to obtain the correlation strength between the pixel location and the track point. Based on the correlation strength, the feature values ​​in the image fusion map are adapted and optimized to obtain the processed fusion map of the image fusion map; The processed fused spectrum is then subjected to interval correction to obtain the focused spectrum of the sensor.

[0011] In a preferred embodiment, the step of performing behavioral semantic interpretation on the track map pairs to obtain the sensor's behavioral intent, and evaluating the situational representations between the behavioral intents to obtain the sensor's situational representation, includes: By performing pattern recognition on the enhanced trajectory and the focused map, a preliminary behavioral inference of the sensor is obtained; The preliminary behavior inference set is conflict-resolved to obtain the effective behavioral intent of the sensor; Topological association is performed on the effective behavioral intentions to obtain a relational network description of the sensors; Based on the relationship network description, a situation assessment is performed on the effective behavioral intentions to obtain the situational characterization of the sensor.

[0012] In a preferred embodiment, the step of evaluating the situational tendency of the effective behavioral intention based on the relationship network description to obtain the situational characterization of the sensor includes: Interaction determination is performed on the relationship network description to obtain the logical interaction type of the relationship network description; Based on the logical interaction type, determine the situational tendency coefficient of the behavioral intention; Based on the situation tendency coefficient, the situation characterization is calculated, wherein the calculation formula for the situation characterization is as follows: ; in, This is the situational characterization. For the first The and the first The effective situation coefficient between individual behavioral intentions For the first The and the first Directional factors between behavioral intentions.

[0013] In a preferred embodiment, the step of performing structured analysis on the track map pairs based on the situational characterization to obtain the target identification results of the sensor includes: Based on the situational characterization, target extraction is performed on the track map pair to obtain the target entity of the track map pair; Intent matching is performed between the target entity and the valid behavioral intent to obtain the intent representation of the target entity; The interaction relationship is parsed to obtain the relationship network of the intent representation; Based on the relationship network, the intent representation is identified and determined to obtain the target recognition result of the sensor.

[0014] To address the aforementioned problems, the present invention also provides a collaborative target recognition system that integrates dot patterns and images, the system comprising: The spatiotemporal registration and synchronization module is used to synchronously acquire point data and image data from the sensor, and to perform spatiotemporal registration on the point data and the image data to obtain the collaborative data of the sensor. The time-series trajectory generation module is used to perform time-series correlation fitting on the point data based on the collaborative data to obtain the target trajectory of the sensor; The image fusion map generation module is used to perform multi-scale feature fusion on the image data based on the collaborative data to obtain the image fusion map of the sensor; The track map association module is used to perform bidirectional communication and association between the target track and the image fusion map to obtain the track map pair of the sensor; The behavioral semantic interpretation module is used to perform behavioral semantic interpretation on the track map pairs to obtain the behavioral intentions of the sensors, and to evaluate the situational representations between the behavioral intentions to obtain the situational representations of the sensors. The target recognition module is used to perform structured analysis on the track map pair based on the situational characterization to obtain the target recognition result of the sensor.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves efficient collaboration between point data and image data through synchronous acquisition and spatiotemporal registration. Temporal correlation fitting technology improves the continuity and accuracy of target tracks, while multi-scale feature fusion enriches the representational dimensions of image information. Bidirectional interconnection further strengthens the intrinsic correlation between target tracks and image fusion maps, effectively improving the completeness and effectiveness of target-related data and laying a solid foundation for subsequent identification work.

[0016] 2. By relying on behavioral semantic interpretation and situational characterization assessment, this invention can deeply explore the behavioral intentions and associated situations of targets. The structured parsing process, combined with situational characterization and target entity intention matching, significantly improves the accuracy and comprehensiveness of target identification. At the same time, it clearly captures the behavioral logic and interaction relationships of targets, making the target identification results more valuable and effectively enhancing the practical performance of target identification technology. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a collaborative target recognition method that integrates dot patterns and images, provided in an embodiment of the present invention. Figure 2 This is a functional block diagram of a collaborative target recognition system that integrates dot patterns and images, provided in an embodiment of the present invention. The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a collaborative target recognition method that integrates point traces and images. The executing entity of this collaborative target recognition method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the collaborative target recognition method integrating point traces and images can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0020] Reference Figure 1The diagram shown is a flowchart illustrating a collaborative target recognition method that fuses point traces and images according to an embodiment of the present invention. In this embodiment, the collaborative target recognition method that fuses point traces and images includes: S1. Simultaneously acquire point data and image data from the sensor, and perform spatiotemporal registration on the point data and the image data to obtain the collaborative data of the sensor; In this embodiment of the invention, the step of synchronously acquiring point data and image data from the sensor, and performing spatiotemporal registration of the point data and the image data to obtain the collaborative data of the sensor includes: Simultaneously acquire point data and image data from the sensor; The point data stream and the image data stream are time-aligned to obtain the intermediate point data and intermediate image data of the sensor; Mapping the intermediate dot data to the coordinate system of the intermediate image data yields the spatial alignment data of the dot data. The spatial alignment data is fused and encapsulated with the intermediate image data to obtain the collaborative data of the sensor.

[0021] By activating the sensor's dual data acquisition channels, the sensor can simultaneously acquire target point data and image data. The point data contains key information such as the target's spatial location and motion state, while the image data consists of image frames of the target and its surrounding environment continuously captured by the sensor. During the acquisition process, each point data and each image frame is marked with the same timestamp to ensure that the two types of data maintain synchronization in the time dimension from the source of acquisition.

[0022] The timestamps of all data in the point data stream and the image data stream are extracted. The two data streams are arranged according to the order of their timestamps. Using the timestamp sequence of the image data stream as a reference, the timestamps in the point data stream are compared one by one. Point data that completely corresponds to the reference timestamp is selected. For cases where there is no completely matching timestamp, valid data in the point data stream whose timestamps are between the timestamps of two adjacent image frames are retained. Invalid data in the point data stream that is outside the time range of the image data stream and redundant image frames in the image data stream that have no corresponding point data are removed. Finally, intermediate point data and intermediate image data that are one-to-one in time are formed.

[0023] Define the coordinate system parameters of the intermediate image data. Set the origin of the coordinate system to the upper left corner of the image frame, with the positive X-axis pointing horizontally to the right and the positive Y-axis pointing vertically downwards. The coordinate unit is pixels. Based on the actual spatial coordinate information of the intermediate point data, establish the correspondence between spatial coordinates and image pixel coordinates. Accurately convert each spatial position information in the intermediate point data into pixel coordinates in the coordinate system of the intermediate image data. This allows the intermediate point data to be precisely mapped to the specific position in the intermediate image data. After the conversion, the spatial alignment data of the point data is obtained.

[0024] Based on the chronological order of timestamps, spatially aligned data and intermediate image data are grouped one-to-one. Each timestamp corresponds to a combination of spatially aligned data and intermediate image data at that time point. The storage format and encoding method of the spatially aligned data and intermediate image data in each group are standardized to ensure data format uniformity. Then, each standardized group of data is associated and bound to make each group of data an indivisible whole. Finally, all the associated and bound grouped data are integrated into a structured data set in chronological order to obtain the sensor's collaborative data.

[0025] The beneficial effects are that synchronous acquisition ensures the initial temporal consistency of point data and image data, temporal alignment and spatial alignment eliminate the deviation of the two types of data in time and space, and the collaborative data formed by fusion and encapsulation contains both motion information of point data and visual information of image data. This provides a unified, accurate and complete data foundation for subsequent steps such as target trajectory generation and image fusion map construction based on collaborative data, effectively improving the stability and reliability of the entire target recognition process.

[0026] S2. Based on the collaborative data, perform time-series correlation fitting on the point data to obtain the target trajectory of the sensor; In this embodiment of the invention, the step of performing time-series correlation fitting on the point data based on the collaborative data to obtain the target trajectory of the sensor includes: Based on the collaborative data, spatiotemporal relationship identification is performed on the point data to obtain the potential correlation of the point data; Based on the potential correlation, the point data is linked in a regular manner to obtain continuous segments of the point data; The continuous segments are smoothed and fitted to obtain the target trajectory of the point data.

[0027] Based on the spatiotemporal registration of point data and image data in the collaborative data, the timestamp and spatial location information corresponding to each point data are extracted. All point data are arranged in chronological order according to the timestamps. At the same time, combined with the visual contour and location range of the target in the image data, the spatial location differences of point data with adjacent timestamps are analyzed one by one to determine whether these point data belong to the motion records of the same target at different times. For point data that are spatially located in the same target area in the image and have consecutive timestamps, the inherent temporal continuity and spatial connection relationship between them is determined, and finally the potential correlation relationship of point data is formed.

[0028] Based on the identified potential relationships, the related point data are sequentially connected in strict order of timestamps from earliest to latest. During the connection process, it is ensured that the timestamps of each point data are continuous and that the spatial position conforms to the movement logic of the same target. Isolated point data without any relationship are not included to avoid time discontinuities or sudden changes in spatial position. This results in the connected point data forming a temporally ordered and spatially coherent sequence. These ordered and coherent sequences are the continuous segments of the point data.

[0029] Observe the spatial distribution of each point data in a continuous segment of point data, identify abnormal point data whose spatial position deviates from the overall movement trend, refer to the spatial position and time interval of adjacent point data before and after the abnormal point data, and adjust the spatial position of the abnormal point data to a reasonable position that conforms to the overall movement trend according to the stability law of the target movement, so that the point data of the entire continuous segment forms a smooth trajectory without discontinuities, abrupt changes, and natural transitions. This smooth trajectory is the target trajectory of the point data.

[0030] The beneficial effects are that the spatiotemporal relationship identification accurately filters the associated point data of the same target, the regularized linking ensures the continuity of the point sequence, the smooth fitting eliminates the interference of abnormal points, and the final target trajectory can truly reflect the actual movement of the target. This provides accurate motion trajectory support for the bidirectional communication and association between the target trajectory and the image fusion map, and improves the effectiveness of the subsequent association process and the accuracy of the target recognition results.

[0031] S3. Based on the collaborative data, perform multi-scale feature fusion on the image data to obtain the image fusion map of the sensor; In this embodiment of the invention, the step of performing multi-scale feature fusion on the image data based on the collaborative data to obtain the image fusion map of the sensor includes: Based on the collaborative data, the image data is deconstructed hierarchically to obtain a multi-scale data subset of the image data; Cross-scale feature interaction is performed on the multi-scale data subset to obtain heterogeneous features of the image data; The heterogeneous features are topologically integrated to obtain the image fusion map of the sensor.

[0032] Based on the spatiotemporal correlation information of image data and point data in collaborative data, and taking the target spatial location corresponding to the point data as the core reference, the core region where the target is located in the image data is determined. Then, a hierarchical division is performed starting from this core region. The first layer is the overall image region containing the core region and a certain range of background around it, covering the global morphological information of the target. The second layer is to evenly divide the overall region of the first layer into several medium-sized sub-regions, each of which contains local target information and corresponding background information. The third layer is to evenly divide each medium-sized sub-region of the second layer into smaller local sub-regions, focusing on the information of the fine parts of the target. Each level extracts the corresponding image pixels, contours, textures, and other information to form an independent data set. These independent data sets at different levels together constitute a multi-scale data subset of the image data.

[0033] Features are extracted from each level of the multi-scale data subset. The first layer extracts macroscopic features such as the overall outline of the target, global brightness distribution, and overall color tendency. The second layer extracts mesoscopic features such as the local structure of the target, regional texture transitions, and local color differences. The third layer extracts microscopic features such as the fine texture details, pixel grayscale gradients, and local edge undulations. Then, on a level-by-level basis, each macroscopic feature of the first layer is compared one by one with the mesoscopic features of the corresponding spatial location in the second layer to filter out information that can complement and improve the target feature description. The integrated features are then compared and supplemented with the microscopic features of the corresponding spatial location in the third layer in the same way. Through this mutual reference and information fusion between features of different scales, heterogeneous features of image data that simultaneously contain macroscopic, mesoscopic, and microscopic features are formed.

[0034] By analyzing the spatial locations and feature attributes of all heterogeneous features, clarifying the target area and feature type described by each heterogeneous feature, determining the spatial adjacency and semantic association logic between different heterogeneous features, and connecting the scattered heterogeneous features in an orderly manner according to the order of spatial location from whole to part and semantic association from core to secondary, the heterogeneous features describing the same target area are closely related, and the heterogeneous features describing different areas but with logical connections are reasonably transitioned. Finally, all heterogeneous features are integrated into a whole with a regular structure, clear feature associations, and the ability to comprehensively represent target information. This whole is the image fusion map of the sensor.

[0035] The beneficial effects are that hierarchical decomposition accurately divides image data into different scale regions, fully explores various image features from macro to micro, cross-scale feature interaction realizes the complementary advantages of features at different scales, avoids the limitations of single-scale features, and topological integration allows scattered heterogeneous features to form a structured organic whole. The final image fusion map has rich, comprehensive and clearly related image features, providing high-quality feature support for the bidirectional communication and association between the target track and the image fusion map, effectively improving the association accuracy and the overall effect of target recognition.

[0036] S4. The target trajectory is bidirectionally interconnected with the image fusion map to obtain the trajectory map pair of the sensor; In this embodiment of the invention, the step of bidirectionally interconnecting the target trajectory with the image fusion map to obtain the sensor trajectory map pair includes: Mapping the track points in the target track to the image fusion map yields the initial associated point set of the sensor; Based on the initial set of associated points, neighborhood features are extracted from the image fusion map; Based on the neighborhood features, the target trajectory is semantically enhanced to obtain the enhanced trajectory of the target trajectory; Based on the enhanced trajectory, the correlation attenuation adjustment is performed on the image fusion map to obtain the focused map of the image fusion map; The enhanced track and the focused map are mapped and matched to obtain the track map pair of the sensor.

[0037] The step of adjusting the correlation attenuation of the image fusion map based on the enhanced trajectory to obtain the focus map of the image fusion map includes: The spatial position, instantaneous velocity, and time information of the enhanced trajectory are obtained; Based on the spatial location and the time information, the correlation between the pixel location in the image fusion map and the track point in the enhanced track is analyzed to obtain the correlation strength between the pixel location and the track point. Based on the correlation strength, the feature values ​​in the image fusion map are adapted and optimized to obtain the processed fusion map of the image fusion map; The processed fused spectrum is then subjected to interval correction to obtain the focused spectrum of the sensor.

[0038] The spatial coordinate information of each track point in the target track is determined. Referring to the coordinate system parameters of the image fusion map, the specific pixel position of each track point in the image fusion map is found through the coordinate correspondence. All these corresponding pixel positions are integrated to form a set, which is the initial associated point set of the sensor.

[0039] Centered on each pixel location in the initial set of associated points, a fixed range of image regions is defined as the neighborhood. Image features within each neighborhood are extracted one by one, including information such as texture distribution, color variations, and contour orientation. These features extracted from the neighborhood collectively constitute the neighborhood features in the image fusion atlas. The target-related semantic information contained in the neighborhood features is analyzed, such as the target's local structural attributes and surface feature attributes. This semantic information is then bound to the corresponding track points in the target track, so that each track point not only contains motion trajectory information but also includes a corresponding image semantic description. After this information supplementation, the target track is upgraded to an enhanced target track.

[0040] The spatial location information, instantaneous velocity information, and corresponding time information of each track point are extracted one by one from the enhanced track to ensure that these three types of information for each track point are complete and accurate, providing basic data support for subsequent correlation analysis.

[0041] By combining the spatial location and temporal information of each track point in the enhanced track, the correspondence between each pixel position in the image fusion map and the track point is analyzed one by one. At the same time, the closer the pixel position is to the spatial location of the track point, the higher the degree of correlation between the two. At the same time, the instantaneous velocity of the track point is used to determine its direction of motion. The correlation between the pixel position in the direction of motion and the track point is also increased accordingly. Through such comprehensive judgment, the correlation strength between each pixel position and the corresponding track point is determined.

[0042] Based on the obtained correlation strength results, the feature values ​​corresponding to each pixel position in the image fusion map are adjusted. For pixels with high correlation strength with track points, their original feature values ​​are retained and appropriately enhanced to make their features more prominent. For pixels with low correlation strength, their feature values ​​are weakened to reduce the interference of irrelevant features. After such adaptation and optimization, the processed fusion map of the image fusion map is obtained.

[0043] A reasonable distribution range for feature values ​​is set, which is determined based on the needs of target recognition and the characteristics of image data. All feature values ​​in the processed fused map that exceed this range are adjusted to the range, while ensuring that the adjusted feature values ​​can accurately reflect the correlation between the corresponding pixels and track points. After range correction, the sensor's focus map is obtained.

[0044] Based on the spatiotemporal and semantic information of the enhanced track, the feature region corresponding to each track point is found in the focus map to ensure that the motion trajectory of the track point is consistent with the feature region in the focus map in terms of time and space and matches each other semantically. The corresponding enhanced tracks are bound to the focus map to form a one-to-one combination. This combination is the track map pair of the sensor.

[0045] The beneficial effects are as follows: an initial association is established by mapping track points to the image fusion map; semantic enhancement of the target track is achieved by leveraging neighborhood features; correlation attenuation is adjusted based on the key information of the enhanced track, so that the image fusion map focuses on the target-related features; and finally, the track map pairs formed by accurate mapping and matching achieve deep binding between the target motion trajectory and image features, providing an accurate and closely related data foundation for subsequent behavioral semantic interpretation, effectively improving the efficiency of subsequent processing steps and the accuracy of target recognition.

[0046] S5. Perform behavioral semantic interpretation on the track map pairs to obtain the behavioral intentions of the sensors, and evaluate the situational representations between the behavioral intentions to obtain the situational representations of the sensors. In this embodiment of the invention, the step of performing behavioral semantic interpretation on the track map pairs to obtain the behavioral intent of the sensor, and evaluating the situational representation between the behavioral intents to obtain the situational representation of the sensor, includes: By performing pattern recognition on the enhanced trajectory and the focused map, a preliminary behavioral inference of the sensor is obtained; The preliminary behavior inference set is conflict-resolved to obtain the effective behavioral intent of the sensor; Topological association is performed on the effective behavioral intentions to obtain a relational network description of the sensors; Based on the relationship network description, a situation assessment is performed on the effective behavioral intention to obtain the situation representation of the sensor; The process of evaluating the situational tendency of the effective behavioral intention based on the relationship network description to obtain the situational characterization of the sensor includes: Interaction determination is performed on the relationship network description to obtain the logical interaction type of the relationship network description; Based on the logical interaction type, determine the situational tendency coefficient of the behavioral intention; Based on the situation tendency coefficient, the situation characterization is calculated, wherein the calculation formula for the situation characterization is as follows: ; in, This is the situational characterization. For the first The and the first The effective situation coefficient between individual behavioral intentions For the first The and the first Directional factors between behavioral intentions.

[0047] By combining the target's motion trajectory, instantaneous velocity, and semantic information contained in the enhanced track with the target's visual features, local details, and spatial distribution information in the focused image, and comparing them with a preset behavior pattern library containing combinations of motion and visual features corresponding to various common target behaviors, the information from the enhanced track and focused image is compared with the various behaviors in the pattern library one by one. Based on the degree of matching, several possible judgments corresponding to the target's current state are made. These judgments together constitute the sensor's preliminary behavior inference.

[0048] By reviewing all inference results in the initial behavior inference set, we check whether there are any inferences that contradict each other at the same time dimension or logical level. For example, if the target is inferred to be in a "moving" state and also inferred to be in a "stationary" state at the same time, we focus on enhancing the continuity of the trajectory and the instantaneous speed change law as the core basis. At the same time, we combine the stability and consistency of the target features in the focus map, retain the inference results that are consistent with the motion law and have a higher matching degree with the focus map features, and eliminate the inferences that are contradictory and do not conform to the actual situation, and finally obtain the effective behavior intention of the sensor.

[0049] Based on the chronological order of the occurrence of effective behavioral intentions, the start time, duration, and end time of each behavioral intention are identified. At the same time, the logical relationships between different behavioral intentions are analyzed. For example, the occurrence of one effective behavioral intention is a prerequisite for another behavioral intention, or multiple behavioral intentions occur simultaneously and cooperate with each other. These temporal relationships and logical connections are presented in a structured form, clarifying the position of each effective behavioral intention in the overall behavioral sequence and its relationship with other behavioral intentions, thus forming a sensor relationship network description.

[0050] Analyze the interaction between each pair of valid behavioral intentions in the relational network description one by one to determine whether they promote each other, inhibit each other, or have no influence on each other. Mutually promoting means that the occurrence of one behavioral intention will promote the implementation of another behavioral intention or strengthen its effect. Mutually inhibiting means that the occurrence of one behavioral intention will hinder the implementation of another behavioral intention or weaken its effect. Having no influence on each other means that the occurrence and development of two behavioral intentions are independent of each other and have no direct relationship. Through such analysis, the logical interaction type of the relational network description is determined.

[0051] For each type of logical interaction, a corresponding fixed value is set. The mutually promoting logical interaction corresponds to a positive fixed value, the mutually inhibiting logical interaction corresponds to a negative fixed value, and the mutually unaffected logical interaction corresponds to a zero value. Based on the logical interaction type between each pair of valid behavioral intentions determined earlier, a corresponding fixed value is assigned to each group of behavioral intentions. These values ​​are the trend tendency coefficients of the behavioral intentions.

[0052] Collect the situational tendency coefficients and corresponding directional factors between all pairs of valid behavioral intentions. The directional factors are determined according to the logical direction between the behavioral intentions, that is, clarifying who has an effect on whom. First, calculate the product of each group of situational tendency coefficients and the corresponding directional factors, then add all the product results to get the sum. At the same time, calculate the sum of squares of all situational tendency coefficients, then take the square root of the sum of squares, and finally divide the sum of the previously obtained products by this square root. The calculation result is the situational representation of the sensor.

[0053] Interaction discrimination is performed on the relation network description to obtain the logical interaction type of the relation network description, and the first interaction type is determined based on the logical interaction type. The and the first The effective situation coefficient between each behavioral intention, the first The and the first The directional factor between two behavioral intentions is based on the directional attributes of the two behavioral intentions themselves, and is directly extracted from the effective behavioral intentions obtained during the behavioral semantic interpretation process and the relational network description after topological association.

[0054] For all the The and the first The effective situation coefficient and directional factor between each behavioral intention are multiplied, and the sum of all multiplication results is used to obtain the numerator. This process is repeated for all the... The and the first The effective situation coefficients between multiple behavioral intentions are squared and summed. The square root of the sum is then taken to obtain the denominator. The numerator is divided by the denominator to obtain the situation characterization. This result can accurately reflect the degree of situation correlation between multiple effective behavioral intentions, providing key data support for subsequent structured analysis of track maps based on situation characterization and obtaining target recognition results.

[0055] When all the first The and the first The sum of the effective situation coefficients and directional factors among the behavioral intentions increases, and all the... The and the first When the square root of the sum of squares of the effective situation coefficients among the individual behavioral intentions decreases, the final situation representation will increase. The and the first The sum of the effective situation coefficients and directional factors among the individual behavioral intentions decreases, or all the... The and the first When the square root of the sum of squares of the effective situation coefficients between behavioral intentions increases, the final situation representation will decrease.

[0056] The beneficial effects are that pattern recognition accurately extracts preliminary inferences related to target behavior, conflict resolution ensures the accuracy of behavioral intent, topological association clearly presents the internal logic of behavioral intent, interaction discrimination and determination of situation tendency coefficient provide a reasonable basis for situation characterization calculation, and the final situation characterization can comprehensively reflect the interrelationships and overall trends between target behavioral intents, providing accurate situational basis for obtaining subsequent target identification results and improving the logic and reliability of target identification.

[0057] S6. Based on the situational characterization, the track map is structured and analyzed to obtain the target recognition result of the sensor.

[0058] In this embodiment of the invention, the step of performing structured analysis on the track map pair based on the situational characterization to obtain the target recognition result of the sensor includes: Based on the situational characterization, target extraction is performed on the track map pair to obtain the target entity of the track map pair; Intent matching is performed between the target entity and the valid behavioral intent to obtain the intent representation of the target entity; The interaction relationship is parsed to obtain the relationship network of the intent representation; Based on the relationship network, the intent representation is identified and determined to obtain the target recognition result of the sensor.

[0059] Based on the interrelationships and overall trends between target behavioral intentions reflected in the situational representation, this study focuses on the enhanced track and focused map in the track map pair. Core data with continuous motion trajectories are extracted from the enhanced track, and image regions with clear and complete visual features that precisely correspond to the motion trajectory are selected from the focused map. These core motion data are then bound to the corresponding image regions to clarify their attributes as independent identification objects. Finally, the target entities with complete motion information and clear visual features in the track map pair are extracted.

[0060] The motion and visual features of each target entity are analyzed one by one. Motion features include the target entity's trajectory and speed changes, while visual features include the target entity's outline and texture details. At the same time, the features of the target entity are compared with the features of all valid behavioral intentions. The features of the target entity are compared one by one to find the valid behavioral intentions that perfectly match the features of the target entity. The valid behavioral intentions are then associated and bound to the target entity, so that the target entity not only contains its own motion and visual information, but also has a clear behavioral direction, forming the intention representation of the target entity.

[0061] A comprehensive analysis is conducted on the temporal sequence, spatial distribution, and behavioral intent types of the target entities corresponding to all intent representations. This analysis determines the logical connections between different intent representations. For example, some target entities corresponding to certain intent representations have sequential relationships in time, some target entities corresponding to certain intent representations have adjacent or interactive relationships in space, and some behavioral intents corresponding to certain intent representations have synergistic or causal relationships. These logical connections across different dimensions are organized and presented in a structured form to construct a relational network of intent representations that clearly reflects the intrinsic connections between each intent representation.

[0062] A target recognition standard library is pre-established, which contains the intention representation relationship network features corresponding to various common targets. The constructed intention representation relationship network is compared with all features in the standard library one by one. The key information such as the number of target entities, behavioral intention type, and association logic pattern in the relationship network is checked to see how well they match the features in the standard library. The target type corresponding to the standard library feature with the highest matching degree is selected and the target type is determined as the target recognition result of the sensor.

[0063] The beneficial effects are that the precise guidance of situational representation ensures the targeting and accuracy of target entity extraction, intent matching realizes the deep binding between target entities and behavioral intents, interaction relationship analysis clarifies the inherent logical connection between various intent representations, and finally, the identification and judgment based on the relationship network fully combines the reference value of the preset standard library, comprehensively improving the accuracy, logic and reliability of target identification results, and providing accurate target information support for subsequent related decisions.

[0064] like Figure 2 The diagram shown is a functional block diagram of a collaborative target recognition system that integrates dot traces and images according to an embodiment of the present invention.

[0065] The collaborative target recognition system 100 that integrates point traces and images, as described in this invention, can be installed in an electronic device. Depending on the functions implemented, the collaborative target recognition system 100 may include a spatiotemporal registration synchronization module 101, a temporal trajectory generation module 102, an image fusion map generation module 103, a trajectory map association module 104, a behavior semantic interpretation module 105, and a target recognition module 106. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0066] In this embodiment, the functions of each module / unit are as follows: The spatiotemporal registration synchronization module 101 is used to synchronously acquire point data and image data from the sensor, and perform spatiotemporal registration on the point data and the image data to obtain the collaborative data of the sensor. The time-series trajectory generation module 102 is used to perform time-series correlation fitting on the point data based on the collaborative data to obtain the target trajectory of the sensor. The image fusion map generation module 103 is used to perform multi-scale feature fusion on the image data based on the collaborative data to obtain the image fusion map of the sensor; The track map association module 104 is used to perform bidirectional interconnection and association between the target track and the image fusion map to obtain the track map pair of the sensor; The behavior semantic interpretation module 105 is used to perform behavior semantic interpretation on the track map pair to obtain the behavior intention of the sensor, and evaluate the situational representation between the behavior intentions to obtain the situational representation of the sensor. The target recognition module 106 is used to perform structured analysis on the track map pair based on the situation characterization to obtain the target recognition result of the sensor.

[0067] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0068] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0069] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0070] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0071] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A collaborative target recognition method integrating dot patterns and images, characterized in that, The method includes: S1. Simultaneously acquire point data and image data from the sensor, and perform spatiotemporal registration on the point data and the image data to obtain the collaborative data of the sensor; S2. Based on the collaborative data, perform time-series correlation fitting on the point data to obtain the target trajectory of the sensor; S3. Based on the collaborative data, perform multi-scale feature fusion on the image data to obtain the image fusion map of the sensor; S4. The target trajectory is bidirectionally interconnected with the image fusion map to obtain the trajectory map pair of the sensor; S5. Perform behavioral semantic interpretation on the track map pairs to obtain the behavioral intentions of the sensors, and evaluate the situational representations between the behavioral intentions to obtain the situational representations of the sensors. S6. Based on the situational characterization, the track map is structured and analyzed to obtain the target recognition result of the sensor.

2. The collaborative target recognition method fusing dot patterns and images as described in claim 1, characterized in that, The process of synchronously acquiring point data and image data from sensors, and performing spatiotemporal registration on the point data and image data to obtain collaborative data from the sensors includes: Simultaneously acquire point data and image data from the sensor; The dot data and the image data are time-aligned to obtain the intermediate dot data and intermediate image data of the sensor; Mapping the intermediate dot data to the coordinate system of the intermediate image data yields the spatial alignment data of the dot data. The spatial alignment data is fused and encapsulated with the intermediate image data to obtain the collaborative data of the sensor.

3. The collaborative target recognition method fusing dot patterns and images as described in claim 1, characterized in that, The step of performing time-series correlation fitting on the point data based on the collaborative data to obtain the target trajectory of the sensor includes: Based on the collaborative data, spatiotemporal relationship identification is performed on the point data to obtain the potential correlation of the point data; Based on the potential correlation, the point data is linked in a regular manner to obtain continuous segments of the point data; The continuous segments are smoothed and fitted to obtain the target trajectory of the point data.

4. The collaborative target recognition method fusing dot patterns and images as described in claim 1, characterized in that, The step of performing multi-scale feature fusion on the image data based on the collaborative data to obtain the image fusion map of the sensor includes: Based on the collaborative data, the image data is deconstructed hierarchically to obtain a multi-scale data subset of the image data; Cross-scale feature interaction is performed on the multi-scale data subset to obtain heterogeneous features of the image data; The heterogeneous features are topologically integrated to obtain the image fusion map of the sensor.

5. The collaborative target recognition method fusing dot patterns and images as described in claim 1, characterized in that, The step of bidirectionally linking the target trajectory with the image fusion map to obtain the sensor trajectory map pair includes: Mapping the track points in the target track to the image fusion map yields the initial associated point set of the sensor; Based on the initial set of associated points, neighborhood features are extracted from the image fusion map; Based on the neighborhood features, the target trajectory is semantically enhanced to obtain the enhanced trajectory of the target trajectory; Based on the enhanced trajectory, the correlation attenuation adjustment is performed on the image fusion map to obtain the focused map of the image fusion map; The enhanced track and the focused map are mapped and matched to obtain the track map pair of the sensor.

6. The collaborative target recognition method fusing dot patterns and images as described in claim 5, characterized in that, The step of adjusting the correlation attenuation of the image fusion map based on the enhanced trajectory to obtain the focus map of the image fusion map includes: The spatial position, instantaneous velocity, and time information of the enhanced trajectory are obtained; Based on the spatial location and the time information, the correlation between the pixel location in the image fusion map and the track point in the enhanced track is analyzed to obtain the correlation strength between the pixel location and the track point. Based on the correlation strength, the feature values ​​in the image fusion map are adapted and optimized to obtain the processed fusion map of the image fusion map; The processed fused spectrum is then subjected to interval correction to obtain the focused spectrum of the sensor.

7. The collaborative target recognition method fusing dot patterns and images as described in claim 1, characterized in that, The process of interpreting the behavioral semantics of the track map pairs to obtain the sensor's behavioral intent, and evaluating the situational representations between the behavioral intents to obtain the sensor's situational representation, includes: By performing pattern recognition on the enhanced trajectory and the focused map, a preliminary behavioral inference of the sensor is obtained; The preliminary behavior inference set is conflict-resolved to obtain the effective behavioral intent of the sensor; Topological association is performed on the effective behavioral intentions to obtain a relational network description of the sensors; Based on the relationship network description, a situation assessment is performed on the effective behavioral intentions to obtain the situational characterization of the sensor.

8. The collaborative target recognition method fusing dot patterns and images as described in claim 7, characterized in that, The process of evaluating the situational tendency of the effective behavioral intention based on the relationship network description to obtain the situational characterization of the sensor includes: Interaction determination is performed on the relationship network description to obtain the logical interaction type of the relationship network description; Based on the logical interaction type, determine the situational tendency coefficient of the behavioral intention; Based on the situation tendency coefficient, the situation characterization is calculated, wherein the calculation formula for the situation characterization is as follows: ； in, This is the situational characterization. For the first The and the first The effective situation coefficient between individual behavioral intentions For the first The and the first Directional factors between behavioral intentions.

9. The collaborative target recognition method fusing dot patterns and images as described in claim 1, characterized in that, The step of performing structured analysis on the track map pairs based on the situational characterization to obtain the target recognition results of the sensor includes: Based on the situational characterization, target extraction is performed on the track map pair to obtain the target entity of the track map pair; Intent matching is performed between the target entity and the valid behavioral intent to obtain the intent representation of the target entity; The interaction relationship is parsed to obtain the relationship network of the intent representation; Based on the relationship network, the intent representation is identified and determined to obtain the target recognition result of the sensor.

10. A collaborative target recognition system integrating dot patterns and images, characterized in that, The system for implementing the collaborative target recognition method that fuses dot traces and images as described in claim 1 includes: The spatiotemporal registration and synchronization module is used to synchronously acquire point data and image data from the sensor, and to perform spatiotemporal registration on the point data and the image data to obtain the collaborative data of the sensor. The time-series trajectory generation module is used to perform time-series correlation fitting on the point data based on the collaborative data to obtain the target trajectory of the sensor; The image fusion map generation module is used to perform multi-scale feature fusion on the image data based on the collaborative data to obtain the image fusion map of the sensor; The track map association module is used to perform bidirectional communication and association between the target track and the image fusion map to obtain the track map pair of the sensor; The behavioral semantic interpretation module is used to perform behavioral semantic interpretation on the track map pairs to obtain the behavioral intentions of the sensor, and evaluate the situational representations between the behavioral intentions to obtain the situational representation of the sensor. The target recognition module is used to perform structured analysis on the track map pair based on the situational characterization to obtain the target recognition result of the sensor.

Images