Methods, devices, electronic equipment, and media for identifying moving objects based on remote sensing images.

Abstract

This application provides a method, apparatus, electronic device, and medium for identifying moving objects based on remote sensing imagery, relating to the field of image processing. The method includes the following steps: acquiring remote sensing imagery containing moving objects; extracting image data information and spatiotemporal information associated with the moving objects from the remote sensing imagery; constructing prompt words based on the image data information and spatiotemporal information; using a large language model to perform type identification on the image data information based on the prompt words to obtain the type of the moving object; using an external application connected to the large language model to verify the spatiotemporal information based on the prompt words to obtain a verification result characterizing whether the spatiotemporal information passes the verification; and performing logical reasoning analysis on the verification result based on the type of the moving object to generate an identification result of the moving object in the remote sensing imagery. This application can solve the technical problems of large detection error and low reliability in related technologies.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device and medium for recognizing moving objects based on remote sensing images. Background Technology

[0002] The demand for aircraft detection and identification in remote sensing imagery continues to grow, with widespread applications in aviation supervision, public safety, and other scenarios. With the rapid development of the aviation industry and remote sensing technology, high-resolution remote sensing satellites generate massive amounts of data (TB or even PB levels) daily, creating an urgent need for automated and intelligent aircraft target detection and identification technologies.

[0003] In related technologies, computer vision models are mainly used. These models rely on pixel-level features (such as aircraft outlines and textures) for pattern recognition. They cannot incorporate prior domain rules such as "an airport only accommodates civilian passenger aircraft", which leads to logical misjudgments in complex scenarios, resulting in large errors in detection results and low reliability. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, electronic device, and medium for identifying moving objects based on remote sensing images. This can solve the technical problems of large detection errors and low reliability in related technologies.

[0005] According to one embodiment of this application, a method for identifying moving objects based on remote sensing imagery is provided, comprising the following steps: acquiring remote sensing imagery containing moving objects; extracting image data information and image spatiotemporal information associated with the moving objects from the remote sensing imagery; constructing prompt words based on the image data information and image spatiotemporal information; using a large language model to perform type identification on the image data information based on the prompt words to obtain the type of moving object; using an external application connected to the large language model to verify the image spatiotemporal information based on the prompt words to obtain a verification result characterizing whether the image spatiotemporal information passes the verification; and performing logical reasoning analysis on the verification result based on the type of moving object to generate an identification result of the moving object in the remote sensing imagery.

[0006] According to embodiments of this application, the image data information includes at least one of the following: bounding box location information of the moving object, initial type identification information of the moving object, and confidence information of the moving object; the image spatiotemporal information includes at least one of the following: the geographical location information of the remote sensing image and the timestamp information of the remote sensing image.

[0007] According to an embodiment of this application, constructing prompt words based on image data information and image spatiotemporal information includes: obtaining standard attribute features of moving objects in remote sensing images based on image data information; obtaining historically occurring object types corresponding to the geographical locations presented in the image spatiotemporal information based on the shooting geographical location information; and using the standard attribute features and historically occurring object types as prompt words.

[0008] According to embodiments of this application, the image data information is type-identified using a large language model based on prompt words to obtain the type of moving object, including: comparing the image data information with a pre-set knowledge base in the large language model based on standard attribute features and historically occurring object types to obtain retrieval results in the knowledge base that match the image data information, wherein the knowledge base includes at least one of a vector database, a relational database, and a time-series database; determining the consistency between the standard attribute features and historically occurring object types based on the retrieval results, and generating the type of moving object based on the determined consistency.

[0009] According to an embodiment of this application, using an external application connected to a large language model to verify the spatiotemporal information of an image based on prompt words to obtain a verification result characterizing whether the spatiotemporal information of the image passes the verification includes: parsing the prompt words to identify the trigger conditions that require calling the external application; calling the interface of the external application according to the trigger conditions to obtain real-time information of the shooting geographical location presented in the spatiotemporal information of the image through the interface, wherein the interface of the external application includes at least one of an air traffic control data interface, a meteorological data interface, and a historical record interface; determining the consistency between the real-time information and the spatiotemporal information of the image, and generating a verification result characterizing whether the spatiotemporal information of the image passes the verification based on the determined consistency.

[0010] According to an embodiment of this application, logical reasoning analysis is performed on the verification result based on the type of the moving object to generate the identification result of the moving object in the remote sensing image, including: comparing the type of the moving object with the verification result; determining whether the verification result supports the appearance of the type of the moving object in the geographical location and timestamp corresponding to the spatiotemporal information of the image; and if the verification result supports the appearance of the type of the moving object in the geographical location and timestamp corresponding to the spatiotemporal information of the image, using the type of the moving object as the identification result.

[0011] According to an embodiment of this application, the verification result is logically reasoned and analyzed based on the type of the moving object to generate the identification result of the moving object in the remote sensing image. The method further includes: if the verification result does not support the appearance of the type of the moving object in the geographical location and timestamp corresponding to the spatiotemporal information of the image, the type of the moving object is determined to be incorrect; using a large language model, the real-time information in the verification result is correlated and compared with the image data information to generate the corrected type of the moving object, and the corrected type of the moving object is used as the identification result.

[0012] According to one embodiment of this application, an apparatus for identifying moving objects based on remote sensing images is provided, including an image acquisition module, a prompt word construction module, a category recognition module, an information verification module, and a recognition module. The image acquisition module is used to acquire remote sensing images containing moving objects and extract image data information and image spatiotemporal information associated with the moving objects from the remote sensing images. The prompt word construction module is used to construct prompt words based on the image data information and image spatiotemporal information. The category recognition module is used to use a large language model to perform type recognition on the image data information based on the prompt words to obtain the type of the moving object. The information verification module is used to use an external application connected to the large language model to verify the image spatiotemporal information based on the prompt words to obtain a verification result characterizing whether the image spatiotemporal information passes the verification. The recognition module is used to perform logical reasoning analysis on the verification result based on the type of the moving object to generate the recognition result of the moving object in the remote sensing image.

[0013] According to one aspect of this application, an electronic device is provided, including one or more processors and a storage device; the storage device is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors perform the method described above.

[0014] According to another embodiment of this application, a computer-readable storage medium is provided, on which a computer program or instructions are stored, which, when executed by a processor, implement the above-described method. The moving object recognition method, electronic device, and computer-readable storage medium based on remote sensing images provided in this application can achieve the following technical effects:

[0015] This application's embodiments involve acquiring remote sensing images containing moving objects, extracting image data information and spatiotemporal information associated with the moving objects from the remote sensing images, constructing prompt words based on the image data information and spatiotemporal information, using a large language model to perform type identification on the image data information based on the prompt words to obtain the type of the moving object, and using an external application connected to the large language model to verify the spatiotemporal information based on the prompt words to obtain a verification result characterizing whether the spatiotemporal information passes the verification, and performing logical reasoning analysis on the verification result based on the type of the moving object to generate the identification result of the moving object in the remote sensing image. By employing a large language model and simultaneously calling an external application, unified storage and associated retrieval of unstructured domain rules and structured data are achieved, realizing domain knowledge integration and providing knowledge support for reasoning. By designing a retrieval, invocation, and fusion process, the logical reasoning of the large language model is integrated, improving the overall logical reasoning ability, reducing the error of the detection results, and further enhancing the credibility of the identification report. Attached Figure Description

[0016] The above and other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 A flowchart illustrating one of the moving object recognition methods based on remote sensing images according to an embodiment of this application is shown schematically;

[0018] Figure 2 This illustration schematically shows one of the flowcharts for step S20 in the moving object recognition method based on remote sensing images according to an embodiment of this application;

[0019] Figure 3 This illustration schematically shows one of the flowcharts for step S40 in the moving object recognition method based on remote sensing images according to an embodiment of this application;

[0020] Figure 4 This schematic diagram illustrates the structural block diagram of a moving object recognition method apparatus based on remote sensing images according to an embodiment of this application;

[0021] Figure 5 A block diagram of an electronic device for a moving object recognition method based on remote sensing images according to an embodiment of this application is shown schematically. Detailed Implementation

[0022] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] In related technologies, computer vision models are mainly used. These models rely on pixel-level features (such as aircraft outlines and textures) for pattern recognition. They cannot incorporate prior domain rules such as "an airport only accommodates civilian passenger aircraft", which leads to logical misjudgments in complex scenarios, resulting in large errors in detection results and low reliability.

[0026] In view of this, the embodiments of this application provide a moving object recognition method based on remote sensing images, which has strong logical reasoning ability, small result error, and high reliability.

[0027] Figure 1 One of the flowcharts illustrating a moving object recognition method based on remote sensing imagery according to an embodiment of this application is shown schematically.

[0028] like Figure 1 As shown, the moving object recognition method based on remote sensing images according to this embodiment may include steps S10 to S50.

[0029] Step S10: Obtain remote sensing images containing moving objects, and extract image data information and image spatiotemporal information associated with the moving objects from the remote sensing images.

[0030] In one embodiment, remote sensing imagery includes satellite remote sensing imagery, aerial remote sensing imagery, low-altitude remote sensing imagery, etc. Satellite remote sensing imagery includes optical imagery, radar imagery, and infrared imagery, etc. Aerial remote sensing imagery includes aerial photogrammetric imagery, airborne LiDAR point cloud imagery, and airborne hyperspectral imagery, etc. Low-altitude remote sensing imagery includes imaging data acquired by unmanned aerial vehicles (UAVs) or airport flight platforms, etc.

[0031] Those skilled in the art will understand that although this application describes remote sensing imagery as the primary data source, its technical solutions are equally applicable to other types of digital imagery, including but not limited to images captured by ground monitoring cameras, photos taken by mobile phones, and images captured by vehicle-mounted cameras. Similar target detection and verification functions can be achieved simply by adapting the visual inspection module accordingly.

[0032] In this embodiment, a moving object refers to a man-made or natural object in remote sensing imagery that can be identified and has the ability or attribute to move. Depending on the application scenario and the granularity of identification, the moving object may include aircraft, vehicles, ships, etc. When the moving object is an aircraft, the aircraft may be a civil airliner, cargo plane, fighter jet, helicopter, drone, airship, etc. Specifically, in this embodiment, an aircraft will be used as the moving object to explain this application in detail.

[0033] In one embodiment, step S10 includes the following steps: processing the input remote sensing image containing moving objects through a visual detection module to obtain image data information associated with the moving objects.

[0034] The image data information includes at least one of the following: bounding box location information of the moving object, initial type identification information of the moving object, and confidence information of the moving object.

[0035] Image data information refers to the inherent information presented in remote sensing images. For example, the bounding box location information, initial type identification information, and confidence level information of moving objects within the image data information. The bounding box location information refers to the range of pixel coordinates of the moving object in the image, such as the coordinates of the upper left corner and upper right corner. Taking an aircraft as an example, the initial type identification information refers to the type of aircraft. When the moving object is an airplane, the category can be directly identified, such as "Type A aircraft" or "Type B aircraft." The confidence level of the moving object refers to the probability distribution of that detection result. Specifically, when the moving object is an airplane, the image data information may also include the aircraft's length, width, wing shape, engine type, and number of fuel tanks.

[0036] In one embodiment, the visual inspection module is communicatively connected to a user client module, which receives user requests. For example, a user can upload remote sensing images through the user client module, which then transmits the uploaded images to the visual inspection module for recognition. The user client module can also receive the recognition results.

[0037] In one embodiment, when the visual inspection module receives remote sensing images transmitted by the user client module, it extracts image data information associated with moving objects from the remote sensing images by executing a target algorithm. Furthermore, the type of visual inspection model is not limited; any model conventionally used in the field can be employed, and will not be elaborated upon here.

[0038] Step S10 further includes the following steps: acquiring spatiotemporal information of the image associated with the moving object, wherein the spatiotemporal information of the image includes at least one of the geographical location information of the remote sensing image and the timestamp information of the remote sensing image.

[0039] In this embodiment, the metadata of the remote sensing image is read, and the center point or boundary latitude and longitude and acquisition time are parsed from the metadata. If necessary, the reverse geocoding API is called to convert the coordinates into readable location names. The extracted geographic location information and timestamp information are saved in a structured form as the spatiotemporal information of the image.

[0040] Spatiotemporal information of images refers to information associated with the scene where a remote sensing image was captured, such as the location and time of capture. This information typically originates from the image file's metadata or external services. Examples include geographic location information and timestamp information. Geographic location information includes geographic coordinates, such as latitude and longitude ranges and center coordinates, and may also include specific location names, such as a specific airport or town. Timestamp information includes the specific date, time, time characteristics, and temporal environmental information. For example, time characteristics include daytime, nighttime, and season. Temporal environmental information includes illumination angle and weather conditions.

[0041] In one embodiment, the visual detection module deploys a model fine-tuned from a remote sensing image dataset. By executing a target detection algorithm, it outputs image data information. Simultaneously, the visual detection module supports model version updates and parameter adjustments, allowing for updates at any time based on user needs. Furthermore, the remote sensing image dataset can be any dataset conventionally used in the field, and will not be elaborated further here.

[0042] Step S20: Construct prompt words based on image data information and image spatiotemporal information.

[0043] It should be noted that the large language model is essentially a text generation model. The large language model cannot understand image data information and image spatiotemporal information. Therefore, it is necessary to convert the image data information and image spatiotemporal information into information that the large language model can understand so that the large language model can process it.

[0044] Figure 2 One of the flowcharts illustrating step S20 in the moving object recognition method based on remote sensing imagery according to an embodiment of this application is shown.

[0045] like Figure 2 As shown, step S20 above may further include steps S21 to S23.

[0046] Step S21: Based on the image data information, obtain the standard attribute features of moving objects in the remote sensing image.

[0047] In this embodiment, standard attribute features refer to the inherent physical or geometric parameters of a specific type of moving object that are pre-stored in the large language model. For example, when the moving object is an airplane, the standard attribute features are the airplane's standard length, standard wingspan, typical number and layout of engines, wing shape, etc.

[0048] Specifically, in one embodiment, taking the Type A aircraft as an example, the standard characteristics of the Type A aircraft are: length 33.6 meters, wingspan 34.3 meters, number of engines 2, and wing shape: swept low-wing monoplane.

[0049] Step S22: Based on the shooting geographical location information in the image spatiotemporal information, obtain the historical object type corresponding to the geographical location presented in the shooting geographical location information.

[0050] In this embodiment, the historical object type refers to the set of moving object types that have appeared in the past within the time range of the shooting geographical location (such as a specific airport or geographical area), statistically analyzed through historical detection records. This information reflects regional prior constraint rules. For example, if the shooting geographical location of the image's spatiotemporal information presented in the remote sensing image is a certain civil airport, detection reveals that Type A aircraft have historically appeared at this civil airport.

[0051] Step S23: Use standard attribute characteristics and historically occurring object types as prompt words.

[0052] In this embodiment, the acquired standard attribute features and historically occurring object types are combined according to a preset prompt word template to generate structured text prompt words. These prompt words are input into the large language model to guide it in performing subsequent logical reasoning tasks.

[0053] In one embodiment, taking an aircraft as an example, the visual detection module detects that the aircraft is a Type A aircraft. Standard attribute features are obtained: the moving object is a Type A aircraft, with a length of 33.6 meters, a wingspan of 34.3 meters, two engines, and a swept-back low-wing configuration. Historical object types are obtained: the shooting location is a civil airport, and historical detection records show that aircraft types that have historically appeared at this civil airport include Type A, Type B, and Type C aircraft.

[0054] In this embodiment, the prompt words simultaneously include "regional rule knowledge" and "physical attribute knowledge". When performing secondary interpretation, the large language model no longer relies solely on the pixel-level pattern matching results output by the visual model. Instead, it can actively determine whether the detection result is abnormal in complex scenarios such as "a large passenger plane was detected in a remote mountainous area" based on the regional rules (there is no large airport in the area) and attribute knowledge (a large passenger plane should have a long fuselage and wingspan) in the prompt words.

[0055] Specifically, based on the geographic location information captured in the image's spatiotemporal information, the historical object type corresponding to that geographic location is obtained (e.g., a civilian airport has historically only seen Type A aircraft), and this empirical constraint information is written into prompt words in natural language form. This allows the large language model to clearly know the prior rule that "this area usually only allows civilian passenger aircraft to park," thus avoiding misclassifying military transport aircraft that meet pixel features but violate regional rules as real targets. Based on the initial type identifier of the moving object in the image data, the standard attribute features of the object (such as standard length, wingspan, number of engines, etc.) are retrieved from the knowledge base, and this standard reference information is written into prompt words. This allows the large language model to compare the actual observed attributes (such as detected length, wing shape) output by the visual detection module with the standard attributes in the knowledge base item by item, promptly detecting attribute deviations caused by light and shadow interference, partial occlusion, or similar ground features, and thus determining false alarms.

[0056] Step S30: Use a large language model to identify the type of the image data information based on the prompt words to obtain the type of the moving object.

[0057] Specifically, step S30 further includes the following steps: based on standard attribute features and historically occurring object types, the image data information is compared with the pre-set knowledge base in the large language model to obtain the retrieval information in the knowledge base that matches the image data information. The knowledge base includes at least one of vector database, relational database and time series database. Based on the retrieval information, the consistency between the standard attribute features and the historically occurring object types is determined, and the type of motion object is generated based on the determined consistency.

[0058] In this embodiment, after receiving the prompt word, the large language model parses the key elements in the prompt word, calls the knowledge base, and retrieves the standard attribute features and historically occurring object types from the knowledge base. The large language model obtains the image data information in the prompt word (including the initial type identifier, bounding box position, confidence level, and auxiliary attribute features output by the visual detection module, such as detected length, wing shape, number of engines, etc.), and compares it item by item with the standard attribute features and historically occurring object types retrieved from the knowledge base, and provides the retrieval results.

[0059] Specifically, the retrieval process includes attribute consistency verification and regional rule attribution verification. Attribute consistency verification involves numerically or logically matching the detected attribute features (e.g., "detected length 32.1 meters") with the corresponding standard attribute features in the knowledge base (e.g., "standard length of type A aircraft 33.6 meters"). Regional rule attribution verification involves comparing the detected initial type identifier (e.g., "type A aircraft") with the set of historically occurring object types in the knowledge base for that geographical location (e.g., "historical aircraft types at a certain international airport: type A aircraft, type B aircraft"). Based on the retrieval results, the large language model performs consistency judgment, comparing the attribute features output by the visual detection module (e.g., detected length, wing shape, number of engines) with the standard attribute features of that object type in the knowledge base item by item, and generating the type of the moving object.

[0060] In one embodiment, a vector database converts object type or attribute descriptions from prompts into embedded vectors, performs semantic similarity retrieval, and obtains unstructured domain knowledge text (such as detailed aircraft technical parameters and airport operation rules); generally, a vector database obtains unstructured domain knowledge text. A relational database extracts structured query conditions (such as geographical location and historical time periods) from prompts, generates SQL (Structured Query Language) statements, and obtains structured historical detection records or standard parameter tables. Generally, a relational database stores historical detection records and basic airport information. A time-series database extracts timestamp information to obtain time-series sensor data near that point in time; such as QAR (Quick Access Recorder) flight parameters and real-time detection data.

[0061] Vector databases include Chroma (an open-source embedded vector database) and Milvus (a high-performance, highly scalable open-source vector database). Vector databases are used to store unstructured text. The unstructured text stored in vector databases can be referenced from the texts commonly used in this field, and will not be elaborated upon here.

[0062] Relational databases (MySQL) are used to store historical detection records and basic airport information. For specific types of relational databases, please refer to the general settings in this field; they will not be elaborated upon here.

[0063] InfluxDB (Time Series Database) is used to store time-series data, such as QAR (Quick Access Recorder) flight parameters and real-time detection data. The specific types of time-series databases can be found in the general settings of this field and will not be elaborated upon here.

[0064] The knowledge base supports multiple search methods, such as semantic similarity search and statement query. You can choose according to the actual situation. For specific search modes, refer to the conventional settings in this field, which will not be elaborated here.

[0065] In one embodiment, the large language model also includes a server. The server communicates with the knowledge base and external applications. The server acts as the core inference engine and has a locally deployed model built in. It is configured with a standardized prompt word template library (adapted to different scenarios, such as airport detection and remote area detection) and function call protocol to coordinate the entire process of tasks: triggering knowledge base retrieval, calling external application modules, performing multi-source fusion inference, and outputting inference conclusions.

[0066] Related technologies point out that the output only includes bounding boxes, types, and confidence levels, lacking reasoning basis. Users cannot know the specific reasons why the model judges "the target is a type A aircraft," leading to the need for extensive manual verification in high-reliability scenarios such as aviation supervision, reducing work efficiency. In step S30, a large language model is introduced to execute the process of "knowledge base retrieval—consistency comparison—structured output." At the same time, an innovative combination of "vector database and relational database (or time-series database)" is adopted to achieve unified storage and associated retrieval of unstructured domain rules and structured data, solving the problem of "inability to integrate domain knowledge" and providing knowledge support for reasoning.

[0067] In this embodiment, the large language model compares the image data information (such as length and wing shape) output by the visual detection module with the standard attribute features retrieved from the knowledge base item by item, and records the comparison process in natural language. Users can directly see that "detected length 32.1 meters vs standard length 33.6 meters, deviation 4.5% within the threshold; wing shape 'swept wing' consistent with the standard; number of engines 2, layout wing-mounted consistent"—these clear and traceable comparison items constitute the specific basis for judging "the target is a type A aircraft".

[0068] Step S40: Using an external application connected to the large language model, the spatiotemporal information of the image is verified based on the prompt words to obtain a verification result representing whether the spatiotemporal information of the image passes the verification.

[0069] In this embodiment, the large language model calls an external real-time data source to cross-validate the spatiotemporal information (geographic location, timestamp) of the image to determine whether it is consistent with the dynamic situation in the real world, thereby identifying potential metadata errors, spoofing attacks or environmental anomalies.

[0070] Figure 3 The flowchart illustrating step S40 of the moving object recognition method based on remote sensing imagery according to an embodiment of this application is shown in the illustration.

[0071] like Figure 3 As shown, step S40 above further includes steps S41 to S43.

[0072] Step S41: Parse the prompt words and identify the trigger conditions that require calling an external application.

[0073] The large language model first performs semantic parsing on the prompt words constructed in step S20, extracting key elements that may trigger external calls. For example, when the geographical location is identified as a place with dynamic operating rules, such as a civil airport, military airport, port, or highway, the corresponding type of external application is triggered; when the timestamp is identified as severe weather, it is necessary to trigger a call to an external application for meteorology or air traffic control.

[0074] Step S42: Call the interface of an external application according to the triggering condition, and obtain the real-time information of the shooting geographical location presented in the spatiotemporal information of the image through the interface. The interface of the external application includes at least one of the air traffic control data interface, meteorological data interface and historical record interface.

[0075] In this embodiment, the air traffic control data interface can query real-time flight schedules, runway closure / open status, delay information, etc.; the meteorological data interface can query real-time weather conditions, weather warnings, etc.; and the historical record interface can query historical detection records and dwell time distribution for that geographical location over a past period. Incorporating real-time data such as air traffic control data, meteorological data, and historical records into the verification chain enables the system to identify false detections caused by metadata errors or sudden environmental changes.

[0076] Step S43: Determine the consistency between real-time information and image spatiotemporal information, and generate a verification result representing whether the image spatiotemporal information passes the test based on the determined consistency.

[0077] In this embodiment, the large language model performs a consistency check on the real-time information obtained from the external application and the image spatiotemporal information (shooting location, timestamp) in the prompt words. The image spatiotemporal information (location, timestamp) is directly read from the metadata, but the metadata may be inaccurate due to sensor failure, human tampering, or transmission errors. In addition, even if the metadata is correct, it cannot directly reflect the dynamic environment at the time (such as temporary air traffic control closure, sudden weather). By calling the external application, a third-party verification source independent of the image itself is provided. The remote sensing image is no longer regarded as an isolated analysis object, but is connected to the real-time dynamic data network, realizing a qualitative change from "single spatiotemporal snapshot" to "multi-source temporal verification" and improving the credibility of the results.

[0078] Step S50: Perform logical reasoning analysis on the verification results based on the type of moving object to generate the identification results of moving objects in the remote sensing image.

[0079] In this embodiment, the type of the moving object is used to fuse the verification results for decision-making. The final recognition result is only output when both are supported or reasonably corrected. The multi-source reasoning mechanism based on a large language model can combine multi-source information for reasoning, explain the reasons for the results, and mark anomalies, significantly improving reasoning ability. Compared with methods that rely solely on visual detection or single-source knowledge, integrating multimodal data such as knowledge bases, air traffic control data, meteorological data, and historical records improves accuracy.

[0080] In some embodiments, logical reasoning analysis of the verification result based on the type of the moving object includes: comparing the type of the moving object with the verification result; and determining whether the verification result supports the appearance of the type of the moving object in the geographic location and timestamp corresponding to the spatiotemporal information of the image.

[0081] In this embodiment, information directly related to the type of moving object is extracted from the verification results, such as the actual aircraft type of the flight in the air traffic control data, whether the weather conditions allow for takeoff and landing of that type, and the typical aircraft type appearing at that location during that time period in the historical records. The type of moving object (e.g., "Type A aircraft") is matched with the aircraft type records appearing in the verification results (e.g., "Type A aircraft, Type B aircraft, Type C aircraft" returned by the air traffic control data interface) using string matching or semantic matching. Further, based on the comparison results, the large language model performs a binary logic judgment to determine whether the verification results explicitly support or in principle support the appearance of that moving object type at the specified geographical location and timestamp. For example, if there is a matching flight plan in the air traffic control data, or if the historical query records that the moving object type frequently appeared during that time period, the verification result is determined to support the type of moving object. If the air traffic control data shows no flight plans during that time period, and the historical records clearly show that the moving object type has never appeared at that location, the verification result is determined to not support the type of moving object.

[0082] In some embodiments, logical reasoning analysis is performed on the verification results based on the type of the moving object to generate the identification result of the moving object in the remote sensing image, including: when the verification result supports the appearance of the type of the moving object in the geographical location and timestamp corresponding to the spatiotemporal information of the image, the type of the moving object is used as the identification result.

[0083] In this embodiment, taking "Type A aircraft" as an example, when there is a matching flight plan in the air traffic control data, or when the type of moving object is frequently recorded in the historical query, and the verification result supports the appearance of the type of moving object in the geographical location and timestamp corresponding to the spatiotemporal information of the image (for example, the air traffic control data interface confirms that there is indeed a "Type A aircraft" flight plan in this period and the weather conditions are normal), then the large language model confirms that the type of moving object output in step S30 is correct and outputs it as the final recognition result.

[0084] The identification results include not only the type of moving object, but also the reasoning process, confidence level description, and potential anomaly markers, thereby enhancing user trust and improving the interpretability of the reasoning results.

[0085] In some embodiments, the verification results are logically reasoned and analyzed based on the type of the moving object to generate the identification result of the moving object in the remote sensing image. The method further includes: if the verification result does not support the occurrence of the type of the moving object in the geographical location and timestamp corresponding to the spatiotemporal information of the image, the type of the moving object is determined to be incorrect; using a large language model, the real-time information in the verification result is correlated and compared with the image data information to generate the corrected type of the moving object, and the corrected type of the moving object is used as the identification result.

[0086] In this embodiment, taking "Type A aircraft" as an example, if air traffic control data shows no flight schedules during the period, and historical records clearly show that this type of moving object has never appeared at that location, and the verification results do not support the presence of the moving object type in the corresponding geographical location and timestamp of the image spatiotemporal information (for example, air traffic control data shows no flights during the period, but Type A aircraft is detected in the image; or air traffic control data shows only Type B aircraft flights during the period, but the model outputs Type A aircraft), then the moving object type identification output in step S30 of the large language model is incorrect. At this time, the large language model starts the correction procedure, and the large language model compares and correlates the real-time information in the verification results (such as the actual aircraft type and flight number in the air traffic control data) with the image data information (the auxiliary attribute features detected initially, such as length and wing shape). For example, if air traffic control data shows that the aircraft type of the flight during that period is "Type B aircraft", while the visual detection module outputs "Type A aircraft", the large language model, based on the model information provided by air traffic control and combined with the observable attributes of the target in the image (such as the detection length of 37 meters, which is closer to 37.6 meters for "Type B aircraft" than 33.6 meters for "Type A aircraft"), infers that the corrected type is "Type B aircraft".

[0087] In one embodiment, after step S20, the method further includes: responding to user requests and switching between a first processing mode and a second processing mode.

[0088] The processing accuracy in the first processing mode is higher than that in the second processing mode, while the processing speed in the second processing mode is higher than that in the first processing mode. In the first processing mode, the large language model simultaneously obtains the type of the moving object and the verification result; in the second processing mode, the large language model only obtains the type of the moving object, and the verification result output is empty. This setting can meet different customer needs. If the customer only wants to quickly understand the type of the moving object, they can choose the second processing mode, which can quickly provide the type of the moving object, and the recognition result only provides the type of the moving object. When the customer wants to understand the detailed situation, they can choose the first processing mode, which not only provides the type of the moving object, but also calls external applications to perform reasonable analysis, and the recognition result includes not only the type of the moving object, but also the reasoning process, confidence level description, and potential anomaly markers.

[0089] The method provided in this application constructs a multi-dimensional knowledge base including a vector database, a relational database, and a time-series database. The vector database stores unstructured domain knowledge such as aircraft model technical parameters and airport operation rules text; the relational database stores structured historical detection records and basic airport information; and the time-series database stores dynamic sensor data such as QAR flight parameters. Through a unified retrieval interface, the large language model can simultaneously acquire semantically similar unstructured knowledge and precisely queried structured data. This enables rule-based verification of visual detection results (e.g., whether an airport only hosts civilian aircraft) and attribute comparison (e.g., whether the detected length matches the standard length). This fundamentally solves the problem of missing domain knowledge integration and avoids logical misjudgments caused by a lack of prior rules.

[0090] The method provided in this application also designs a three-step reasoning process of retrieval, invocation, and fusion. Using a large language model as the core reasoning engine, it first triggers a multi-dimensional knowledge base retrieval based on prompt words to obtain standard attributes and historical experience. Secondly, it accesses external applications via function calls to obtain real-time data such as air traffic control plans and meteorological conditions. Finally, it fuses the initial output of the visual detection module, the knowledge base retrieval results, and the data provided by the external application to perform multi-source information fusion, and performs attribute consistency verification and regional rule attribution judgment. This method enables the system to proactively determine the anomalies of detection results, such as the appearance of a large passenger aircraft in a remote area, explain the causes of the anomalies, and mark false alarms. This represents a qualitative leap from statistical matching to logical verification, significantly improving reasoning capabilities.

[0091] The method provided in this application addresses the information limitations of related technologies that rely on single remote sensing image data. By systematically integrating pixel-level features of remote sensing images, standard attributes and historical experience from a multi-dimensional knowledge base, real-time air traffic control and meteorological data provided by external applications, and historical detection records from a time-series database, this method uses a unified reasoning framework based on a large language model to enable mutual verification and supplementation of these diverse and structurally varied data. Experiments show that in complex scenarios such as low resolution, partial occlusion, and unconventional aircraft models, the false detection rate of this application can be reduced by more than 40% compared to pure visual detection models, significantly improving the reliability of the detection results.

[0092] The method provided in this application addresses the limitation of traditional models in related technologies, which only output a single pattern of bounding boxes, type, and confidence level. The method generates a structured recognition report containing the complete reasoning process. This report includes at least the finally identified moving object type and its confidence level, the item-by-item comparison results of attribute consistency verification, the judgment basis for region rule attribution verification, the conclusions of external application verification, and potential anomaly markers and manual review suggestions. Users can directly read the reasoning chain in the report, clearly understanding the specific basis for the system's judgments, thereby significantly increasing trust in automated detection results. In high-reliability scenarios such as aviation supervision, this effectively reduces the workload of manual review and improves overall work efficiency.

[0093] In summary, the method provided in this application solves the core problems that have long existed in related technologies, such as the lack of domain knowledge integration, weak reasoning ability, single information dimension, and poor interpretability of results, through four major technical means: multi-dimensional knowledge base, large language model intelligent agent reasoning, multi-source information fusion, and structured report generation.

[0094] According to a second aspect of this application, this application also proposes a device for recognizing moving objects based on remote sensing images, which will be described below in conjunction with... Figure 4 The device is described in detail.

[0095] Figure 4 A schematic block diagram of a moving object recognition method apparatus based on remote sensing imagery according to an embodiment of this application is shown. Figure 4 As shown, the device 400 includes an image acquisition module 401, a prompt word construction module 402, a category recognition module 403, an information verification module 404, and a recognition module 405.

[0096] The image acquisition module 401 is used to acquire remote sensing images containing moving objects, and extract image data information and image spatiotemporal information associated with the moving objects from the remote sensing images respectively; in one embodiment, the image acquisition module 401 can also be used to perform the operation S10 described above, which will not be repeated here.

[0097] The prompt word construction module 402 is used to construct prompt words based on image data information and image spatiotemporal information; in one embodiment, the prompt word construction module 402 can also be used to perform the operation S20 described above, which will not be repeated here.

[0098] The category recognition module 403 is used to perform type recognition on image data information based on prompt words using a large language model to obtain the type of moving object; in one embodiment, the category recognition module 403 can also be used to perform the operation S30 described above, which will not be repeated here.

[0099] The information verification module 404 is used to verify the spatiotemporal information of the image based on the prompt words using an external application connected to the large language model, and to obtain a verification result that characterizes whether the spatiotemporal information of the image passes the verification. In one embodiment, the information verification module 404 can also be used to perform the operation S40 described above, which will not be repeated here.

[0100] The identification module 405 is used to perform logical reasoning analysis on the verification results based on the type of the moving object, and generate the identification results of the moving object in the remote sensing image. In one embodiment, the identification module 405 can also be used to perform the operation S50 described above, which will not be repeated here.

[0101] According to a third aspect of this application, this application also proposes an electronic device 500. Figure 5 A block diagram of an electronic device for a moving object recognition method based on remote sensing images according to an embodiment of this application is shown schematically. Figure 5 The electronic device 500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0102] like Figure 5 As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory configured for caching purposes. The processor 501 may include a single processing unit or multiple processing units configured to perform different actions of the method flow according to an embodiment of this application.

[0103] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0104] According to embodiments of this application, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0105] According to embodiments of this application, the method flow according to embodiments of this application can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code configured to perform the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0106] According to a fourth aspect of this application, a computer-readable storage medium is also provided. This computer-readable storage medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The aforementioned computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0107] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0108] For example, according to embodiments of this application, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0109] Embodiments of this application also include a computer program product comprising a computer program containing program code configured to perform the methods provided in the embodiments of this application. When the computer program product is run on an electronic device, the program code is configured to enable the electronic device to implement the method for rapid flight target association based on low-quality infrared remote sensing images provided in the embodiments of this application.

[0110] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0111] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0112] According to embodiments of this application, program code configured to execute the computer programs provided in embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0114] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the word "a" or "an" preceding an element does not exclude the presence of multiple such elements.

[0115] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0116] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for recognizing a moving object based on a remote sensing image, characterized in that, Includes the following steps: Acquire remote sensing images containing moving objects, and extract image data information and image spatiotemporal information associated with the moving objects from the remote sensing images respectively; Based on the image data information and the image spatiotemporal information, prompt words are constructed; The type of the moving object is obtained by using a large language model to identify the type of the image data information based on the prompt words; An external application connected to the large language model is used to verify the spatiotemporal information of the image based on the prompt words, and a verification result representing whether the spatiotemporal information of the image passes the verification is obtained. Logical reasoning analysis is performed on the verification results based on the type of the moving object to generate the identification result of the moving object in the remote sensing image.

2. The method according to claim 1, characterized in that, The image data information includes at least one of the following: the bounding box position information of the moving object, the initial type identification information of the moving object, and the confidence information of the moving object; The spatiotemporal information of the image includes at least one of the following: the geographical location information of the remote sensing image and the timestamp information of the remote sensing image.

3. The method according to claim 1, characterized in that, The step of constructing prompt words based on the image data information and the image spatiotemporal information includes: Based on the image data information, obtain the standard attribute features of the moving object in the remote sensing image; Based on the shooting geographical location information in the image spatiotemporal information, obtain the historical object type corresponding to the geographical location presented in the shooting geographical location information; The standard attribute features and the historically occurring object types are used as the prompt words.

4. The method according to claim 3, characterized in that, The method of using a large language model to identify the type of the image data information based on the prompt words to obtain the type of the moving object includes: Based on the standard attribute features and the historical object types, the image data information is compared with the pre-set knowledge base in the large language model to obtain the retrieval information that matches the image data information in the knowledge base. The knowledge base includes at least one of vector database, relational database and time series database. Based on the search results, the consistency between the standard attribute features and the historically occurring object types is determined, and the type of the motion object is generated based on the determined consistency.

5. The method according to any one of claims 1-4, characterized in that, The external application communicating with the large language model verifies the spatiotemporal information of the image based on the prompt words to obtain a verification result characterizing whether the spatiotemporal information of the image passes the verification, including: The prompt words are analyzed to identify the triggering conditions that require calling the external application; The external application's interface is invoked according to the triggering condition, and the real-time information of the shooting geographical location presented in the image spatiotemporal information is obtained through the interface. The external application's interface includes at least one of an air traffic control data interface, a meteorological data interface, and a historical record interface. The consistency between the real-time information and the image spatiotemporal information is determined, and a verification result representing whether the image spatiotemporal information passes the test is generated based on the determined consistency.

6. The method according to any one of claims 1-4, characterized in that, The step of performing logical reasoning analysis on the verification result based on the type of the moving object to generate the identification result of the moving object in the remote sensing image includes: Compare the type of the moving object with the verification result; Determine whether the verification result supports the appearance of the type of the moving object in the geographic location and timestamp corresponding to the spatiotemporal information of the image; If the verification result supports the existence of the geographic location and timestamp corresponding to the spatiotemporal information of the image, the type of the moving object shall be used as the identification result.

7. The method according to claim 6, characterized in that, The step of performing logical reasoning analysis on the verification result based on the type of the moving object to generate the identification result of the moving object in the remote sensing image further includes: If the verification result does not support the appearance of the geographic location and timestamp corresponding to the spatiotemporal information of the image, then the type of the moving object is determined to be incorrectly identified. The large language model is used to correlate and compare the real-time information in the verification result with the image data information to generate the corrected type of the moving object, and the corrected type of the moving object is used as the recognition result.

8. A moving object recognition device based on remote sensing imagery, characterized in that, include: The image acquisition module is used to acquire remote sensing images containing moving objects, and to extract image data information and image spatiotemporal information associated with the moving objects from the remote sensing images. The prompt word construction module is used to construct prompt words based on the image data information and the image spatiotemporal information; The category recognition module is used to perform type recognition on the image data information based on the prompt words using a large language model to obtain the type of the moving object; The information verification module is used to verify the spatiotemporal information of the image based on the prompt words using an external application connected to the large language model, and to obtain a verification result characterizing whether the spatiotemporal information of the image passes the verification. The identification module is used to perform logical reasoning analysis on the verification result based on the type of the moving object, and generate the identification result of the moving object in the remote sensing image.

9. An electronic device, characterized in that, include: One or more processors; as well as Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program or instructions thereon, which, when executed by a processor, implement the method according to any one of claims 1-7.

Images