A method and system for monitoring image open vocabulary target world coordinate solving

Abstract

The application provides a kind of monitoring image open vocabulary target world coordinate solution method and system, method includes: receiving monitoring image and open vocabulary target category prompt word, identify target instance in monitoring image, and output the boundary box of each target instance, instance segmentation mask, target center pixel coordinates, target ground point pixel coordinates and confidence; Monitoring image is input to monocular depth estimation model, and pixel level measurement depth map is output, and depth map is projected as three-dimensional coordinates under camera coordinate system based on camera internal parameter;According to instance segmentation mask, the pixel region in the bottom of the target set range is extracted, and the center coordinates in the pixel region are calculated as the ground anchor point of target instance;According to camera external parameter matrix, the three-dimensional coordinates corresponding to ground anchor point are converted into internal metric projection world coordinates, and are converted into external output geodetic coordinates.The application reduces the deployment cost while having stronger universality, and improves the coordinate solution accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for calculating the world coordinates of open vocabulary targets in surveillance images. Background Technology

[0002] Determining the world coordinates of targets in surveillance images typically involves three technical steps: target detection / segmentation, 3D geometric reconstruction, and transformation from pixel coordinates to world coordinates. Existing technologies can be broadly categorized into two types: one is target localization schemes based on traditional target detection, camera calibration, and homography / extrinsic parameter transformation; the other is open-vocabulary detection, segmentation, or 3D perception schemes based on visual fundamental models.

[0003] However, this type of solution generally suffers from the following shortcomings: First, target recognition usually relies on closed-category detectors, and the detection categories are pre-defined by the training set, making it difficult to achieve open-vocabulary target discovery based on natural language prompts; Second, some solutions rely on multi-camera collaboration or dedicated depth cameras, resulting in high deployment costs and making them unsuitable for direct reuse of a large number of existing ordinary RGB surveillance cameras; Third, existing solutions mostly use the center of the detection box, the midpoint of the bottom edge, or the ground homography projection point as the target landing point, lacking fine-grained semantic constraints on the actual grounding area of ​​the target, which can easily lead to a systematic deviation between the world coordinate landing point and the actual location of the target; Fourth, although some solutions obtain local world coordinates, they do not form a complete engineering conversion link from image coordinates, camera coordinates, local projection coordinates to geographic coordinates such as WGS84.

[0004] The second type of solution typically focuses on open-vocabulary 3D detection, 3D annotation, or 3D scene understanding. Its 3D geometry sources mostly rely on LiDAR, point clouds, multi-view data, or specially constructed 3D datasets. It primarily serves autonomous driving, 3D annotation, and 3D understanding tasks, rather than targeting fixed surveillance camera scenarios by directly calculating the target's world coordinates in real geographic space from a single frame of ordinary RGB surveillance image. Even with the introduction of visual foundation models and monocular depth estimation, existing publicly available solutions rarely address the following issues: how to stably correspond the open-vocabulary instance segmentation results with the monocular metric geometry results; how to determine pixel anchor points that better match the target's actual grounding location; and how to further transform the 3D points corresponding to these anchor points into a unified geographic coordinate system to directly serve digital twin, GIS (Geographic Information System) platforms, or urban monitoring operations.

[0005] Therefore, there is an urgent need for a method for calculating the target world coordinates of surveillance images using an open vocabulary that can reduce deployment costs, adapt to existing surveillance networks, and improve the accuracy of coordinate calculation. Summary of the Invention

[0006] Therefore, it is necessary to provide a method and system for calculating the world coordinates of open vocabulary targets in surveillance images to address the aforementioned technical problems.

[0007] A method for calculating target world coordinates in surveillance images using an open vocabulary includes the following steps: receiving a surveillance image and target category hints from an open vocabulary; identifying target instances in the surveillance image based on the target category hints; and outputting the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score for each target instance; inputting the surveillance image into a monocular depth estimation model to output a pixel-level depth map; back-projecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters; extracting pixel regions within a set range at the bottom of the target based on the instance segmentation mask; calculating the center coordinates within the pixel regions as ground anchor points for the target instances; and converting the three-dimensional coordinates corresponding to the ground anchor points into internal metric projected world coordinates based on the camera extrinsic matrix, and then converting them into geodetic coordinates for external output.

[0008] In one embodiment, the step of backprojecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters includes: Z c (u,v)=D(u,v) ; X c (u,v)=[(u-c x ) / f x ]*Z c ; Y c (u,v)= [(v-c y ) / f y ]* Z c (u,v) ; In the formula, (u,v) For pixels, D(u,v) The depth of a pixel in the depth map is ( X c ,Y c ,Z c ( ) represents three-dimensional coordinates. c x 、c y As the camera's main point, f x 、f y This refers to the camera's focal length.

[0009] In one embodiment, the set range is 10%.

[0010] In one embodiment, the step of extracting a pixel region within a predetermined range at the bottom of the target based on the instance segmentation mask, and calculating the center coordinates within the pixel region as ground anchor points of the target instance, includes: ; ; In the formula, M(u,v) Represents the pixels obtained from segmentation (u,v) The corresponding mask value, B Represents pixels that meet specific conditions (u,v) gather, For ground anchor points.

[0011] In one embodiment, the step of converting the three-dimensional coordinates corresponding to the ground anchor point into internal metric projected world coordinates and then into externally output geodetic coordinates based on the camera extrinsic parameter matrix includes: ; In the formula, lon represents longitude, lat represents latitude, alt represents absolute elevation, and T is the camera extrinsic parameter matrix. x cam , y cam , z cam ,1) represents the three-dimensional coordinates of the ground anchor point in the camera coordinate system.

[0012] A system for calculating the target world coordinates of an open vocabulary in a surveillance image is provided to implement the aforementioned method for calculating the target world coordinates of an open vocabulary in a surveillance image. The system includes: a target instance recognition module, used to receive a surveillance image and target category prompts from an open vocabulary, identify target instances in the surveillance image based on the target category prompts, and output the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score for each target instance; a depth estimation module, used to input the surveillance image into a monocular depth estimation model, output a pixel-level depth map, and back-project the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters; a pixel region extraction module, used to extract pixel regions within a set range at the bottom of the target based on the instance segmentation mask, and calculate the center coordinates within the pixel regions as ground anchor points for the target instances; and a coordinate calculation module, used to convert the three-dimensional coordinates corresponding to the ground anchor points into internal metric projected world coordinates based on the camera extrinsic parameter matrix, and then convert them into geodetic coordinates for external output.

[0013] Compared to existing technologies, the advantages and beneficial effects of this invention are as follows: By receiving surveillance images and target category prompts from open vocabulary, it identifies target instances in the surveillance images and outputs the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score for each target instance. This avoids the solidification of target categories, has strong generalization ability, and improves adaptability to new targets. The surveillance images are input into a monocular depth estimation model, which outputs a pixel-level depth map. Based on camera intrinsic parameters, the depth map is back-projected into three-dimensional coordinates in the camera coordinate system. The pixel region within a set range at the bottom of the target is extracted according to the instance segmentation mask, and the center coordinates within this pixel region are calculated as ground anchor points for the target instance. This effectively reduces the systematic deviation of the target and improves the accuracy of the target instance coordinate position. Based on the camera extrinsic parameter matrix, the three-dimensional coordinates corresponding to the ground anchor points are converted into internal metric projected world coordinates and then into externally output geodetic coordinates. This reduces deployment costs while having stronger versatility, adapting to existing urban surveillance networks, and improving coordinate calculation accuracy. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a method for calculating the target world coordinates of an open vocabulary in a surveillance image, as shown in one embodiment.

[0015] Figure 2 This is a schematic diagram of the structure of a target world coordinate solution system for open vocabulary in a surveillance image, as shown in one embodiment. Detailed Implementation

[0016] Before describing the specific embodiments of the present invention, the overall concept of the present invention will be explained as follows: This invention is mainly based on the process of monitoring image processing. Currently, the coordinate calculation of monitoring images suffers from problems such as high deployment cost and low coordinate calculation accuracy.

[0017] Therefore, this invention proposes a method for calculating the world coordinates of targets in surveillance images using an open vocabulary. By receiving surveillance images and target category hints from an open vocabulary, the method identifies target instances in the surveillance images and outputs the bounding box, instance segmentation mask, target center pixel coordinates, target ground point pixel coordinates, and confidence score for each target instance. This avoids the fixation of target categories, exhibits strong generalization ability, and improves adaptability to new targets. The surveillance image is input into a monocular depth estimation model, outputting a pixel-level depth map. Based on camera intrinsic parameters, the depth map is back-projected into three-dimensional coordinates in the camera coordinate system. Pixel regions within a defined range at the bottom of the target are extracted based on the instance segmentation mask, and the center coordinates within this pixel region are calculated as ground anchor points for the target instances, effectively reducing systematic deviations of the targets and improving the accuracy of target instance coordinate positions. Based on the camera extrinsic parameter matrix, the three-dimensional coordinates corresponding to the ground anchor points are converted into internal metric projected world coordinates and then into externally output geodetic coordinates. This reduces deployment costs while possessing stronger versatility, adapting to existing urban surveillance networks, and improving coordinate calculation accuracy.

[0018] Having introduced the overall concept of the present invention, to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0019] In one embodiment, such as Figure 1 As shown, a method for calculating the world coordinates of open vocabulary targets in surveillance images is provided, including the following steps: Step S110: Receive the monitoring image and target category hint words from the open vocabulary, identify target instances in the monitoring image based on the target category hint words, and output the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score for each target instance.

[0020] Specifically, fixed urban surveillance cameras are typically ordinary RGB cameras. They can iterate through all cameras in the camera registry at set intervals (e.g., every ten minutes), retrieve the latest frames from their video streams, obtain the surveillance images, and upload them to object storage to obtain accessible URLs for subsequent processing. This step also allows for the management of local image caches, deleting the earliest retrieved images from the cache to avoid excessive memory consumption. Furthermore, if an anomaly is detected in the acquired images, such as a green screen or near-solid colors, the surveillance images for the corresponding camera are re-acquired. If the number of consecutive abnormal images exceeds an anomaly threshold, a camera malfunction is detected, and image acquisition for that camera is stopped.

[0021] It utilizes the open-vocabulary instance segmentation capability of a visual base model, receives target category prompts such as vehicles, personnel, and construction machinery, and simultaneously detects and segments all instances in the acquired monitoring images that match the target category prompts. The output includes the bounding box, instance segmentation mask, target center pixel coordinates, target ground anchor pixel coordinates, and confidence score for each target instance. It can detect any target outside the training set categories using natural language text prompts without retraining or fine-tuning the model for new categories. It also supports fine segmentation of complex textures and maintains consistent identity tracking across frames.

[0022] Step S120: Input the monitoring image into the monocular depth estimation model, output a pixel-level depth map, and back-project the depth map into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameters.

[0023] Specifically, the acquired surveillance images are input into a monocular depth estimation model, such as Depth Anything, to perform pseudo-depth labeling on target instances in the surveillance images and output a pixel-level depth map, improving the efficiency and accuracy of pseudo-depth labeling of target instances and exhibiting high robustness even in complex scenes. Based on the camera's intrinsic parameters, the obtained depth map is back-projected into a 3D point cloud in the camera coordinate system, where the camera intrinsic parameters include the camera focal length and the camera principal point, obtaining the 3D coordinates of each pixel in the surveillance image in the camera coordinate system, thereby obtaining the pixel points and depths of all target instances, facilitating subsequent coordinate calculation.

[0024] The formula for obtaining the three-dimensional coordinates through back projection is: Z c (u,v)=D(u,v) ; X c (u,v)=[(u-c x ) / f x ]*Z c ; Y c (u,v)= [(v-c y ) / f y ]* Z c (u,v) ; In the formula, (u,v) For pixels, D(u,v) The depth of a pixel in the depth map is ( X c ,Y c ,Zc ( ) represents three-dimensional coordinates. c x 、c y As the camera's main point, f x 、f y This refers to the camera's focal length.

[0025] Step S130: Extract the pixel region within a set range at the bottom of the target based on the instance segmentation mask, and calculate the center coordinates within the pixel region as the ground anchor point of the target instance.

[0026] Specifically, since the bottommost pixel cluster of the mask is closest to the actual contact point when the object is standing upright on the ground, it can be used as a reference point for depth calibration coefficients. Therefore, based on the obtained instance segmentation mask, a horizontal strip of pixels within a set range at the bottom of the target instance is extracted. The set range can be set to 10%, and this set range can be adjusted according to the shape of the object. The center coordinates of the extracted pixel area are calculated as the ground anchor point of the target instance, which can more accurately reflect the actual landing position of targets such as vehicles, pedestrians, and construction equipment. This effectively reduces the systematic deviation of the target's world coordinates in terms of planar position and elevation, and improves the semantic consistency and spatial authenticity of the positioning results.

[0027] Among them, the ground anchor point represents the approximate pixel position of the area where the target contacts the ground. For example, for a vehicle, the ground anchor point is located in the area near the wheel / chassis; for a pedestrian, the ground anchor point is located in the foot area; and for construction machinery, the ground anchor point is located in the area where the track / tire touches the ground.

[0028] If the proportion of the extracted bottom pixel region is lower than the threshold, such as 5%, or the depth value is outlier, the anchor point is discarded to avoid error propagation.

[0029] Of course, for non-standing objects, the geometric center or the lowest convex hull vertex can be used as the depth anchor point of the target to accommodate diverse poses.

[0030] The formula for calculating the center coordinates is as follows: ; ; In the formula, M(u,v) Represents the pixels obtained from segmentation (u,v) The corresponding mask value, B Represents pixels that meet specific conditions (u,v) gather, For ground anchor points.

[0031] Step S140: Based on the camera extrinsic parameter matrix, convert the three-dimensional coordinates corresponding to the ground anchor points into internal metric projection world coordinates, and then into geodetic coordinates for external output.

[0032] Specifically, in order to determine the position and orientation of the camera in three-dimensional space, the three-dimensional coordinates corresponding to the anchor points of the seedlings are converted into internal metric projected world coordinates based on the camera's extrinsic parameter matrix. Furthermore, a coordinate reference system transformation operator is used to convert them into WGS84 geodetic coordinates for external output, thereby achieving high-precision coordinate calculation of the target instance. These geodetic coordinates can be directly connected to digital twin platforms, geographic information systems, and spatial analysis systems for overlay display, calculation of distances between targets, spatial analysis, and risk warning. They are suitable for application scenarios that directly obtain the spatial location of targets from surveillance videos.

[0033] Through the above steps, a clear hierarchical transformation system was constructed, from the original image pixel coordinates, camera 3D coordinates, internal metric projection world coordinates to external WGS84 geodetic coordinates. This system ensures that the coordinate semantics of each layer are uniquely determined, the errors of each layer can be traced independently, and the internal and external coordinate standards are strictly consistent throughout the entire process.

[0034] In addition, while outputting the target geodetic coordinates, the intermediate solution products and visualization verification results are retained, which facilitates the review and traceability of processes such as target detection, anchor point extraction, geometric sampling, coordinate transformation and distance calculation between targets. This is conducive to forming a credible result chain for urban governance, risk warning and digital twin display, and improves the verifiability and business usability of the results.

[0035] The formula for converting to geodetic coordinates is: ; In the formula, lon represents longitude, lat represents latitude, alt represents absolute elevation, and T is the camera extrinsic parameter matrix. x cam , y cam , z cam ,1) represents the three-dimensional coordinates of the ground anchor point in the camera coordinate system.

[0036] In this embodiment, by receiving surveillance images and target category prompts from open vocabulary, target instances in the surveillance images are identified, and the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score of each target instance are output. This avoids the solidification of target categories, has strong generalization ability, and improves the adaptability to new targets. The surveillance images are input into a monocular depth estimation model, which outputs a pixel-level depth map. Based on camera intrinsic parameters, the depth map is back-projected into three-dimensional coordinates in the camera coordinate system. The pixel region within a set range at the bottom of the target is extracted according to the instance segmentation mask, and the center coordinates of the pixel region are calculated as the ground anchor point of the target instance, effectively reducing the systematic deviation of the target and improving the accuracy of the target instance coordinate position. According to the camera extrinsic parameter matrix, the three-dimensional coordinates corresponding to the ground anchor point are converted into internal metric projected world coordinates and then into externally output geodetic coordinates. This reduces deployment costs while having stronger versatility, adapting to existing urban surveillance networks, and improving coordinate calculation accuracy.

[0037] In summary, this invention achieves an integrated closed loop of open vocabulary target recognition, monocular 3D geometric calculation, and world coordinate consensus expression. It has stronger versatility, lower deployment cost, higher positioning semantic consistency, and better engineering practical value. It is particularly suitable for application scenarios that require direct acquisition of target spatial location from existing surveillance videos, such as urban lifelines, smart parks, traffic monitoring, and digital twins.

[0038] like Figure 2 As shown, a surveillance image open vocabulary target world coordinate calculation system 20 is provided to implement the surveillance image open vocabulary target world coordinate calculation method described above. It includes: a target instance recognition module 21, a metric depth estimation module 22, a pixel region extraction module 23, and a coordinate calculation module 24, wherein: The target instance recognition module 21 is used to receive the monitoring image and the target category prompt words of the open vocabulary, identify the target instance in the monitoring image according to the target category prompt words, and output the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates and confidence score of each target instance; The depth estimation module 22 is used to input the monitoring image into the monocular depth estimation model, output a pixel-level depth map, and back-project the depth map into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameters. The pixel region extraction module 23 is used to extract the pixel region within a set range at the bottom of the target based on the instance segmentation mask, and calculate the center coordinates within the pixel region as the ground anchor point of the target instance; The coordinate calculation module 24 is used to convert the three-dimensional coordinates corresponding to the ground anchor points into internal metric projected world coordinates based on the camera extrinsic matrix, and then into geodetic coordinates for external output.

[0039] In this embodiment, through the target instance identification module 21, the depth estimation module 22, the pixel region extraction module 23, and the coordinate calculation module 24, it is possible to automatically discover any text-descriptive target from a single frame of ordinary RGB monitoring image without relying on a dedicated depth sensor, multi-sensor fusion, or multi-view collaboration, accurately locate and calculate its three-dimensional coordinates in the world geographic coordinate system.

[0040] In summary, compared with the prior art, the present invention has at least the following beneficial effects: First, this invention can achieve automatic discovery, fine segmentation and world coordinate calculation of open vocabulary targets by relying only on a single frame of ordinary RGB monitoring image, without the need for additional configuration of LiDAR, dedicated depth camera or multi-camera collaborative acquisition system. Therefore, it significantly reduces the hardware deployment threshold and transformation cost, and is more suitable for direct promotion and application in existing urban monitoring networks.

[0041] Second, this invention organically combines the open-vocabulary semantic understanding capability of the visual basic model with the three-dimensional geometric calculation capability, enabling the system not only to identify preset category targets, but also to locate and calculate the coordinates of any describable target based on natural language prompts. This overcomes the problems of fixed categories and weak generalization ability of traditional closed set detection methods, and significantly improves the system's adaptability to long-tail targets, temporary targets and newly added target categories in complex urban scenes.

[0042] Third, this invention extracts anchor points from the bottom region of the target instance segmentation mask and uses the pixel position corresponding to the actual contact area between the target and the ground as the basis for calculating the three-dimensional landing point. Compared with the existing technology that uses the center point of the detection box, the midpoint of the bottom edge, or empirical proportions to estimate the landing point, this invention can more accurately reflect the real landing position of targets such as vehicles, pedestrians, and construction equipment. This effectively reduces the systematic deviation of the target's world coordinates in terms of planar position and elevation, and improves the semantic consistency and spatial authenticity of the positioning results.

[0043] Fourth, this invention constructs a hierarchical coordinate transformation system from image pixel coordinates, camera coordinates, internal metric projection world coordinates to external WGS84 geodetic coordinates, making the coordinate semantics of the input and output of each module clear and unified. This not only facilitates the direct integration of results into digital twin platforms, GIS platforms, and spatial analysis systems, but also enables rapid location of error sources when deviations occur in the results, improving the system's debuggability, maintainability, and interpretability.

[0044] Fifth, this invention can output the world coordinates of the target while retaining intermediate solution products and visual verification results, which facilitates the review and traceability of processes such as target detection, anchor point extraction, geometric sampling, coordinate transformation and distance calculation between targets. Therefore, it is conducive to forming a credible result chain for urban governance, risk warning and digital twin display, and improves the verifiability and business usability of the results.

[0045] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0046] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a computer storage medium (ROM / RAM, magnetic disk, optical disk) for execution by the computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, the present invention is not limited to any particular hardware and software combination.

[0047] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for calculating the world coordinates of an open vocabulary target in a surveillance image, characterized in that, Includes the following steps: Receive monitoring images and target category hints from open vocabulary, identify target instances in the monitoring images based on the target category hints, and output the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score for each target instance; The monitoring image is input into a monocular depth estimation model, and a pixel-level depth map is output. The depth map is then back-projected into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameters. The pixel region within a set range at the bottom of the target is extracted based on the instance segmentation mask, and the center coordinates within the pixel region are calculated as the ground anchor point of the target instance. Based on the camera extrinsic matrix, the three-dimensional coordinates corresponding to the ground anchor points are converted into internal metric projected world coordinates, and then into geodetic coordinates for external output.

2. The method for calculating the target world coordinates of an open vocabulary in a surveillance image according to claim 1, characterized in that, The step of back-projecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters includes: Z c (u,v)=D(u,v) ； X c (u,v)=[(uc x ) / f x ]*Z c ； Y c (u,v)= [(vc y ) / f y ]* Z c (u,v) ； In the formula, (u,v) For pixels, D(u,v) The depth of a pixel in the depth map is ( X c ,Y c ,Z c ( ) represents three-dimensional coordinates. c x 、c y As the camera's main point, f x 、f y This refers to the camera's focal length.

3. The method for calculating the world coordinates of an open vocabulary target in a surveillance image according to claim 1, characterized in that, The set range is 10%.

4. The method for calculating the world coordinates of an open vocabulary target in a surveillance image according to claim 3, characterized in that, The step of extracting a pixel region within a set range at the bottom of the target based on the instance segmentation mask, and calculating the center coordinates within the pixel region as the ground anchor point of the target instance, includes: ； ； In the formula, M(u,v) Represents the pixels obtained from segmentation (u,v) The corresponding mask value, B Represents pixels that meet specific conditions (u,v) gather, For ground anchor points.

5. The method for calculating the world coordinates of an open vocabulary target in a surveillance image according to claim 1, characterized in that, The step of converting the three-dimensional coordinates corresponding to the ground anchor points into internal metric projected world coordinates, and then into externally output geodetic coordinates, based on the camera extrinsic parameter matrix, includes: ； In the formula, lon represents longitude, lat represents latitude, alt represents absolute elevation, and T is the camera extrinsic parameter matrix. x cam , y cam , z cam ,1) represents the three-dimensional coordinates of the ground anchor point in the camera coordinate system.

6. A system for calculating the world coordinates of targets in an open vocabulary of surveillance images, characterized in that, A method for calculating the target world coordinates of an open vocabulary in a surveillance image as described in any one of claims 1-5, comprising: The target instance recognition module is used to receive monitoring images and target category prompts from open vocabulary, identify target instances in the monitoring images based on the target category prompts, and output the bounding box, instance segmentation mask, target center pixel coordinates, target ground plotting point pixel coordinates, and confidence score for each target instance. The depth estimation module is used to input the monitoring image into the monocular depth estimation model, output a pixel-level depth map, and back-project the depth map into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameters. The pixel region extraction module is used to extract the pixel region within a set range at the bottom of the target based on the instance segmentation mask, and calculate the center coordinates within the pixel region as the ground anchor point of the target instance; The coordinate calculation module is used to convert the three-dimensional coordinates corresponding to the ground anchor points into internal metric projected world coordinates based on the camera extrinsic parameter matrix, and then into geodetic coordinates for external output.

Images