Target tracking method, device and storage medium

Abstract

The application discloses a target tracking method, device and storage medium. The target tracking method comprises the following steps: acquiring position information of a target object, querying scene modeling data matched with the position information in a preset scene modeling database, and acquiring real-time images collected by an image collection device matched with the position information, so as to accurately call resources according to the position of the target object, improve subsequent modeling efficiency, then, the scene modeling data and the real-time images are fused to obtain more accurate and convenient observation real scene modeling results, and the target object is tracked based on the real scene modeling results, so that more image details and visual ranges are provided for security, tracking and other scenes on the premise that the target object is not lost, and comprehensive research and judgment on the motion scene of the target object are facilitated.

Description

Technical Field

[0001] This application relates to the field of security technology, and in particular to a target tracking method, device and storage medium. Background Technology

[0002] Target tracking refers to the detection, extraction, recognition, and tracking of moving targets in an image sequence, obtaining the motion parameters of the moving targets, processing and analyzing them, and realizing the understanding of the behavior of the moving targets in order to complete higher-level detection tasks.

[0003] Currently, target tracking services in related technologies still suffer from problems such as low accuracy in target positioning and inability to accurately display the target's trajectory, which affect the performance of target tracking services. Summary of the Invention

[0004] This application provides at least one target tracking method, device, and storage medium.

[0005] The first aspect of this application provides a target tracking method, including: acquiring the location information of a target object; querying scene modeling data that matches the location information in a preset scene modeling database, and acquiring real-time images acquired by an image acquisition device that matches the location information; fusing the scene modeling data and the real-time images to obtain a real-scene modeling result; and tracking the target object based on the real-scene modeling result.

[0006] In one embodiment, obtaining the location information of a target object includes: obtaining candidate location information of the target object, the candidate location information including candidate locations of the target object at at least two time points; calculating a buffer region corresponding to the candidate location of the target object at each time point; obtaining an overlapping region between the corresponding buffer regions at each time point; filtering the candidate locations in the candidate location information based on the overlapping region, and using the filtered candidate location information as the location information of the target object.

[0007] In one embodiment, calculating the buffer region corresponding to the candidate position of the target object at each time step includes: obtaining the velocity information of the target object at each candidate position; and calculating the buffer region corresponding to the target object at each candidate position based on the velocity information.

[0008] In one embodiment, a preset scene modeling database stores street modeling data; querying scene modeling data in the preset scene modeling database that matches location information includes: determining the street associated with the location information to obtain the associated street range; and using the street modeling data in the preset scene modeling database that is within the associated street range as scene modeling data.

[0009] In one embodiment, acquiring real-time images acquired by image acquisition devices that match location information includes: determining a device selection range corresponding to the location information, and acquiring the device coordinates of each image acquisition device; identifying image acquisition devices whose device coordinates are within the device selection range as image acquisition devices that match the location information; and acquiring real-time images acquired by image acquisition devices that match the location information.

[0010] In one embodiment, scene modeling data and real-time images are fused to obtain a real-scene modeling result, including: cropping the scene modeling data based on the location information of the target object to obtain an initial modeling result; and mapping the real-time image onto the initial modeling result to obtain the real-scene modeling result.

[0011] In one embodiment, mapping a real-time image to an initial modeling result to obtain a real-scene modeling result includes: performing keypoint detection on the real-time image and the initial modeling result respectively to obtain an image keypoint set corresponding to the real-time image and a modeling keypoint set corresponding to the initial modeling result; extracting associated keypoint pairs between the image keypoint set and the modeling keypoint set; and mapping the real-time image to the initial modeling result based on the associated keypoint pairs to obtain the real-scene modeling result.

[0012] In one embodiment, mapping a real-time image to an initial modeling result based on associated keypoint pairs to obtain a real-scene modeling result includes: obtaining the mapping point corresponding to each pixel in the real-time image in the initial modeling result based on the mapping relationship of each associated keypoint pair; and mapping the pixel value of each pixel in the real-time image to the mapping point in the initial modeling result to obtain the real-scene modeling result.

[0013] A second aspect of this application provides a target tracking device, comprising: a position acquisition module for acquiring position information of a target object; a modeling data acquisition module for querying scene modeling data matching the position information in a preset scene modeling database, and acquiring real-time images acquired by an image acquisition device matching the position information; a fusion module for fusing the scene modeling data and the real-time images to obtain a real-scene modeling result; and a tracking module for tracking the target object based on the real-scene modeling result.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the target tracking method described above.

[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the target tracking method described above.

[0016] The above scheme obtains the location information of the target object, queries the scene modeling data that matches the location information in the preset scene modeling database, and acquires real-time images captured by image acquisition devices that match the location information. This allows for accurate resource allocation based on the target object's location, improving subsequent modeling efficiency. Then, the scene modeling data and real-time images are fused to obtain a more accurate and easily observable real-scene modeling result. Based on the real-scene modeling result, the target object is tracked. While ensuring that the target object is not lost, this provides more image details and a wider field of view for security, tracking, and other scenarios, facilitating comprehensive research and judgment of the target object's movement scene.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a schematic diagram of the implementation environment involved in this application;

[0020] Figure 2 This is a flowchart illustrating a target tracking method in an exemplary embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a target tracking system shown in an exemplary embodiment of this application;

[0022] Figure 4 This is a schematic diagram illustrating a location acquisition service as shown in an exemplary embodiment of this application;

[0023] Figure 5 This is a flowchart illustrating an exemplary embodiment of the acquisition and analysis algorithm of this application;

[0024] Figure 6 This is a flowchart illustrating the analysis of the location information of a target object, as shown in an exemplary embodiment of this application;

[0025] Figure 7 This is a schematic diagram illustrating a fusion modeling service performing reality modeling, as shown in an exemplary embodiment of this application;

[0026] Figure 8 This is a block diagram illustrating a target tracking device in an exemplary embodiment of this application;

[0027] Figure 9 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application;

[0028] Figure 10 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0029] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0031] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment involved in the target tracking method of this application. For example... Figure 1 As shown, the implementation environment includes an image acquisition device 110, a server 120, a terminal 130, and a scene modeling database 140. The image acquisition device 110, the terminal 130, and the scene modeling database 140 are directly or indirectly connected to the server 120 via wired or wireless communication.

[0033] The image acquisition device 110 can be a monocular camera, a multi-view camera, a depth camera, an infrared camera, etc., and this application does not limit the type of image acquisition device 110. Image acquisition device 110 can generally refer to one of multiple image acquisition devices; this embodiment only uses image acquisition device 110 as an example. Those skilled in the art will understand that the number of the above-mentioned image acquisition devices can be more, such as dozens or hundreds, or even more. In this case, the implementation environment of the above-mentioned target tracking method also includes other image acquisition devices. This application embodiment does not limit the number of image acquisition devices.

[0034] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not limit the type of server 120.

[0035] Terminal 130 can be a mobile phone, computer, smart home appliance, vehicle terminal, aircraft, etc., and this application does not limit the type of terminal 130. Terminal 130 can refer to one of multiple terminals, and this embodiment only uses terminal 130 as an example. Those skilled in the art will know that the number of terminals can be more, such as dozens or hundreds of terminals, or even more. In this case, the implementation environment of the target tracking method also includes other terminals. This application embodiment does not limit the number of terminals.

[0036] The scene modeling database 140 is used to store scene modeling data. The scene modeling database 140 can be set up independently of the server 120 or integrated with the server 120. This application does not limit this.

[0037] For example, terminal 130 sends a tracking request for a target object to server 120. Server 120 obtains the location information of the target object based on the tracking request, queries the scene modeling database 140 for scene modeling data that matches the location information, and obtains real-time images captured by image acquisition device 110 that match the location information. The scene modeling data and real-time images are fused to obtain a real-scene modeling result, which is then sent to terminal 130. Terminal 130 tracks and displays the target object based on the real-scene modeling result.

[0038] It is understood that in the specific implementation of this application, data such as scene modeling data and real-time images are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0039] Please see Figure 2 , Figure 2 This is a flowchart illustrating a target tracking method in an exemplary embodiment of this application. This target tracking method can be applied to… Figure 1The implementation environment shown is specifically executed by a server within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.

[0040] The target tracking method proposed in this application embodiment will be described in detail below, taking the server as the specific execution subject.

[0041] like Figure 2 As shown, the target tracking method includes at least steps S210 to S240, which are described in detail below:

[0042] Step S210: Obtain the location information of the target object.

[0043] The target objects include, but are not limited to, pedestrians and vehicles.

[0044] The location information of the target object is obtained based on the location acquisition device. The location acquisition device may be a device equipped with a Global Positioning System (GPS) or a device equipped with sensing devices such as distance sensors and image sensors. This application does not limit the specific device to this type.

[0045] Step S220: Query the scene modeling data that matches the location information in the preset scene modeling database, and obtain the real-time image acquired by the image acquisition device that matches the location information.

[0046] The scene modeling database is used to store scene modeling data, which can be 3D modeling data. This scene modeling data is used to represent the spatial structure of the scene.

[0047] Image acquisition devices are used to acquire images, which can be infrared images, RGB (red, green, blue) images, depth images, etc. The type of image is determined by the image acquisition method of the image acquisition device.

[0048] Query scene modeling data that matches the location information in the scene modeling database, and obtain real-time images captured by image acquisition devices that match the location information.

[0049] If the location information of the target object indicates that the target object is within the scene modeling range of the scene modeling database, then the scene modeling data near the target object in the scene modeling database can be obtained based on the location information; if the location information of the target object indicates that the target object is within the image acquisition range of the image acquisition device, then the image acquisition device near the target object can be determined based on the location information, so as to obtain the real-time images acquired by these devices.

[0050] Step S230: Fuse the scene modeling data and real-time images to obtain the real-scene modeling result.

[0051] Although scene modeling data contains the spatial structure of the scene, it cannot reflect the actual scene at the current moment. Therefore, by fusing real-time images and scene modeling data, we can obtain a real-scene modeling result that contains the actual scene at the current moment. This facilitates the display of scene images for subsequent tracking of target objects and improves the display effect.

[0052] For example, features such as image texture, image color, and spatial relationship of real-time images are extracted, and scene spatial features of scene modeling data are extracted. Based on the scene spatial features of scene modeling data and the spatial relationship features of real-time images, the scene fusion region of each real-time image in the scene modeling data is determined, and the image texture, image color, and other features of real-time images are fused into the scene fusion region in the scene modeling data to obtain the real-scene modeling result.

[0053] Step S240: Track the target object based on the results of the real-scene modeling.

[0054] The target objects are marked in the reality modeling results so that the marked reality modeling results can be displayed.

[0055] It is understandable that when the target object is in a moving state, the target object's position information is updated in real time. In response to the update of the position information, the target object is re-marked in the reality modeling results based on the updated position information.

[0056] Additionally, real-scene modeling can be repeated based on the movement information of the target object. For example, if the target object is detected at position p1 at time T1, scene modeling data and real-time images are acquired based on position p1 to construct a real-scene modeling result, and the target object's position p1 is marked in the real-scene modeling result; if the target object is detected at time T2 to move from position p1 to p2, and p2 is not within the preset range of the real-scene modeling result, then scene modeling data and real-time images are acquired again for position p2 to construct a real-scene modeling result, and the target object's position p2 is marked in the real-scene modeling result.

[0057] The above technical solution can obtain more realistic and detailed scene information of the target object while ensuring the location and tracking of the target object, thereby improving the tracking effect.

[0058] The following section provides a further explanation of each step in the target tracking method of this application.

[0059] For example, please refer to Figure 3 , Figure 3The target tracking system shown in this application is an exemplary embodiment. The target tracking system can be deployed on a server and includes a location acquisition service, a fusion modeling service, and a tracking display service.

[0060] Location acquisition services can acquire the location information of a target object based on at least one of GPS devices, monitoring devices, etc. For example, combining GPS devices and monitoring devices to acquire the location information of a target object ensures the reliability of the acquisition. Within the video coverage area of ​​the monitoring device, the location of the target object can be identified through the acquired video stream data containing the target object; in areas not covered by the monitoring device, the GPS device carried by the target object performs GPS positioning to prevent the target object from being lost.

[0061] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating a location acquisition service as an exemplary embodiment of this application, such as... Figure 4 As shown, the location acquisition service includes at least one of the following components: a data access module, a data analysis module, a real-time data push module, and a data persistence module.

[0062] The data access module is implemented based on a distributed cluster to collect location data for various objects. This distributed cluster includes a master node and child nodes. The master node manages the child nodes, which communicate with the location acquisition devices to receive location data uploaded by these devices. For example, if the location acquisition devices include monitoring equipment, the uploaded data includes the device's location and video stream data containing the object; if the location acquisition devices include GPS devices carried by the object, the uploaded data includes GPS positioning data. Each child node can be configured with different location data conversion plugins to process different types of location data, converting it into a unified location data format. Then, in response to a tracking request for a target object, the master node filters out the target object's location data based on the location data converted by the child nodes.

[0063] It is understandable that the location data received by the data access module from various location acquisition devices may be inaccurate, and the location data collected by these location acquisition devices needs to be analyzed as candidate location information.

[0064] The data access module sends the location data of the target object to the message middleware so that other modules can process the location data.

[0065] The data analysis module is configured with analysis algorithms used to analyze location data collected by various location acquisition devices. These algorithms can be default algorithms or user-configured algorithms, such as... Figure 5 As shown, the user-side terminal initiates an analysis algorithm configuration request to the server to determine the algorithm template. The algorithm template can be a custom algorithm template or a default template. Then, it checks if a custom algorithm step event exists. If so, a new custom algorithm step is added to the algorithm step library, meaning the library includes both default and custom algorithm steps. Next, it obtains the algorithm step sorting specification sent by the user-side terminal and sorts the specified default or custom algorithm steps in the library to populate the algorithm template with algorithm steps. The populated algorithm template is then saved, resulting in the configured analysis algorithm. Custom analysis algorithms improve the flexibility of target tracking.

[0066] For example, the target positioning data analysis module in the data analysis module analyzes the location data collected by each location acquisition device according to the configured analysis algorithm to obtain the location information of the target object, and the steps may include steps S211 to S214:

[0067] Step S211: Obtain candidate location information of the target object, which includes the candidate locations of the target object at at least two time points.

[0068] Among them, at least two moments can be adjacent moments.

[0069] For example, the location of the target object uploaded by the data access module at each time moment is saved. In response to the location of the target object received at the current time moment, the location of the target object corresponding to the saved historical time moment is queried. The location of the target object at the current time moment and the location of the target object at the historical time moment are both used as candidate locations to obtain candidate location information.

[0070] Step S212: Calculate the buffer region corresponding to the candidate position of the target object at each time step.

[0071] The size of the buffer zone can be determined based on the target object's speed information, the type of the target object, etc.

[0072] For example, calculating the buffer region corresponding to the candidate position of the target object at each time step includes: obtaining the velocity information of the target object at each candidate position; and calculating the buffer region corresponding to the target object at each candidate position based on the velocity information.

[0073] Speed ​​information can be obtained based on the positional changes of a target object within a preset time period, or it can be obtained by deploying speed detection devices around the target object and detecting the target object using these devices. For example, by acquiring video stream data containing the target object uploaded by a monitoring device, analyzing the video stream data, the spatial position of the target object can be obtained, and then based on the changes in the target object's spatial position, speed information, direction information, and so on can be obtained.

[0074] The buffer region corresponding to each candidate position of the target object is calculated based on the velocity information. It can be understood that the faster the velocity information represents the speed of the target object, the larger the corresponding buffer region is, and the slower the velocity information represents the speed of the target object, the smaller the corresponding buffer region is.

[0075] Step S213: Obtain the overlapping area between the corresponding buffer areas at each time step.

[0076] Map the corresponding buffer region at each time step to the same coordinate system to obtain the overlapping area between the corresponding buffer regions at each time step.

[0077] Step S214: Filter the candidate locations in the candidate location information based on the overlapping region, and use the filtered candidate location information as the location information of the target object.

[0078] The overlapping area between each buffer area can reflect the accuracy of the candidate position corresponding to the target object. Based on the overlapping area between the buffer areas, abnormal positions in the candidate positions are filtered out, normal positions are retained, and normal positions are used as the position information of the target object, thereby improving the accuracy of target object position acquisition.

[0079] For example, if the area of ​​the overlapping region between the current buffer zone and other buffer zones is less than a preset area threshold, then the candidate position at the current moment is considered an abnormal position; if the area of ​​the overlapping region between the current buffer zone and other buffer zones is not less than the preset area threshold, then the candidate position at the current moment is considered a normal position. Each candidate position is traversed to delete abnormal positions in the candidate position information, and the traversed candidate position information is used as the position information of the target object.

[0080] For example, please refer to Figure 6 , Figure 6 A flowchart illustrating the analysis of the location information of a target object, as shown in an exemplary embodiment of this application, is as follows: Figure 6As shown, the candidate location information includes GPS positioning data uploaded by GPS devices and monitoring positioning data obtained based on video stream data uploaded by monitoring devices. Both GPS positioning data and monitoring positioning data represent the latitude and longitude of the target object's location. The location information analysis process may include: determining the correctness of the latitude and longitude of the GPS positioning data and monitoring positioning data, such as determining whether the GPS positioning data and monitoring positioning data are within the normal longitude and latitude ranges. Generally, the normal longitude range is 0°-180°, and the normal latitude range is 0°-90°. If they are not within the normal longitude and latitude ranges, the GPS positioning data or monitoring positioning data is directly marked as having data anomalies. If they are within the normal longitude and latitude ranges, the correctness of the monitoring positioning data is further checked. Referring to steps S211 to S214, the speed information of the target object is obtained. Based on the speed information, the buffer area corresponding to the target object at adjacent time points in the monitoring and positioning data is obtained. It is determined whether there is an overlapping area between the buffer areas corresponding to adjacent time points. If there is, the monitoring and positioning position corresponding to the adjacent time point in the monitoring and positioning data is considered normal; if not, the monitoring and positioning position corresponding to the adjacent time point in the monitoring and positioning data is considered abnormal. For monitoring and positioning positions with abnormalities, the location data of the target object can be supplemented based on GPS positioning data at the same time point. Then, all normal monitoring and positioning data and GPS positioning data are pushed to the message middleware as the final location information of the target object so that other modules can process the location information.

[0081] The data persistence module stores location data collected by various location acquisition devices and target object location information uploaded by the data access module in the message middleware into the database. It can also provide data query services to users to query the data in the database, such as querying the trajectory of the target object.

[0082] The real-time data push module receives the location information of the target object from the message middleware and pushes the location information of the target object to the fusion modeling service.

[0083] The fusion modeling service performs real-scene modeling based on the location information of the target object, and obtains the real-scene modeling result.

[0084] For example, please refer to Figure 7 , Figure 7 A schematic diagram illustrating reality modeling using a fusion modeling service as shown in an exemplary embodiment of this application, such as... Figure 7 As shown, the fusion modeling service includes a location information receiving module, a scene modeling data acquisition module, a real-time image acquisition module, and a fusion module.

[0085] The location information receiving module is used to receive the location information of the target object in the message middleware and send the location information to the scene modeling data acquisition module and the real-time image acquisition module.

[0086] The scene modeling data acquisition module is connected to a preset scene modeling database, which stores street modeling data. The scene modeling data acquisition module queries the preset scene modeling database for scene modeling data that matches the location information, including steps S2211 to S2212:

[0087] Step S2211: Determine the streets associated with the location information to obtain the range of associated streets.

[0088] You can first obtain the street range selection threshold corresponding to the target object, then obtain the street range selection reference point based on the location information, and then obtain the associated street range based on the street range selection reference point and the street range selection threshold.

[0089] The street range selection threshold is used to limit the size of the associated street range. This associated street range can be a pre-set fixed value or a value flexibly calculated according to the actual scenario. For example, the street range selection threshold can be flexibly calculated based on at least one of the following: the target object's speed information, the target object's type, or the target object's location information indicating the type of street where the target object is located, so that the obtained street range selection threshold is more consistent with the target object's current scenario.

[0090] The street range selection reference point is used to define the location of the associated street range, such as the center point of the associated street range. The street range selection reference point is obtained based on the location information of the target object. If the location information of the target object contains the location of the target object at a single moment, the coordinates of the location at that moment are directly used as the street range selection reference point. If the location information of the target object contains the location of the target object at multiple moments, the location with the highest real-time performance can be selected from the multiple locations, and the coordinates of the location corresponding to that location can be used as the street range selection reference point. Alternatively, the average coordinates of the multiple location coordinates at multiple moments can be calculated to obtain the street range selection reference point. This application does not impose any limitations on this method.

[0091] The associated street range is obtained by selecting a reference point and a street range selection threshold. For example, starting from the street range selection reference point, all streets within the street range selection threshold are considered as streets associated with the location information. The associated street range is obtained by combining all streets associated with the location information.

[0092] Step S2212: Use the street modeling data within the associated street range in the preset scene modeling database as scene modeling data.

[0093] Based on the associated street range, street modeling data is extracted from the preset scene modeling database to obtain scene modeling data.

[0094] The real-time image acquisition module is connected to the image acquisition device management module, which manages multiple image acquisition devices. Each image acquisition device has a different location and is used to acquire real-time images of different spatial scenes. The real-time image acquisition module acquires the real-time images acquired by the image acquisition devices that match the location information, including steps S2221 to S2223:

[0095] Step S2221: Determine the device selection range corresponding to the location information, and obtain the device coordinates of each image acquisition device.

[0096] The device coordinates of the image acquisition device are used to reflect the device location corresponding to the image acquisition device. If the image acquisition device is a mobile device, then it is necessary to obtain the device coordinates uploaded by the image acquisition device in real time; if the image acquisition device is also a non-mobile device, then the device coordinates uploaded by the image acquisition device can be obtained in advance and stored, and then the device coordinates of the image acquisition device can be obtained by querying the stored set of device coordinates.

[0097] The process of determining the range of equipment selection can be detected by the process of determining the range of associated streets in step S2211, which will not be elaborated here.

[0098] Step S2222: Select the image acquisition device whose coordinates are within the selected range as the image acquisition device that matches the location information.

[0099] Based on the device coordinates and device selection range of each image acquisition device, the image acquisition devices are filtered to obtain the image acquisition devices within the device selection range.

[0100] Step S2223: Acquire real-time images captured by an image acquisition device that matches the location information.

[0101] After obtaining the image acquisition device that matches the location information, or the real-time images acquired by these image acquisition devices.

[0102] Among them, the image acquisition device can be a panoramic camera deployed at high altitudes to expand the image acquisition range of the image acquisition device and improve the spatial coverage of real-time images.

[0103] After obtaining scene modeling data and real-time images based on the above steps, the scene modeling data and real-time images are pushed to the fusion module. The fusion module fuses the scene modeling data and real-time images to obtain the reality modeling result, including steps S231 to S232:

[0104] Step S231: Based on the location information of the target object, perform scene clipping on the scene modeling data to obtain the initial modeling result.

[0105] For example, the scene modeling data is three-dimensional modeling data. Scene clipping of the scene modeling data includes: identifying the target object in the scene modeling data based on the position information of the target object; determining the observation box for the target object in the scene modeling data based on the identified target object; clipping the scene modeling data based on the observation box to obtain an initial modeling structure, so as to keep the target object in a preset position in the observation box, such as keeping the target object in the center position of the observation box, so as to facilitate the subsequent observation of the target object tracking scene by the user.

[0106] The size of the observation box can be determined based on the target tracking observation request sent by the user, or based on the size of the target object. For example, the larger the target object, the larger the observation box; the smaller the target object, the smaller the observation box. The direction of the observation box can be determined based on the target tracking observation request sent by the user, or based on the direction of movement of the target object.

[0107] Step S232: Map the real-time image onto the initial modeling result to obtain the real-scene modeling result.

[0108] Real-time images are mapped onto the initial modeling results so that the resulting real-world modeling results can display real-time images for easy observation by users.

[0109] In some implementations, mapping real-time images to initial modeling results to obtain real-scene modeling results includes: performing keypoint detection on real-time images and initial modeling results respectively to obtain image keypoint sets corresponding to real-time images and modeling keypoint sets corresponding to initial modeling results; extracting associated keypoint pairs between image keypoint sets and modeling keypoint sets; and mapping real-time images to initial modeling results based on associated keypoint pairs to obtain real-scene modeling results.

[0110] Among them, the associated key points are used to reflect the one-to-one mapping relationship between the key points of the real-time image and the key points of the initial modeling result.

[0111] If the keypoints in the real-time image and the keypoints in the initial modeling result both refer to the same location, it indicates that there is a mapping relationship between these two keypoints. Extracting all keypoint pairs with mapping relationships between the image keypoint set and the modeling keypoint set yields the associated keypoint pairs.

[0112] Then, based on each associated key point pair, the real-time image is mapped to the initial modeling result to obtain the real-scene modeling result.

[0113] For example, mapping a real-time image to an initial modeling result based on associated keypoint pairs to obtain a real-scene modeling result includes: obtaining the mapping point corresponding to each pixel in the real-time image in the initial modeling result based on the mapping relationship of each associated keypoint pair; mapping the pixel value of each pixel in the real-time image to the mapping point in the initial modeling result to obtain a real-scene modeling result.

[0114] By mapping the key points of the real-time image to the key points of the initial modeling result, the mapping coordinates of each pixel in the real-time image relative to the initial modeling result are obtained. That is, the mapping point corresponding to each pixel in the real-time image in the initial modeling result is obtained. Then, the pixel value of each pixel in the real-time image is mapped to the mapping point in the initial modeling result to obtain the real-scene modeling result containing real-time scene image content.

[0115] Then, the fusion module sends the reality modeling results to the tracking and display service for display.

[0116] The tracking and display service processes the reality modeling results, enabling users to observe the real-time scene of the target object. For example, the tracking and display service is used to establish a communication connection with the user's terminal to send the reality modeling results to the terminal for display.

[0117] Understandably, in order to improve the user's observation experience, in addition to displaying the results of real-scene modeling, other tracking data can also be displayed.

[0118] For example, the aforementioned data analysis module also includes a streaming device selection module. This module determines the monitoring device to be used for streaming based on the analysis algorithm configured in the data analysis module. For instance, it acquires GPS positioning data uploaded by a GPS device carried by the target object, determines the identifier d1 of a monitoring device existing within a preset range of the GPS positioning data, and compares this identifier d1 with the identifier d2 of the monitoring device corresponding to the received video stream data. If they match, it indicates that the video stream data of d2 contains the target object; if they do not match, it indicates that the target object has left the monitoring range of d2, and the streaming device needs to be switched to d1. Furthermore, it can analyze the speed and direction information of the target object contained in the video stream data collected by the monitoring device to predict the target object's movement trajectory, and then determine the monitoring device that can be used for streaming within a preset time period based on the target object's movement trajectory.

[0119] Then, the video stream data corresponding to the identified monitoring device is uploaded to the message middleware. This allows the real-time data push module to push the video stream data containing the object from the message middleware to the tracking and display service. The tracking and display service then performs real-time display processing on the video stream data containing the object. The number of identified monitoring devices can be one or more. If there are multiple devices, the tracking and display service performs real-time display processing on the video stream data of each monitoring device separately, with each video stream data corresponding to a different real-time display window.

[0120] For example, the tracking and display service can also acquire Geographic Information System (GIS) maps, and the real-time data push module in the location acquisition service can push the location information of the target object to the tracking and display service, which then displays the location information of the target object in the GIS map in real time.

[0121] It should be noted that the above embodiments are based only on Figure 3 The target tracking system shown is an exemplary illustration of the target tracking method. The specific framework of the target tracking system does not limit the target tracking method of this application. The implementation of each step described above can also be applied to other systems, and this application does not limit it.

[0122] The target tracking method provided in this application obtains the location information of the target object, queries the scene modeling data that matches the location information in the preset scene modeling database, and acquires real-time images acquired by the image acquisition device that matches the location information. This allows for accurate resource retrieval based on the target object's location, improving subsequent modeling efficiency. Then, the scene modeling data and real-time images are fused to obtain a more accurate and easily observable real-scene modeling result. Based on the real-scene modeling result, the target object is tracked. While ensuring that the target object is not lost, this method provides more image details and a wider field of view for security, tracking, and other scenarios, facilitating a comprehensive study and judgment of the target object's motion scene.

[0123] Figure 8 This is a block diagram illustrating a target tracking device as shown in an exemplary embodiment of this application. Figure 8 As shown, the exemplary target tracking device 800 includes: a position acquisition module 810, a modeling data acquisition module 820, a fusion module 830, and a tracking module 840. Specifically:

[0124] Location acquisition module 810 is used to acquire the location information of the target object;

[0125] The modeling data acquisition module 820 is used to query scene modeling data that matches the location information in the preset scene modeling database, and to acquire real-time images acquired by the image acquisition device that matches the location information.

[0126] The fusion module 830 is used to fuse scene modeling data and real-time images to obtain real-scene modeling results;

[0127] The tracking module 840 is used to track target objects based on the results of reality modeling.

[0128] In the aforementioned exemplary target tracking device, by acquiring the location information of the target object, querying the scene modeling data that matches the location information in the preset scene modeling database, and acquiring the real-time image acquired by the image acquisition device that matches the location information, resources can be accurately called according to the location of the target object, thereby improving the efficiency of subsequent modeling. Then, the scene modeling data and the real-time image are fused to obtain a more accurate and easier-to-observe real-scene modeling result. Based on the real-scene modeling result, the target object is tracked. Under the premise of ensuring that the target object is not lost, more image details and visible range are provided for security, tracking and other scenarios, which facilitates comprehensive research and judgment on the movement scene of the target object.

[0129] The functions of each module can be found in the target tracking method implementation examples, and will not be repeated here.

[0130] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 900 includes a memory 901 and a processor 902. The processor 902 is used to execute program instructions stored in the memory 901 to implement the steps in any of the above-described target tracking method embodiments. In a specific implementation scenario, the electronic device 900 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 900 may also include mobile devices such as laptops and tablets, which are not limited here.

[0131] Specifically, processor 902 controls itself and memory 901 to implement the steps in any of the above-described target tracking method embodiments. Processor 902 can also be referred to as a Central Processing Unit (CPU). Processor 902 may be an integrated circuit chip with signal processing capabilities. Processor 902 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 902 can be implemented using integrated circuit chips.

[0132] Please see Figure 10 , Figure 10 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 1000 stores program instructions 1010 that can be executed by a processor. The program instructions 1010 are used to implement the steps in any of the above-described target tracking method embodiments.

[0133] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0134] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0136] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A target tracking method, characterized in that, The method includes: Obtain the location information of the target object; Query the scene modeling data in the preset scene modeling database that matches the location information, and obtain the real-time image captured by the image acquisition device that matches the location information; The scene modeling data and the real-time image are fused to obtain the real-scene modeling result; The target object is tracked based on the real-scene modeling results; The step of obtaining the location information of the target object includes: Acquire candidate location information of a target object, wherein the candidate location information includes candidate locations of the target object at at least two time points; Calculate the buffer region corresponding to the candidate position of the target object at each time step; Obtain the overlapping area between the corresponding buffer regions at each time point; Based on the overlapping region, the candidate locations in the candidate location information are filtered, and the filtered candidate location information is used as the location information of the target object.

2. The method according to claim 1, characterized in that, The calculation of the buffer region corresponding to the candidate position of the target object at each time step includes: Obtain the velocity information of the target object at each candidate position; Based on the speed information, the buffer area corresponding to each candidate position of the target object is calculated.

3. The method according to claim 1, characterized in that, The preset scene modeling database stores street modeling data; The query for scene modeling data in the preset scene modeling database that matches the location information includes: Identify the streets associated with the location information to obtain the range of associated streets; The street modeling data within the associated street range in the preset scene modeling database is used as the scene modeling data.

4. The method according to claim 1, characterized in that, The acquisition of real-time images captured by the image acquisition device that matches the location information includes: Determine the device selection range corresponding to the location information, and obtain the device coordinates of each image acquisition device; Image acquisition devices whose coordinates are within the selected range of the device are selected as image acquisition devices that match the location information; Acquire real-time images captured by an image acquisition device that matches the location information.

5. The method according to claim 1, characterized in that, The process of fusing the scene modeling data and the real-time image to obtain the real-scene modeling result includes: Based on the location information of the target object, the scene modeling data is cropped to obtain the initial modeling result; The real-time image is mapped onto the initial modeling result to obtain the real-scene modeling result.

6. The method according to claim 5, characterized in that, The step of mapping the real-time image onto the initial modeling result to obtain the real-scene modeling result includes: Keypoint detection is performed on the real-time image and the initial modeling result respectively to obtain the image keypoint set corresponding to the real-time image and the modeling keypoint set corresponding to the initial modeling result; Extract the associated key point pairs between the image key point set and the modeling key point set; Based on the associated key point pairs, the real-time image is mapped to the initial modeling result to obtain the real-scene modeling result.

7. The method according to claim 6, characterized in that, The process of mapping the real-time image to the initial modeling result based on the associated key point pairs to obtain the real-scene modeling result includes: Based on the mapping relationship of each associated key point pair, obtain the mapping point corresponding to each pixel in the real-time image in the initial modeling result; The pixel value of each pixel in the real-time image is mapped to the mapping point in the initial modeling result to obtain the real-scene modeling result.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-7.

Images