A cross-camera multi-target tracking method, device, equipment and medium

Abstract

Embodiments of the present application disclose a cross-camera multi-target tracking method, device, equipment and medium, which comprises the following steps: acquiring images of a current moment collected by a plurality of preset cameras; determining the targets, positions and target features of the targets appearing in each image by using a neural network model; determining a target as a non-stable target if the number of times of appearance of the target is less than a preset first threshold value; and determining a first stable target matched with each non-stable target by using a Hungarian algorithm, i.e. performing global optimal matching, so as to ensure that the same target is still accurately identified when crossing different camera collection areas, and further ensure the uniqueness of the target identity.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, and in particular to a method, apparatus, device and medium for multi-target tracking across cameras. Background Technology

[0002] In recent years, with the intelligent development of fields such as smart justice, smart buildings, smart parks, and smart transportation, the intelligence level of their perception systems is transforming from isolated intelligence relying on single sensors to fused intelligence that coordinates multiple sensors. Against this backdrop, cross-camera multi-target tracking technology, as one of the core research directions of multi-sensor intelligence, has emerged. This technology deploys multiple surveillance cameras to form a collaborative perception network with a wider coverage area, aiming to continuously and seamlessly track specific targets, such as pedestrians and vehicles, across multiple camera acquisition areas, and ultimately generate their complete motion trajectory within the global monitoring range.

[0003] However, existing cross-camera multi-target tracking technologies still face a series of pressing technical challenges in practical deployment and application. Specifically, when a target crosses a camera's blind spot, continuous tracking is required, and the target's identification can easily switch or become confused when traversing different camera acquisition areas. Therefore, ensuring that the same target is assigned and maintains a unique identification in cross-camera or relay monitoring is a significant challenge.

[0004] Therefore, there is an urgent need in this field for a cross-camera multi-target tracking scheme that can effectively overcome the above challenges in order to achieve real-time and accurate global target tracking, especially in dealing with complex scenarios such as cross-blind zone, multi-view differences and target high similarity, while still ensuring the uniqueness of the target identity. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for multi-target tracking across cameras, to ensure that the same target is assigned and maintains a unique identity in cross-camera or relay monitoring of multiple cameras.

[0006] In a first aspect, embodiments of this application provide a cross-camera multi-target tracking method, the method comprising: Images captured at the current moment by multiple preset cameras are acquired, wherein there are adjacent cameras among the preset cameras; a neural network model is used to determine the target appearing in each image, the target's location, and the target's features; For each target identified in an image, determine the local ID of the target in the camera that acquired the image, and determine the number of times the target with that local ID has appeared in the image acquired by the camera; For each unstable target, the distance between the unstable target and each stable target is determined based on the target characteristics and position of the unstable target and each stable target, wherein the unstable target is the target that has appeared less than a preset first threshold, and the stable target is the target that has appeared not less than the preset first threshold. The Hungarian algorithm is used to determine the first stable target that matches each unstable target. For any unstable target, the first stable target and the unstable target are determined to be the same target based on the distance between the unstable target and the first stable target.

[0007] In one possible implementation, before determining the distance between each unstable target and each stable target based on the target characteristics and position of the unstable target and each stable target, the method further includes: For each target in all images acquired at the current moment, the number of times the target has appeared is updated. If the number of times the target has appeared after the update is less than a preset first threshold, the target is identified as an unstable target; if the number of times the target has appeared after the update is not less than the preset first threshold, the target is identified as a stable target.

[0008] In one possible implementation, determining the distance between each unstable target and each stable target, based on the target characteristics and positions of the unstable target and each stable target, includes: For each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same as other stable targets in the image to which the unstable target belongs, then the distance between the unstable target and the stable target is determined based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, the distance between the unstable target and the stable target is determined as a set value.

[0009] In one possible implementation, determining that the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target includes: If the distance between the unstable target and the matched first stable target is not the set value, then the first stable target and the unstable target are determined to be the same target.

[0010] In one possible implementation, if the distance between the unstable target and the matched first stable target is the set value, or if the first stable target of the unstable target cannot be determined using the Hungarian algorithm, the method further includes: Determine each stable target and its first feature that have been identified within a preset time period prior to the current moment; For the unstable target, based on the target features of the unstable target and the first features of each stable target, the similarity between the unstable target and the first features of each stable target is determined respectively; if the first feature similarity is greater than a preset similarity threshold, the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target, otherwise the distance between the unstable target and the stable target is determined as the set value. The Hungarian algorithm is used to determine a second stable target that matches the unstable target; if the distance between the unstable target and the matching second stable target is not the set value, then the second stable target and the unstable target are determined to be the same target.

[0011] In one possible implementation, determining the similarity between the unstable target and the first feature of each stable target based on the target feature of the unstable target and the first feature of each stable target includes: For each stable target, calculate the feature similarity between each first feature of the stable target and the target feature; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0012] In one possible implementation, after identifying the target as a stable target, the method further includes: Generate a global tracking ID for this stable target; After determining that the stable objective and the unstable objective are the same objective, the method further includes: Use the global tracking ID of the stable target as the global tracking ID for this match of the unstable target, increment the count corresponding to the global tracking ID by 1, and increment the match count corresponding to the unstable target by 1; The number of matches for each global tracking ID matched by the unstable target is determined. When the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, the global tracking ID that is not less than the preset second threshold is selected as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0013] Secondly, embodiments of this application provide an apparatus for cross-camera multi-target tracking, the apparatus comprising: An acquisition module is used to acquire images captured by multiple preset cameras at the current moment, wherein there are adjacent cameras among the preset cameras; The determination module is used to determine the targets appearing in each image, their locations, and their features using a neural network model; for each target identified in an image, it determines the local ID of the target in the camera that acquired the image, and determines the number of times the target with that local ID has appeared in the image acquired by the camera; for each unstable target, it determines the distance between the unstable target and each stable target based on the target features and locations of the unstable target and each stable target, wherein the unstable target is the target whose number of appearances is less than a preset first threshold, and the stable target is the target whose number of appearances is not less than the preset first threshold; it uses a Hungarian algorithm to determine the first stable target that matches each unstable target, and for any unstable target, it determines that the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target.

[0014] In one possible implementation, the determining module is further configured to update the number of times each target has appeared in all images acquired at the current time. If the number of times the target has appeared after the update is less than a preset first threshold, the target is determined as an unstable target; if the number of times the target has appeared after the update is not less than the preset first threshold, the target is determined as a stable target.

[0015] In one possible implementation, the determining module is specifically used to, for each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same target as other stable targets in the image to which the unstable target belongs, then determine the distance between the unstable target and the stable target based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, determine the distance between the unstable target and the stable target as a set value.

[0016] In one possible implementation, the determining module is specifically configured to determine that the first stable target and the unstable target are the same target if the distance between the unstable target and the matching first stable target is not the set value.

[0017] In one possible implementation, the determining module is further configured to determine each stable target and its first feature that have been determined within a preset time period prior to the current time; for the unstable target, based on the target feature of the unstable target and the first feature of each stable target, determine the first feature similarity between the unstable target and each stable target respectively; if the first feature similarity is greater than a preset similarity threshold, then the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target; otherwise, the distance between the unstable target and the stable target is determined to be the preset value; the Hungarian algorithm is used to solve for and determine a second stable target that matches the unstable target; if the distance between the unstable target and the matched second stable target is not the preset value, then the second stable target and the unstable target are determined to be the same target.

[0018] In one possible implementation, the determining module is specifically used to calculate the feature similarity between each first feature of the stable target and the target feature for each stable target; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0019] In one possible implementation, the determining module is further configured to generate a global tracking ID for the stable target; The determining module is further configured to use the global tracking ID of the stable target as the global tracking ID of the unstable target in this match, increment the number corresponding to the global tracking ID by 1, and increment the number of matches corresponding to the unstable target by 1; determine the number of matches for each global tracking ID matched by the unstable target, and when the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, select the global tracking ID that is not less than the preset second threshold as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0020] Thirdly, embodiments of this application also provide an electronic device, the electronic device including a processor, the processor being configured to execute a computer program stored in a memory to implement the steps of any of the methods described above.

[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0022] In this application, images of the current moment are acquired by multiple preset cameras; a neural network model is used to determine the target, the target's location, and the target's features in each image; targets that have appeared less than a preset first threshold are identified as unstable targets, and the Hungarian algorithm is used to determine the first stable target that matches each unstable target, i.e., global optimal matching is performed, thereby ensuring that the same target is still accurately identified when it crosses different camera acquisition areas, thus guaranteeing the uniqueness of the target's identity. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating a cross-camera multi-target tracking method provided in an embodiment of this application. Figure 2 A schematic diagram illustrating the distribution of multiple cameras provided in an embodiment of this application; Figure 3 This application provides a schematic diagram of a method for determining a stable target corresponding to an unstable target when the distance between the unstable target and the matching first stable target is a set value, or when the first stable target of the unstable target is not determined using the Hungarian algorithm. Figure 4 A schematic diagram of a device structure for cross-camera multi-target tracking provided in an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0026] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0027] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0028] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0029] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0031] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0032] Before introducing the cross-camera multi-target tracking method provided in the embodiments of this application, for ease of understanding, some terms involved in the embodiments of this application will be explained below.

[0033] Cross-camera multi-target tracking refers to tracking targets across images acquired by multiple synchronized cameras.

[0034] Example 1: Figure 1 This application provides a schematic diagram of a cross-camera multi-target tracking method, which includes the following steps: S101: Acquire images of the current moment captured by multiple preset cameras, wherein there are adjacent cameras among the preset cameras; use a neural network model to determine the target appearing in each image, the location of the target, and the target features.

[0035] The cross-camera multi-target tracking method provided in this application is applied to an electronic device, which may be a camera, a webcam, or the like.

[0036] There are adjacent cameras in the preset cameras, where adjacent cameras are those with overlapping acquisition areas. Figure 2 This is a schematic diagram of the distribution of multiple cameras provided in an embodiment of this application, such as... Figure 2 As shown, multiple preset cameras include camera 1, camera 2, ..., camera i, camera j, and camera N. Camera 1 and camera 2 have overlapping acquisition areas, so camera 1 and camera 2 are adjacent cameras. Camera i, camera j, and camera N also have overlapping acquisition areas, so camera i, camera j, and camera N are adjacent cameras. However, camera 1 and camera N, and camera 2 and camera i do not have overlapping acquisition areas, so camera 1 and camera N, and camera 2 and camera i are not adjacent cameras.

[0037] In practice, there may be blind spots between areas captured by multiple cameras, i.e., areas that cannot be captured, such as... Figure 2 As shown, area A is the area that cameras 1 and 2 can capture, and area B is the area that cameras i, j, and N can capture. The area outside of areas A and B is the blind zone, which is the area that no camera can capture.

[0038] An electronic device acquires images of the current moment captured by multiple preset cameras. For each image, a neural network model is used to determine the target appearing in the image, the target's location, and the target's features. This neural network model can be a deep learning object detection model. The target can be a pedestrian, a motor vehicle, or a non-motor vehicle, etc. The target's location can be the coordinates of a bounding box, such as the coordinates of the top-left and bottom-right corners of the bounding box. Typically, the target is on the ground, so its location can be expressed using two-dimensional coordinates. However, this application does not impose constraints on whether three-dimensional or two-dimensional coordinates are used for the target's location.

[0039] Furthermore, it should be noted that the electronic device can pre-calibrate the parameters of each preset camera using the Perspective-n-Point (PnP) algorithm. Based on the calibrated cameras, the electronic device can calculate the world coordinates of the pixels in the images captured by each preset camera, determine which areas of the image each preset camera can capture, and also determine which preset cameras can capture a certain spatial coordinate. In addition, the electronic device can pre-store a list of observable cameras for any given spatial coordinate. This list of observable cameras refers to all preset cameras that can capture that spatial coordinate. Therefore, the target position determined by the electronic device is the global coordinate in the world coordinate system, which can be represented as pos. The use of the n-point perspective problem algorithm to calibrate the parameters of each preset camera is existing technology and will not be elaborated upon here.

[0040] In one possible implementation, when a target appears incomplete in an image, the electronic device can supplement the height of the incomplete target to a preset height value, thereby determining the target's position. The incomplete target situation can be categorized into two types: only the upper part of the target and only the lower part of the target.

[0041] Optionally, the electronic device can also create a target tracking state (track_state) for each target within the same camera. This target tracking state includes target creation (Ta), target update (Tb), target loss (Tc), and target deletion (Td). When the target's tracking state within the camera is "target deletion," that target does not participate in any of the following steps.

[0042] In one possible implementation, to reduce computing resources, the user can pre-set the time intervals for the electronic device to acquire images from multiple preset cameras, according to actual needs. The electronic device can then acquire images from these preset cameras according to these time intervals. These multiple preset cameras share the same timing system, and the electronic device can obtain synchronized images through a time synchronization method.

[0043] In one possible implementation, the electronic device can further input the determined target features into a pre-trained neural network for feature quality scoring to obtain a feature quality score for the target features. The determined target features and their corresponding feature quality scores are then stored in a database for subsequent updates to the target features. Optionally, when storing the target features corresponding to the target in the image at the current time, the electronic device can also store them based on the feature quality score Q of the target at each time step in the database. Specifically, it selects the target feature with the highest feature quality score among all stored features up to the current time step as the target feature for the current time step and stores it in the database.

[0044] S102: For each target identified in an image, determine the local ID of the target in the camera that acquired the image, and determine the number of times the target with that local ID has appeared in the image acquired by the camera.

[0045] For each target identified in an image, the electronic device determines the target's local ID within the camera that captured the image. This local ID, also known as the target tracking ID within the camera that captured the image, can be denoted as tid. The process of determining the target's local ID within the camera that captured the image is as follows: using the first image from the previous timeframe, which is closest to the current timeframe, a deep learning tracker calculates the similarity between the target and each target in the first image. The local ID corresponding to the first target with the highest similarity greater than a preset threshold is selected as the target's local ID. If no first target has a similarity greater than the preset threshold, the target is determined to be a newly appearing target, and its local ID within the camera that captured the image is generated. The electronic device can generate the target's local ID by combining the camera identifier of the camera that captured the target with a randomly generated value. Different cameras have different camera identifiers, and the randomly generated value can be the same or different. For example, if the camera identifier of the camera to which the target belongs is a randomly generated value of 1 in A, then the local ID of the target is determined to be A1; if the camera identifier of the camera to which the target belongs is a randomly generated value of 1 in B, then the local ID of the target is determined to be B1.

[0046] Based on the identified local ID of the target in the camera that captured the image, the number of times the target with that local ID has appeared in the images captured by the camera is determined from a pre-stored database of local IDs. This number of appearances is then updated to obtain the updated number of appearances. This updated number of appearances is then determined as the total number of appearances of the target with that local ID in the images captured by the camera. Each record in the pre-stored database stores the target's local ID, the current time when the image containing the target was captured, the target's features, its location, and the number of times it has appeared. Specifically: for each target identified in an image, based on the current time when the image containing the target was captured, the first record of the most recent previous time corresponding to the target's current time is determined. The number of appearances of the target stored in this first record is determined and updated to obtain the updated number of appearances. This updated number of appearances is then determined as the total number of appearances of the target with that local ID in the images captured by the camera. The number of appearances of the target stored in the first record refers to the number of appearances of the target corresponding to that local ID in the image captured by the camera at the previous time, after the update.

[0047] Optionally, to facilitate recording the actual situation of each target, each record in the database can also store the camera ID of the camera to which the target belongs, which can be represented as cid, target tracking status, feature quality score of target features, and the current target's observation camera list (obs_neighbor). The current target's observation camera list refers to the list of observable cameras at the spatial coordinates of the current target, indicating which preset cameras can observe the target's location.

[0048] S103: For each unstable target, determine the distance between the unstable target and each stable target based on the target characteristics and position of the unstable target and each stable target, wherein the unstable target is the target that has appeared less than a preset first threshold, and the stable target is the target that has appeared not less than the preset first threshold.

[0049] Based on the number of times each target's local ID has appeared in the camera-captured image, as determined in step 102, targets whose appearance frequency is less than a preset first threshold are identified as unstable targets. All targets in the preset database whose appearance frequency is not less than the preset first threshold are identified as stable targets.

[0050] For each unstable target, based on each stable target already stored in a preset database, the distance between the unstable target and each stable target is determined according to the target characteristics and location of the unstable target and each stable target. The preset first threshold can be pre-set by the user according to actual conditions.

[0051] The process of determining the distance between the unstable target and each stable target based on the target features and positions of the unstable target and each stable target is as follows: For each stable target, based on the target features of the unstable target and the target features of the stable target, a second feature similarity between the unstable target and the stable target is determined using a preset vector space model; based on the positions of the unstable target and the stable target, a spatial distance algorithm is used to determine the spatial distance between the unstable target and the stable target; subsequently, a first inverse number corresponding to the second feature similarity is determined, and a first sum of a preset value and the first inverse number is determined; based on the first product of the first sum and the spatial distance, the distance between the unstable target and the stable target is determined. In other words, the distance between the unstable target and the stable target can be expressed as... Where sim represents the second feature similarity of the target features of two targets, dist represents the spatial distance between the two targets, and X represents a preset value, which is usually set to 1. Optionally, to ensure the accuracy of distance calculation, when the second feature similarity is less than the preset similarity threshold, the electronic device can directly determine the second feature similarity as 0.

[0052] Optionally, to facilitate recording the actual situation of each target, each record in the pre-saved database can also store the target's target matching state (match_state), the number of matches (match_num), and the number of times the target has failed to match (unmatched_num). The target matching state includes pending match (Ma), unstable match (Mb), and stable match (Mc). The pending match state refers to the target's first appearance in the image captured by the camera; for example, the target moving from one camera's capture area to the capture area of ​​another camera with an overlapping region. Figure 2 For example, a target enters the acquisition area of ​​camera 2 from the acquisition area of ​​camera 1, meaning it first appears within the acquisition area of ​​camera 2; another example is a target moving from one area through a blind spot into another area, such as... Figure 2 For example, a target moves from area A to area B, disappearing in area A and reappearing in area B; another example is when a target first appears in the overlapping area of ​​several cameras, such as... Figure 2 The overlapping area of ​​Camera 1 and Camera 2 refers to the area where the target appears simultaneously within both Camera 1 and Camera 2. An unstable matching target state refers to a target appearing in the image captured by the camera more than once and with a frequency less than a preset first threshold. A stable matching target state refers to a target appearing in the image captured by the camera more than or equal to a preset first threshold. The number of target matches refers to the number of times the target has successfully matched with other targets, where matching refers to the process of determining whether a target to be matched or an unstable matching target is the same target as a stable target. The number of times a target has continuously failed to match refers to the number of times the target has attempted to match with other targets but has not been matched. Targets with a matching state of "to be matched Ma" or "unstable matching Mb" are considered unstable targets.

[0053] S104: Use the Hungarian algorithm to determine the first stable target that matches each unstable target. For any unstable target, determine that the first stable target and the unstable target are the same target based on the distance between the unstable target and the first stable target.

[0054] The Hungarian algorithm is a combinatorial optimization algorithm for solving allocation problems. Its core principle is to find the allocation scheme that minimizes total cost, total distance, or time by adjusting the matrix. The final output is an optimal matching scheme, i.e., the correspondence between each person (or resource, or goal) and the task (or goal). In this application, the Hungarian algorithm is used to assign an optimal stable goal to each unstable goal.

[0055] The electronic device constructs a distance matrix based on the distances between the unstable targets and each stable target determined in step 103, and uses the Hungarian algorithm to find the allocation scheme with the shortest total distance, that is, to determine the first stable target matched by each unstable target. The process of using the Hungarian algorithm to determine the first stable target matched by each unstable target is prior art and will not be described in detail here.

[0056] After using the Hungarian algorithm to determine the first stable target that matches each unstable target, for any unstable target, if the distance between the unstable target and the matched first stable target is less than a preset distance threshold, then the first stable target and the unstable target are determined to be the same target. This preset distance threshold can be set by the user according to the actual situation.

[0057] In this embodiment, images captured by multiple preset cameras at the current moment are acquired; a neural network model is used to determine the target, the target's location, and the target's features in each image; targets that have appeared less than a preset first threshold are identified as unstable targets, and the Hungarian algorithm is used to determine the first stable target that matches each unstable target, i.e., global optimal matching is performed, thereby ensuring that the same target is still accurately identified when it crosses different camera acquisition areas, thus guaranteeing the uniqueness of the target's identity.

[0058] Example 2: To ensure the accuracy of target matching and the real-time requirements of target tracking, this application updates the occurrence count of each target in all images acquired at the current moment before determining the distance between unstable targets and each stable target. This ensures the accuracy of the subsequent matching of unstable and stable targets. Specifically: For each target in all images acquired at the current moment, the number of times the target has appeared is updated. If the number of times the target has appeared after the update is less than a preset first threshold, the target is identified as an unstable target; if the number of times the target has appeared after the update is not less than the preset first threshold, the target is identified as a stable target.

[0059] For each target in all images captured at the current moment, the electronic device first updates the count of occurrences of that target. Specifically, based on the target's local ID, it determines the first record of the previous moment closest to the current moment in a preset database and determines the count of occurrences stored in that first record. Then, it increments this count by 1 to obtain the updated count of occurrences that should be saved in the current moment's record for that target. In other words, based on the target's local ID, it queries the preset database for the first record of the previous moment closest to the current moment. If a corresponding first record is found, the count of occurrences of the target is determined and incremented by 1 to obtain the updated count of occurrences that should be saved in the current moment's record for that target. If no corresponding first record is found, a corresponding record is created for that target corresponding to that local ID and stored in the preset database, with the count of occurrences set to 1, resulting in the updated count of occurrences that should be saved in the current moment's record for that target. The count of occurrences of the target stored in the record created for that target corresponding to that local ID is the count of occurrences of that target in the images captured by the camera.

[0060] It should be noted that, optionally, if the above query is successful, it is also possible to further determine whether there is an anomaly in the tracking of the target based on whether the global coordinates pos at the current time of the target and the global coordinates pos in the first record meet the given distance threshold, and thus decide whether it should be regarded as a newly appeared target.

[0061] After updating the occurrence count of the target, for each target in all images acquired at the current moment, if the updated occurrence count of the target is less than a preset first threshold, the target is identified as an unstable target; if the updated occurrence count of the target is not less than the preset first threshold, the target is identified as a stable target. That is, it is determined whether each target in all images acquired at the current moment is an unstable target, and thus it is determined whether there is a stable target in the images acquired by multiple preset cameras at the current moment.

[0062] Subsequently, if stable targets exist in the images captured by multiple preset cameras at the current moment, then for each unstable target in each image at the current moment, the distance between the unstable target and each stable target in the current moment's images is determined. Based on the determined distances, the Hungarian algorithm is used to determine the first stable target that matches each unstable target. In other words, for all targets in the images captured by all cameras at the current moment, targets with a matching state of Mc are selected to form a stable matching target set C_Mc; and for each image captured by each preset camera at the current moment, targets with a matching state of Ma or Mb are selected to form an unstable matching target set Ci_Mab. The unstable matching target set Ci_Mab is then matched with the stable matching target set C_Mc using the Hungarian algorithm to determine the first stable target that matches each unstable target.

[0063] In this embodiment, the accuracy of the target matching is ensured by updating the number of occurrences of each target in all images acquired at the current moment, thereby ensuring the accuracy of the subsequent use of unstable and stable targets.

[0064] Example 3: To ensure the accuracy of target matching, the specific operation for calculating the distance between unstable and stable targets in this embodiment of the application is as follows: For each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same as other stable targets in the image to which the unstable target belongs, then the distance between the unstable target and the stable target is determined based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, the distance between the unstable target and the stable target is determined as a set value.

[0065] When calculating the distance between an unstable target and each stable target, the electronic device first determines, for each unstable target and each stable target separately, whether the area captured by the camera that acquired the stable target includes the current location of the unstable target, and whether the stable target and other stable targets in the image to which the unstable target belongs are not the same target. If both are true, the distance between the unstable target and the stable target is determined based on the target features and location of the unstable target and the target features and location of the stable target, which is the specific process described in step 103 above, and will not be repeated here; otherwise, the distance between the unstable target and the stable target is determined as a set value. This set value is a relatively large value.

[0066] To facilitate understanding, a simple example is provided below to illustrate how this application determines the distances between the unstable target and each stable target.

[0067] Assume that cameras A, B, and C are pre-deployed in the area to be monitored. Cameras A and B are adjacent and their acquisition areas both include overlapping region 1. Cameras A and C are not adjacent, as are cameras B and C. Electronic devices acquire images captured by cameras A, B, and C at the current moment. A neural network model is used to determine the targets appearing in each image, their locations, and their features. Specifically, in the overlapping region 1 of the image captured by camera A, there are unstable targets A1 and stable targets A2; in the overlapping region 1 of the image captured by camera B, there are stable targets B1 and B2; and in the image captured by camera C, there are stable targets. C1; where it has been determined that stable target A2 and stable target B2 are the same target, when calculating the distance between unstable target A1 and each stable target, for unstable target A1 and stable target B1, the electronic device can know from the pre-saved list of observation cameras that the area collected by camera B to which stable target B1 belongs includes the current location of unstable target A1, and that stable target B1 is not the same target as other stable target A2 in the image to which unstable target A1 belongs. Therefore, based on the target characteristics and location of unstable target A1 and the target characteristics and location of stable target B1, the distance between unstable target A1 and stable target B1 is determined. For unstable target A1 and stable target B2, the electronic device, based on a pre-saved list of observation cameras, knows that the area captured by camera B, which belongs to stable target B2, includes the current location of unstable target A1. However, stable target B2 and other stable targets A2 in the image containing unstable target A1 are the same target. To ensure the uniqueness of the target, the distance between unstable target A1 and stable target B2 is set as a preset value. For unstable target A1 and stable target C1, the electronic device, based on a pre-saved list of observation cameras, knows that the area captured by camera C, which belongs to stable target C1, does not include the current location of unstable target A1. Therefore, the distance between unstable target A1 and stable target C1 is set as a preset value.

[0068] In one possible implementation, to further ensure the accuracy of distance calculation, optionally, after determining that the area captured by the camera that captures the stable target at the current moment includes the location of the unstable target at the current moment, and that the stable target is not the same as other stable targets in the image to which the unstable target belongs, it can also be determined whether the spatial distance between the stable target and the unstable target is less than a preset distance threshold. If so, the distance between the unstable target and the stable target is then determined based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, the distance between the unstable target and the stable target is determined as a set value.

[0069] In this embodiment, the distance between the unstable target and the stable target is calculated by determining whether the area captured by the camera that captured the stable target at the current moment includes the location of the unstable target at the current moment, and whether the stable target and other stable targets in the image to which the unstable target belongs are the same target. This ensures the uniqueness of the target and thus guarantees the accuracy of target matching. Furthermore, this application combines two dimensions, position and feature, when calculating the distance between the unstable target and the stable target. When feature similarity exists, the influence of position on the distance between the unstable target and the stable target is reduced. When feature similarity does not exist, position completely dominates the distance between the unstable target and the stable target. This approach uses position to avoid the case of no features and introduces features to eliminate position estimation errors. Therefore, it can eliminate the influence of occlusion, incomplete target, calibration errors, and differences in multi-camera perspectives to a certain extent, while also solving the problem of highly similar target features, thereby improving the accuracy of matching.

[0070] Example 4: To ensure the accuracy of target matching, this application further specifies that the determination of whether the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target is as follows: If the distance between the unstable target and the matched first stable target is not the set value, then the first stable target and the unstable target are determined to be the same target.

[0071] Based on the scheme described in Embodiment 3 above, to ensure the accuracy of target matching, the electronic device determines the distance between any unstable target and the first stable target it matches. If the distance between the unstable target and the first stable target matches is the set value, it proves that the unstable target and the first stable target matches violate the prior assumption and are therefore almost impossible to be the same target. That is, it proves that the distance between the unstable target and the first stable target matches is large, meaning that the probability of them being the same target is small. If the distance between the unstable target and the first stable target matches is not the set value, it proves that the unstable target and the first stable target matches satisfy the prior assumption and are likely the same target. That is, it proves that the distance between the unstable target and the first stable target matches is small, meaning that the probability of them being the same target is large. Therefore, if the distance between the unstable target and the first stable target matches is not the set value, the electronic device directly determines that the first stable target and the unstable target are the same target.

[0072] Example 5: To achieve accurate target tracking, this application also specifies limitations for situations where the distance between the unstable target and the first stable target is a set value, or where the Hungarian algorithm fails to identify the first stable target for the unstable target. Figure 3 This application provides a schematic diagram of a method for determining a stable target corresponding to an unstable target when the distance between the unstable target and the matched first stable target is a set value, or when the first stable target of the unstable target cannot be determined using the Hungarian algorithm. The process includes the following steps: S301: Determine each stable target and its first feature that have been determined within a preset time period prior to the current time.

[0073] If the distance between the unstable target and the first stable target is a set value, or if the Hungarian algorithm fails to identify the first stable target for the unstable target, it proves that the electronic device has not identified the first stable target matching the unstable target; that is, no matching target has been found. In this case, the electronic device determines each stable target and its first feature identified within a preset time period prior to the current time, based on pre-saved records. The preset time period is set by the user according to actual needs. For example, if the current time is 10:30:50 and the preset time period is 30 seconds, the electronic device determines each stable target and its first feature identified between 10:30:19 and 10:30:49, stored in the preset database.

[0074] Furthermore, it should be noted that when the number of unstable targets in the current moment is greater than the number of stable targets, there may be situations where the Hungarian algorithm fails to identify the first stable target for some unstable targets. In other words, if the Hungarian algorithm fails to identify the first stable target for an unstable target, subsequent target matching operations are required for that unstable target in order to achieve accurate target tracking.

[0075] S302: For the unstable target, based on the target features of the unstable target and the first features of each stable target, determine the similarity between the first features of the unstable target and each stable target.

[0076] For an unstable target that has not been matched with the first stable target, the electronic device compares it with each stable target and the first feature determined in step 301, and determines the similarity between the unstable target and the first feature of each stable target based on the target feature of the unstable target and the first feature of each stable target.

[0077] The step of determining the similarity between the first feature of the unstable target and the first feature of each stable target based on the target feature of the unstable target and the first feature of each stable target includes: For each stable target, calculate the feature similarity between each first feature of the stable target and the target feature; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0078] For each stable target, the electronic device determines each first feature that the stable target has determined within a preset time period before the current time, then calculates the feature similarity between each first feature of the stable target and the target feature, and selects the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0079] S303: If the first feature similarity is greater than the preset similarity threshold, then the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target; otherwise, the distance between the unstable target and the stable target is determined as the set value.

[0080] Regarding the first feature similarity between the unstable target and each stable target determined in step 302, if the first similarity is greater than a preset similarity threshold, it indicates a high probability that the unstable target and the stable target are the same target. Therefore, the electronic device determines the negative value of the first feature similarity as the distance between the unstable target and the stable target. Otherwise, it indicates a low probability that the unstable target and the stable target are the same target, and the distance between the unstable target and the stable target is directly determined as a set value. That is, let the similarity between the unstable target and the stable target be sim. If sim is greater than the preset similarity threshold, then the distance d = -sim; otherwise, it is the set value. The range of the first feature similarity between two targets is 0~1, with larger values ​​indicating greater similarity. If one of the two targets has no corresponding feature, then their feature similarity is defined as the set value 0.

[0081] S304: Use the Hungarian algorithm to determine a second stable target that matches the unstable target; if the distance between the unstable target and the matching second stable target is not the set value, then determine that the second stable target and the unstable target are the same target.

[0082] The electronic device constructs a distance matrix based on the distances between the unstable target and each stable target determined in step 303, and uses the Hungarian algorithm to find the allocation scheme with the shortest total distance, thus determining the second stable target that matches the unstable target. The process of using the Hungarian algorithm to determine the second stable target that matches the unstable target is prior art and will not be described in detail here.

[0083] After using the Hungarian algorithm to determine the second stable target that matches the unstable target, if the distance between the unstable target and the matched second stable target is not a set value, then the second stable target and the unstable target are determined to be the same target.

[0084] In one possible implementation, before step 301, the electronic device can further divide the unstable targets in all images acquired at the current time that have not yet determined a matching first stable target into target clusters, thereby forming multiple target clusters. Specifically: traverse all targets in all images acquired by all cameras at the current time; for the unstable targets in all images acquired at the current time that have not yet determined a matching first stable target, select all targets with a target matching state of Ma, and group them according to camera ID; place camera pairs that are adjacent to each other into set C, and place cameras that do not have adjacent cameras into set C separately; traverse set C, and perform Hungarian algorithm matching on the targets with a target matching state of Ma between pairs of cameras. The matching process is the same as steps 103 and 104, and will not be repeated here; divide the unstable targets into target clusters, ultimately forming multiple target clusters. A target cluster consists of targets acquired by multiple cameras, representing observations of the same actual target by multiple cameras. For example, targets with a target matching state of Ma are selected, including target 1 in camera L, target 2 in camera K, target 3 in camera Y, and target 11 in camera Z. Cameras L, K, and Z are adjacent cameras, while camera Y has no adjacent cameras. Hungarian algorithm matching is performed on each pair of targets with a target matching state of Ma. If target 1, target 2, and target 11 are found to be matched, it proves that target 1, target 2, and target 11 are the same actual target. At this point, target 1, target 2, and target 11 are grouped into the same target cluster, and target 3 is also grouped into the same target cluster. Subsequently, the electronic device performs the above step 302 operation on the same target cluster. Specifically, for each target with a target matching state of Ma in the target cluster, the similarity between the target and the third feature of each stable target is determined based on the target feature of the target and the first feature of each stable target. When any target in the same target cluster matches the third stable target, it is determined that all targets in the target cluster and the third stable target are the same target, and each target in the target cluster is considered a stable target.

[0085] In this embodiment, each stable target and its first feature determined within a preset time period prior to the current time are identified. For unstable targets that do not match the first stable target, the similarity between the unstable target and the first feature of each stable target is determined based on the target feature of the unstable target and the first feature of each stable target. If the first feature similarity is greater than a preset similarity threshold, the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target; otherwise, the distance between the unstable target and the stable target is determined to be a set value. The Hungarian algorithm is used to solve for and determine the second stable target that matches the unstable target. If the distance between the unstable target and the matched second stable target is not a set value, the second stable target and the unstable target are determined to be the same target. This effectively solves the problem of continuous tracking across blind zones, i.e., when a target crosses a blind zone, it is no longer possible to use spatiotemporal information for continuous tracking. When a target crosses a blind zone, it will inevitably disappear from all camera views and then reappear from some camera views. Matching the target features of newly added targets with historical target features in the database can handle this situation.

[0086] Example 6: To ensure the accuracy of target matching, after identifying the target as a stable target, this application also generates a global tracking ID for that target to enable cross-camera target tracking of the same target. Specifically: Generate a global tracking ID for this stable target; After determining that the stable objective and the unstable objective are the same objective, the method further includes: Use the global tracking ID of the stable target as the global tracking ID for this match of the unstable target, increment the count corresponding to the global tracking ID by 1, and increment the match count corresponding to the unstable target by 1; The number of matches for each global tracking ID matched by the unstable target is determined. When the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, the global tracking ID that is not less than the preset second threshold is selected as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0087] For each target, when the number of times the target appears in the image exceeds a preset first threshold, the electronic device identifies the target as a stable target and generates a global tracking ID for that stable target. The global tracking ID refers to the tracking identifier of the target within multiple preset camera-captured areas, and can consist of numbers and / or letters. The global tracking ID can also be represented as gid.

[0088] In other words, each stable target has a trajectory, which is formed by the actual displacement of the corresponding stable target. Each trajectory corresponds to a unique global tracking ID, which can also be called the trajectory ID, represented as track_id. Each target belongs to only one trajectory.

[0089] After determining in Examples 4 and 5 that the stable target and the unstable target are the same target, the electronic device uses the global tracking ID of the stable target as the global tracking ID of the unstable target for this match, increments the number of times corresponding to the global tracking ID by 1, and increments the number of times corresponding to the unstable target by 1.

[0090] For example, if the stable target is A1, its global tracking ID is 500, the unstable target is B1 with 3 matching attempts, and the results of the first three matches show that the unstable target B1 with global tracking ID 400 occurred once and with global tracking ID 500 occurred twice, then after determining that the stable target A1 and the unstable target B1 are the same target, the electronic device will use the global tracking ID 500 of the stable target A1 as the global tracking ID of the unstable target for this match, update the number of matches corresponding to the global tracking ID 500 of the unstable target B1 to 3, keep the number of matches corresponding to the global tracking ID 400 of the unstable target B1 unchanged, and update the number of matches corresponding to the unstable target to 4.

[0091] After updating the number of times corresponding to this matching result, the electronic device determines the number of times each global tracking ID matched by the unstable target is matched. When the number of times any global tracking ID matched by the unstable target is not less than a preset second threshold, the global tracking ID that is not less than the preset second threshold is selected as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0092] For example, if the preset second threshold is 3, after the number of times corresponding to this matching result is updated, the number of times corresponding to the global tracking ID 500 of the unstable target B1 is 3, and the number of times corresponding to the global tracking ID 400 of the unstable target B1 is 1. Then the electronic device determines that the global tracking ID of the unstable target B1 is 500 and 3 times, and 400 and 1 time. Since the number of times of the global tracking ID 500 is not less than the preset second threshold of 3, the global tracking ID 500 corresponding to 3 times is selected as the final global tracking ID of the unstable target.

[0093] In one possible implementation, the electronic device deletes timeout records according to a preset time interval. It iterates through each record in a pre-saved database; if the difference between the current time recorded in the record and the actual current time is less than a given threshold, for example, 10000ms, then the target is considered to have disappeared, and the record can be deleted.

[0094] In this embodiment, a global tracking ID is generated for targets whose occurrence frequency is greater than a preset first threshold. The number of matches for each global tracking ID matched by an unstable target is used as a criterion. When the number of matches for any global tracking ID matched by an unstable target is not less than a preset second threshold, the global tracking ID that is not less than the preset second threshold is selected as the final global tracking ID. This improves the robustness of the matching and avoids a chain reaction of errors caused by a single matching error.

[0095] Example 7: Based on the same concept, embodiments of this application provide a device for cross-camera multi-target tracking. Figure 4 Please refer to the schematic diagram of a device structure for cross-camera multi-target tracking provided in this application embodiment. Figure 4 The device includes: The acquisition module 401 is used to acquire images captured by multiple preset cameras at the current moment, wherein there are adjacent cameras among the preset cameras; The determination module 402 is used to determine the targets appearing in each image, their positions, and their features using a neural network model; for each target identified in an image, it determines the local ID of the target in the camera that acquired the image, and determines the number of times the target with that local ID has appeared in the image acquired by the camera; for each unstable target, it determines the distance between the unstable target and each stable target based on the target features and positions of the unstable target and each stable target, wherein the unstable target is the target whose number of appearances is less than a preset first threshold, and the stable target is the target whose number of appearances is not less than the preset first threshold; it uses a Hungarian algorithm to determine the first stable target that matches each unstable target, and for any unstable target, it determines that the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target.

[0096] In one possible implementation, the determining module 402 is further configured to update the number of times each target has appeared in all images acquired at the current time. If the number of times the target has appeared after the update is less than a preset first threshold, the target is determined as an unstable target; if the number of times the target has appeared after the update is not less than the preset first threshold, the target is determined as a stable target.

[0097] In one possible implementation, the determining module 402 is specifically used to, for each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same target as other stable targets in the image to which the unstable target belongs, then determine the distance between the unstable target and the stable target based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, determine the distance between the unstable target and the stable target as a set value.

[0098] In one possible implementation, the determining module 402 is specifically configured to determine that the first stable target and the unstable target are the same target if the distance between the unstable target and the matching first stable target is not the set value.

[0099] In one possible implementation, the determining module 402 is further configured to determine each stable target and its first feature that have been determined within a preset time period prior to the current time; for the unstable target, based on the target feature of the unstable target and the first feature of each stable target, determine the first feature similarity between the unstable target and each stable target respectively; if the first feature similarity is greater than a preset similarity threshold, then the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target; otherwise, the distance between the unstable target and the stable target is determined to be the set value; the Hungarian algorithm is used to solve for and determine a second stable target that matches the unstable target; if the distance between the unstable target and the matched second stable target is not the set value, then the second stable target and the unstable target are determined to be the same target.

[0100] In one possible implementation, the determining module 402 is specifically used to calculate the feature similarity between each first feature of the stable target and the target feature for each stable target; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0101] In one possible implementation, the determining module 402 is further configured to generate a global tracking ID for the stable target; The determining module 402 is further configured to use the global tracking ID of the stable target as the global tracking ID of the unstable target in this match, increment the number corresponding to the global tracking ID by 1, and increment the number of matches corresponding to the unstable target by 1; determine the number of matches for each global tracking ID matched by the unstable target, and when the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, select the global tracking ID that is not less than the preset second threshold as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0102] Example 8: Based on the same concept, embodiments of this application provide an electronic device that can implement the steps of a cross-camera multi-target tracking method described above. Figure 5 This application provides a schematic diagram of an electronic device structure, such as... Figure 5 As shown, it includes: processor 501, communication interface 502, memory 503 and communication bus 504, wherein processor 501, communication interface 502 and memory 503 communicate with each other through communication bus 504. The memory 503 stores a computer program. When the program is executed by the processor 501, the processor 501 performs the following steps: Images captured at the current moment by multiple preset cameras are acquired, wherein there are adjacent cameras among the preset cameras; a neural network model is used to determine the target appearing in each image, the target's location, and the target's features; For each target identified in an image, determine the local ID of the target in the camera that acquired the image, and determine the number of times the target with that local ID has appeared in the image acquired by the camera; For each unstable target, the distance between the unstable target and each stable target is determined based on the target characteristics and position of the unstable target and each stable target, wherein the unstable target is the target that has appeared less than a preset first threshold, and the stable target is the target that has appeared not less than the preset first threshold. The Hungarian algorithm is used to determine the first stable target that matches each unstable target. For any unstable target, the first stable target and the unstable target are determined to be the same target based on the distance between the unstable target and the first stable target.

[0103] In one possible implementation, the processor 501 is further configured to update the number of times each target has appeared in all images acquired at the current time; if the updated number of times the target has appeared is less than a preset first threshold, the target is determined to be an unstable target; if the updated number of times the target has appeared is not less than the preset first threshold, the target is determined to be a stable target.

[0104] In one possible implementation, the processor 501 is specifically configured to, for each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same target as other stable targets in the image to which the unstable target belongs, then determine the distance between the unstable target and the stable target based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, determine the distance between the unstable target and the stable target as a set value.

[0105] In one possible implementation, the processor 501 is specifically configured to determine that the first stable target and the unstable target are the same target if the distance between the unstable target and the matched first stable target is not the set value.

[0106] In one possible implementation, the processor 501 is further configured to: determine each stable target and its first feature identified within a preset time period prior to the current time; for the unstable target, determine the first feature similarity between the unstable target and each stable target based on the target feature of the unstable target and the first feature of each stable target; if the first feature similarity is greater than a preset similarity threshold, then the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target; otherwise, the distance between the unstable target and the stable target is determined to be the preset value; use the Hungarian algorithm to solve for and determine a second stable target that matches the unstable target; if the distance between the unstable target and the matched second stable target is not the preset value, then the second stable target and the unstable target are determined to be the same target.

[0107] In one possible implementation, the processor 501 is specifically configured to calculate the feature similarity between each first feature of the stable target and the target feature for each stable target; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0108] In one possible implementation, the processor 501 is also configured to generate a global tracking ID for the stable target; The processor 501 is further configured to use the global tracking ID of the stable target as the global tracking ID of the unstable target in this match, increment the number corresponding to the global tracking ID by 1, and increment the number of matches corresponding to the unstable target by 1; determine the number of matches for each global tracking ID matched by the unstable target, and when the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, select the global tracking ID that is not less than the preset second threshold as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0109] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. Communication interface 502 is used for communication between the aforementioned electronic device and other devices. The memory can include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0110] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0111] Example 9: Based on the same technical concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, the electronic device performs the following steps: Images captured at the current moment by multiple preset cameras are acquired, wherein there are adjacent cameras among the preset cameras; a neural network model is used to determine the target appearing in each image, the target's location, and the target's features; For each target identified in an image, determine the local ID of the target in the camera that acquired the image, and determine the number of times the target with that local ID has appeared in the image acquired by the camera; For each unstable target, the distance between the unstable target and each stable target is determined based on the target characteristics and position of the unstable target and each stable target, wherein the unstable target is the target that has appeared less than a preset first threshold, and the stable target is the target that has appeared not less than the preset first threshold. The Hungarian algorithm is used to determine the first stable target that matches each unstable target. For any unstable target, the first stable target and the unstable target are determined to be the same target based on the distance between the unstable target and the first stable target.

[0112] In one possible implementation, before determining the distance between each unstable target and each stable target based on the target characteristics and position of the unstable target and each stable target, the method further includes: For each target in all images acquired at the current moment, the number of times the target has appeared is updated. If the number of times the target has appeared after the update is less than a preset first threshold, the target is identified as an unstable target; if the number of times the target has appeared after the update is not less than the preset first threshold, the target is identified as a stable target.

[0113] In one possible implementation, determining the distance between each unstable target and each stable target, based on the target characteristics and positions of the unstable target and each stable target, includes: For each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same as other stable targets in the image to which the unstable target belongs, then the distance between the unstable target and the stable target is determined based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, the distance between the unstable target and the stable target is determined as a set value.

[0114] In one possible implementation, determining that the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target includes: If the distance between the unstable target and the matched first stable target is not the set value, then the first stable target and the unstable target are determined to be the same target.

[0115] In one possible implementation, if the distance between the unstable target and the matched first stable target is the set value, or if the first stable target of the unstable target cannot be determined using the Hungarian algorithm, the method further includes: Determine each stable target and its first feature that have been identified within a preset time period prior to the current moment; For the unstable target, based on the target features of the unstable target and the first features of each stable target, the similarity between the unstable target and the first features of each stable target is determined respectively; if the first feature similarity is greater than a preset similarity threshold, the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target, otherwise the distance between the unstable target and the stable target is determined as the set value. The Hungarian algorithm is used to determine a second stable target that matches the unstable target; if the distance between the unstable target and the matching second stable target is not the set value, then the second stable target and the unstable target are determined to be the same target.

[0116] In one possible implementation, determining the similarity between the unstable target and the first feature of each stable target based on the target feature of the unstable target and the first feature of each stable target includes: For each stable target, calculate the feature similarity between each first feature of the stable target and the target feature; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

[0117] In one possible implementation, after identifying the target as a stable target, the method further includes: Generate a global tracking ID for this stable target; After determining that the stable objective and the unstable objective are the same objective, the method further includes: Use the global tracking ID of the stable target as the global tracking ID for this match of the unstable target, increment the count corresponding to the global tracking ID by 1, and increment the match count corresponding to the unstable target by 1; The number of matches for each global tracking ID matched by the unstable target is determined. When the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, the global tracking ID that is not less than the preset second threshold is selected as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

[0118] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0119] Based on the same concept, this application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to execute any of the cross-camera multi-target tracking methods described above. Since the principle by which the above computer program product solves the problem is similar to that of a cross-camera multi-target tracking method, the implementation of the above computer program product can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0120] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] Computer programs used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing status information from the computer-readable program instructions to implement various aspects of this disclosure.

[0122] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0123] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0127] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for multi-target tracking across cameras, characterized in that, The method includes: Images captured at the current moment by multiple preset cameras are acquired, wherein there are adjacent cameras among the preset cameras; a neural network model is used to determine the target appearing in each image, the target's location, and the target's features; For each target identified in an image, determine the local ID of the target in the camera that acquired the image, and determine the number of times the target with that local ID has appeared in the image acquired by the camera; For each unstable target, the distance between the unstable target and each stable target is determined based on the target characteristics and position of the unstable target and each stable target, wherein the unstable target is the target that has appeared less than a preset first threshold, and the stable target is the target that has appeared not less than the preset first threshold. The Hungarian algorithm is used to determine the first stable target that matches each unstable target. For any unstable target, the first stable target and the unstable target are determined to be the same target based on the distance between the unstable target and the first stable target.

2. The method according to claim 1, characterized in that, Before determining the distance between each unstable target and each stable target based on the target characteristics and position of the unstable target and each stable target, the method further includes: For each target in all images acquired at the current moment, the number of times the target has appeared is updated. If the number of times the target has appeared after the update is less than a preset first threshold, the target is identified as an unstable target; if the number of times the target has appeared after the update is not less than the preset first threshold, the target is identified as a stable target.

3. The method according to claim 2, characterized in that, For each unstable target, determining the distance between the unstable target and each stable target based on the target characteristics and position of the unstable target and each stable target includes: For each unstable target, if the area captured by the camera that acquired the stable target includes the location of the unstable target at the current moment, and the stable target is not the same as other stable targets in the image to which the unstable target belongs, then the distance between the unstable target and the stable target is determined based on the target features and location of the unstable target and the target features and location of the stable target; otherwise, the distance between the unstable target and the stable target is determined as a set value.

4. The method according to claim 3, characterized in that, The step of determining that the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target includes: If the distance between the unstable target and the matched first stable target is not the set value, then the first stable target and the unstable target are determined to be the same target.

5. The method according to claim 4, characterized in that, If the distance between the unstable target and the matched first stable target is the set value, or if the first stable target of the unstable target cannot be determined using the Hungarian algorithm, the method further includes: Determine each stable target and its first feature that have been identified within a preset time period prior to the current moment; For the unstable target, based on the target features of the unstable target and the first features of each stable target, the similarity between the unstable target and the first features of each stable target is determined respectively; if the first feature similarity is greater than a preset similarity threshold, the negative value of the first feature similarity is determined as the distance between the unstable target and the stable target, otherwise the distance between the unstable target and the stable target is determined as the set value. The Hungarian algorithm is used to determine a second stable target that matches the unstable target; if the distance between the unstable target and the matching second stable target is not the set value, then the second stable target and the unstable target are determined to be the same target.

6. The method according to claim 5, characterized in that, The step of determining the similarity between the target features of the unstable target and the first features of each stable target includes: For each stable target, calculate the feature similarity between each first feature of the stable target and the target feature; and select the highest feature similarity as the first feature similarity between the unstable target and the stable target.

7. The method according to claim 4 or 5, characterized in that, After determining the target as a stable target, the method further includes: Generate a global tracking ID for this stable target; After determining that the stable objective and the unstable objective are the same objective, the method further includes: Use the global tracking ID of the stable target as the global tracking ID for this match of the unstable target, increment the count corresponding to the global tracking ID by 1, and increment the match count corresponding to the unstable target by 1; The number of matches for each global tracking ID matched by the unstable target is determined. When the number of matches for any global tracking ID matched by the unstable target is not less than a preset second threshold, the global tracking ID that is not less than the preset second threshold is selected as the final global tracking ID of the unstable target, wherein the preset second threshold is less than the preset first threshold.

8. A device for multi-target tracking across cameras, characterized in that, The device includes: An acquisition module is used to acquire images captured by multiple preset cameras at the current moment, wherein there are adjacent cameras among the preset cameras; The determination module is used to determine the targets appearing in each image, their locations, and their features using a neural network model; for each target identified in an image, it determines the local ID of the target in the camera that acquired the image, and determines the number of times the target with that local ID has appeared in the image acquired by the camera; for each unstable target, it determines the distance between the unstable target and each stable target based on the target features and locations of the unstable target and each stable target, wherein the unstable target is the target whose number of appearances is less than a preset first threshold, and the stable target is the target whose number of appearances is not less than the preset first threshold; it uses a Hungarian algorithm to determine the first stable target that matches each unstable target, and for any unstable target, it determines that the first stable target and the unstable target are the same target based on the distance between the unstable target and the matched first stable target.

9. An electronic device, characterized in that, The electronic device includes a processor for implementing the method as described in any one of claims 1-7 when executing a computer program stored in a memory.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images