Object tracking method, device, electronic device and storage medium

Abstract

The present disclosure provides an object tracking method, device, electronic device and storage medium, the method comprising: obtaining position information of a current object detected in a current frame image, and position information of a historical object detected in a historical frame image before the current frame image; generating predicted position information of the historical object in the current frame image based on the position information of the historical object; determining a neighbor topology map of at least one first object in the current object based on the position information of the current object, and determining a neighbor topology map of at least one second object in the historical object based on the predicted position information of the historical object; and updating an object tracking result based on the neighbor topology map of the first object and the neighbor topology map of the second object.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an object tracking method, an apparatus, an electronic device, and a storage medium. Background Art

[0002] Object tracking is a technology that determines the motion trajectory and motion state of an object in an image based on multiple frames of images. It can be applied to autonomous driving scenarios of intelligent driving devices (such as autonomous vehicles, vehicles equipped with assisted driving systems, robots, etc.). With the improvement of technology, intelligent driving devices are widely used, and high-precision object tracking is an important part of vehicle intelligence and automation, and is the basis of the perception, control, path planning and other modules of intelligent driving devices. Usually, an intelligent driving device can be equipped with an image acquisition device such as a laser radar to locate objects around it, track the position of the identified objects, associate the continuous detection results in time series, use the object tracking results to determine the object's motion trajectory, estimate the motion state of the detected object, and then control the driving plan of the intelligent driving device.

[0003] However, in special scenarios such as dense crowds, there are complex occlusion relationships between detected objects. When tracking objects, it is easy to incorrectly associate objects detected in different frames, resulting in incorrect object tracking results and making driving risky. Summary of the invention

[0004] The embodiments of the present disclosure at least provide an object tracking method, an electronic device, and a storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides an object tracking method, including:

[0006] Acquire the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold;

[0007] Based on the position information of the historical object, generating predicted position information of the historical object in the current frame image;

[0008] Determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map includes a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connecting edge between the first node and the second node;

[0009] The object tracking result is updated based on the neighbor topology map of the first object and the neighbor topology map of the second object.

[0010] In this aspect, the location information of the current object and the historical object is used to determine the neighbor topology map of the first object and the second object, so that object tracking is achieved using the neighbor topology map of the first object and the neighbor topology map of the second object, thereby improving the object tracking accuracy.

[0011] In a possible implementation, the generating, based on the position information of the historical object, predicted position information of the historical object in the current frame image includes:

[0012] Based on the position information of the image acquisition device when acquiring the current frame image and the position information of the image acquisition device when acquiring the historical frame image, determine the position offset vector generated by the image acquisition device from the time when the historical frame image is acquired to the time when the current frame image is acquired;

[0013] Based on the position offset vector, the position of the historical object is offset to obtain predicted position information of the historical object in the current frame image.

[0014] In this implementation, the position of the historical object is offset by using the position offset vector of the image acquisition device to obtain the predicted position information of the historical object in the current frame image, thereby improving the tracking accuracy of the historical object and the current object.

[0015] In a possible implementation, determining a neighbor topology map of a first object in the current object based on the location information of the current object includes:

[0016] Determine a neighbor object of the first object based on the location information of the current object;

[0017] Determining position features of each object in the current object based on the position information of the current object;

[0018] generating a first node of the first object based on a position feature of the first object;

[0019] generating a second node of a neighbor object of the first object based on a position feature of a neighbor object of the first object;

[0020] A connection edge connecting the first node and the second node is generated to obtain a neighbor topology graph of the first object.

[0021] This implementation enables the neighbor topology map to reflect the position features of the current object and its neighbor objects, realizes the determination of object tracking results using the position features of the current object and its neighbor objects, and improves the accuracy of object tracking.

[0022] In a possible implementation, updating the object tracking result based on the neighbor topology map of the first object and the neighbor topology map of the second object includes:

[0023] For each first object, determining a similarity between a neighbor topology map of the first object and a neighbor topology map of each second object;

[0024] The object tracking result is updated based on the determined similarity.

[0025] In this implementation, the object tracking result is determined by determining the similarity of the neighbor topology graphs, and the similarity between the graphs is used to replace the similarity between the objects, so that the obtained tracking result is more accurate.

[0026] In a possible implementation, determining the similarity between a neighbor topology map of a first object and a neighbor topology map of a second object includes:

[0027] When the Euclidean distance between the first object and the second object is less than or equal to a preset threshold, determining a first similarity between a node of the first object and a node of the second object based on a neighbor topology graph of the first object and a neighbor topology graph of the second object;

[0028] Determine, based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, a second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object;

[0029] Based on the first similarity and the second similarity, a similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is determined.

[0030] This implementation makes the similarity between neighbor topology graphs determined based on the first similarity and the second similarity more reliable, thereby improving the accuracy of object tracking, by determining a first similarity between nodes of a first object and nodes of a second object, and a second similarity between nodes of a neighbor object of the first object and nodes of a neighbor object of the second object.

[0031] In a possible implementation, determining the similarity between a neighbor topology map of a first object and a neighbor topology map of a second object includes:

[0032] When the Euclidean distance between the first object and the second object is greater than a preset threshold, it is determined that the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is 0.

[0033] In this implementation, the similarity between the neighbor topological maps of the first object and the second object whose Euclidean distance is greater than a preset threshold is set to 0, thereby reducing the amount of calculation required for calculating the similarity and improving the efficiency of object tracking.

[0034] In a possible implementation, determining a first similarity between a node of the first object and a node of the second object based on the neighbor topology graph of the first object and the neighbor topology graph of the second object includes:

[0035] The first similarity is determined based on a difference between a feature vector corresponding to the position feature of the first object and a feature vector corresponding to the position feature of the second object.

[0036] In this implementation, the first similarity is determined by calculating the difference between the feature vectors corresponding to the position features, so that the first similarity can reflect the difference between the node of the current object and the node of the historical object, thereby improving the accuracy of the first similarity.

[0037] In a possible implementation, determining the second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object based on the neighbor topology graph of the first object and the neighbor topology graph of the second object includes:

[0038] For a first neighbor object, based on a neighbor topological map of the first object and a neighbor topological map of the second object, determining a third similarity between the first neighbor object and each neighbor object of the second object; the first neighbor object is any one of the neighbor objects of the first object;

[0039] Based on the determined third similarity, selecting a target neighbor object matching the first neighbor object from each neighbor object of the second object;

[0040] Based on the third similarity between each first neighbor object and the target neighbor object matched therewith, a second similarity between the node of the neighbor object of the first object and the node of the neighbor object of the second object is determined.

[0041] In this implementation, the target neighbor object matching the first neighbor object among the neighbor objects of the second object is determined by the determined third similarity, and the second similarity is then determined according to the third similarity between each first neighbor object and the target neighbor object matching it, thereby improving the accuracy of the second similarity.

[0042] In a possible implementation, updating the object tracking result based on the determined similarity includes:

[0043] For a first object, based on the determined similarity, determining a matching result between the first object and each of the second objects;

[0044] When the first object matches the second object, the object tracking result corresponding to the matched second object is updated using the position information of the first object.

[0045] In this implementation, the first object and the second object are matched based on similarity. When the first object matches the second object, the position information of the first object is used to update the object tracking result corresponding to the matched second object, thereby achieving object tracking based on the similarity between the objects.

[0046] In a possible implementation, the method further includes:

[0047] In the case that the first object is not matched to the second object, the position information of the first object is used to establish an object tracking result corresponding to the first object.

[0048] In this implementation, by establishing the object tracking result of the first object, the tracking of the newly detected object is achieved.

[0049] In a possible implementation, the method further includes:

[0050] In response to the existence of a second object that is not matched to the first object, it is determined whether to retain or clear the object tracking result of the second object based on the time when the second object was last detected and the current time.

[0051] In this implementation, whether to retain or clear the object tracking result of the second object is determined based on the time when the second object was last detected and the current time, thereby saving computing resources.

[0052] In a possible implementation, the method further includes:

[0053] Based on the updated object tracking result, an intelligent driving device equipped with an image acquisition device for acquiring the current frame image and the historical frame images is controlled.

[0054] In this implementation, the safety of the driving plan can be improved by controlling the intelligent driving device equipped with the above-mentioned image acquisition device through the above-mentioned determined object tracking results.

[0055] In a second aspect, an embodiment of the present disclosure further provides an object tracking device, including:

[0056] An acquisition module, used to acquire the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold;

[0057] A generating module, configured to generate predicted position information of the historical object in the current frame image based on the position information of the historical object;

[0058] A determination module, configured to determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and to determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map comprises a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connection edge between the first node and the second node;

[0059] An updating module is used to update the object tracking result based on the neighbor topology map of the first object and the neighbor topology map of the second object.

[0060] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0061] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0062] For a description of the effects of the above-mentioned object tracking device, electronic device, and computer-readable storage medium, please refer to the description of the object tracking method, which will not be repeated here.

[0063] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0065] Figure 1 A flow chart of an object tracking method provided by an embodiment of the present disclosure is shown;

[0066] Figure 2 A flowchart of generating a neighbor topology map provided by an embodiment of the present disclosure is shown;

[0067] Figure 3 A schematic diagram of an object tracking device provided by an embodiment of the present disclosure is shown;

[0068] Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0070] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0071] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0072] In view of the defect of low object tracking accuracy in dense scenes in the prior art, the embodiments of the present disclosure provide an object tracking method, device, electronic device and computer-readable storage medium. The embodiments of the present disclosure first obtain the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold, and then, based on the position information of the historical object, the predicted position information of the historical object in the current frame image is generated, and then, based on the position information of the current object, a neighbor topology map of at least one first object in the current object is determined, and based on the predicted position information of the historical object, a neighbor topology map of at least one second object in the historical object is determined; wherein the neighbor topology map includes a first node representing the position feature of the corresponding object, a second node representing the position feature of the neighbor object of the corresponding object, and a connecting edge between the first node and the second node, and finally, based on the neighbor topology map of the first object and the neighbor topology map of the second object, the object tracking result is updated. The present disclosure uses the location information of the current object and the historical object to determine the neighbor topology map of the first object and the second object, realizes object tracking using the neighbor topology map of the first object and the neighbor topology map of the second object, and improves the object tracking accuracy.

[0073] The object tracking method, device, electronic device, and computer-readable storage medium disclosed in the present disclosure are described below through specific embodiments.

[0074] like Figure 1 As shown, the embodiment of the present disclosure discloses an object tracking method, which can be applied to an electronic device with computing capabilities, such as a server, etc. Specifically, the object tracking method may include the following steps:

[0075] S110, obtaining position information of a current object detected in a current frame image, and position information of a historical object detected in a historical frame image before the current frame image; wherein a time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold.

[0076] The current frame image may be acquired by an image acquisition device, which may be a monocular camera, a multi-camera camera, a laser radar, an acoustic radar, etc. The acquired image may be point cloud data, a depth image, a normal image, etc. The image acquisition device may be deployed on an intelligent driving device, which may be an autonomous driving vehicle, a vehicle equipped with an assisted driving system, or a robot, etc. The image acquisition device of this embodiment takes a laser radar as an example. The laser radar can acquire point cloud data of surrounding objects and determine the position information of each detected object based on the point cloud data. In addition to the coordinates of the object in a preset coordinate system, the position information may also include the posture information of the object, such as the length, width, height, and deflection angle of the object.

[0077] The current frame image and the historical frame image can be two consecutive frames, that is, the historical frame image is the previous frame image of the current frame image, the current object is the object that appears in the detection result of the current frame point cloud data, and the historical object is the object that appears in the detection result of the historical frame point cloud data. The historical frame image and the current frame image can also be two non-consecutive frames, and the time interval between the two frames is less than or equal to a preset threshold to ensure that the two frames can be used for object tracking.

[0078] In the detection results of the laser radar, at least one object may appear, and the same object may be detected in the current frame image and the historical frame image at the same time. For example, 60 objects are detected in the historical frame image, and 70 objects are detected in the current frame image, among which 40 objects may be detected in both the historical frame image and the current frame image, 20 objects in the historical frame image are not detected in the current frame image, and 30 objects in the current frame image are not detected in the historical frame image. For objects detected in both the historical frame image and the current frame image, their motion path can be determined based on their position information in the historical frame image and the current frame image, and the object can be tracked.

[0079] To achieve object tracking, it is necessary to first determine whether the objects in the historical frame and the current frame can match each other, that is, whether they are the same object.

[0080] S120: Generate predicted position information of the historical object in the current frame image based on the position information of the historical object.

[0081] The position information of the historical object mentioned above is measured based on the position of the image acquisition device when the historical frame image is acquired, and cannot represent the position information of the historical object in the current frame image. Therefore, the position information of the historical object can be used to predict the position information of the historical object in the current frame image to obtain the predicted position information.

[0082] The above-mentioned predicted position information is the prediction result of the historical object in the current frame image, and can be determined by using the motion state information of the historical object in the historical frame image and / or the motion state information of the image acquisition device.

[0083] Exemplarily, the predicted position information of the historical object in the current frame image may be generated by the following steps:

[0084] Based on the position information of the image acquisition device when acquiring the current frame image and the position information of the image acquisition device when acquiring the historical frame image, determine the position offset vector generated by the image acquisition device from the time when the historical frame image is acquired to the time when the current frame image is acquired;

[0085] Based on the position offset vector, the position of the historical object is offset to obtain predicted position information of the historical object in the current frame image.

[0086] Usually, the coordinate system used in the laser radar detection result is based on itself as the origin. Since the intelligent driving device is usually in motion, the image acquisition device on the intelligent driving device is also in motion, which results in the coordinate system of the detection result of the historical frame image being inconsistent with the detection result of the current frame image. However, the position offset vector can be determined by the position information when the image acquisition device collects the current frame image and the position information when collecting the historical frame image, thereby determining the offset of the origin of the coordinate system in the process of the two frames, and using the position offset vector to offset the position of the historical object to obtain the predicted position information of the historical object in the current frame image.

[0087] In this way, by using the position offset vector of the target vehicle to offset the position information of the historical object, the predicted position information of the historical object in the current frame image is obtained, thereby improving the tracking accuracy of the historical object and the current object.

[0088] S130. Determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map includes a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connecting edge between the first node and the second node.

[0089] The current object may be an object detected in the current frame image, and there may be multiple objects. Based on the position information of each current object, a neighbor topology map of at least one first object in the current object may be determined. The first object may be any object in the current object.

[0090] The above-mentioned neighbor topology map can be composed of nodes and connecting edges. Each object in the current object can have its corresponding neighbor topology map, wherein there is a first node in the neighbor topology map, which is used to represent the location characteristics of the first object or the second object. The multiple second nodes corresponding to the first node are used to represent the location characteristics of the neighbor nodes corresponding to the first node. The neighbor node of a node is the neighbor object of the object corresponding to the node. Here, the location characteristics can be determined according to the location information corresponding to the node.

[0091] In a possible implementation manner, the following steps may be used to determine a neighbor topology map of a first object in the current object:

[0092] Based on the location information of the current object, determine the neighbor objects of the first object; based on the location information of the current object, determine the location features of each object in the current object; based on the location features of the first object, generate a first node of the first object; based on the location features of the neighbor objects of the first object, generate a second node of the neighbor objects of the first object; generate a connection edge connecting the first node and the second node, and obtain a neighbor topology map of the first object.

[0093] In this implementation, the neighbor object of the first object can be determined based on the location information of the current object. For example, the Euclidean distance between the first object and other objects in the current object can be determined. If the determined Euclidean distance is less than or equal to a preset threshold, the other object can be used as the neighbor object of the first object.

[0094] At the same time, the position features of each first object in the current object can also be determined based on the position information of the current object. Specifically, the data of each dimension in the position information can be extracted to obtain the position features of each first object, and then they can be combined into an N-dimensional feature vector to obtain a feature vector corresponding to the position feature. Exemplarily, if the position information includes coordinate information, size information and deflection angle, the position feature can include the x-axis feature, y-axis feature, and z-axis feature of the coordinate system, as well as the length feature length, width feature width, height feature height of the size information, and deflection angle feature yaw.

[0095] Afterwards, based on the determined location features of each current object, a node corresponding to the first object can be generated, and based on the location features of the neighbor objects of the first object, a node corresponding to the first object can be generated. In the neighbor topology graph, the location information of each node matches its corresponding location feature.

[0096] After generating the nodes in the neighbor topology graph, connection edges connecting the nodes can be generated. The connection edges connect the first node to its corresponding second node respectively. There are no connection edges between the second nodes. Exemplarily, the connection edges can be directed edges from the first node to the second node. If a first node has K second nodes, its corresponding neighbor topology graph contains K+1 nodes and K connection edges.

[0097] This implementation enables the neighbor topology map to reflect the position features of the first object and its neighbor objects, realizes the determination of object tracking results using the position features of the first object and its neighbor objects, and improves the accuracy of object tracking.

[0098] For example, we can Represents the detection result of the tth frame, where I t is the number of detected objects, is the location information of the i-th object, denoted as in It can be the original point cloud data corresponding to the detection result.

[0099] The position feature can be Represents the center point coordinates, length, width, height and orientation angle of the i-th object;

[0100] use Represents a set of location features of an object and its neighboring objects, where is the location feature of the object, is the position feature of the kth neighbor object of the detection result. To simplify the notation, we will Recorded as the position feature of its own 0th neighbor object, the above set can be recorded as

[0101] Will The topological relationship between the 0th neighbor object and its neighbor objects in the set is represented by a directed graph, and the neighbor topology graph is obtained, which is recorded as It has K+1 vertices and K directed edges, all of which are directed edges from the 0th neighbor node to other nodes. Since the total number of nodes is K+1, the number of edges is K. is a node set, Indicates the i k The features of the vertices, here the features are the position features, which can be represented by (x, y, z, l, w, h, yaw), or they can be obtained by a neural network. It can be extracted from the original point cloud data, or it can be a combination of the above two features. For convenience, (x, y, z, l, w, h, yaw) can be used as an example for description.

[0102] The set of edges is defined as where f diff (...) is the function of the directed edge calculated based on the node. Since the node is generally an N-dimensional vector, for example, two vectors can be directly subtracted to obtain the vector f connecting the edge. diff (…).

[0103] Similarly, the neighbor topology graph of the historical object can be determined in a similar manner.

[0104] like Figure 2 As shown, a flowchart for generating a neighbor topology map provided in an embodiment of the present disclosure can first generate each node according to the location characteristics of each object, and determine the neighbor objects of each object respectively, calculate the connection edges according to the location characteristics of the neighbor objects and the location characteristics of the current object, and splice the generated nodes and the connection edges to obtain a neighbor topology map.

[0105] S140: Update the object tracking result based on the neighbor topology map of the first object and the neighbor topology map of the second object.

[0106] Based on the above-mentioned neighbor topology map, the first object and the second object can be matched to determine the corresponding relationship between the first object and the second object. Exemplarily, the corresponding relationship may include matching, addition and disappearance, wherein, if a first object and a second object are the same object, the relationship between the first object and the second object may be matching; if a first object and each second object are not the same object, the first object is a newly added object, and the corresponding relationship between it and the second object may be addition; if a second object and each first object are not the same object, the corresponding relationship between the second object and the first object may be disappearance.

[0107] After the corresponding relationship between the first object and the second object is determined, the tracking results of each object can be determined according to the determined corresponding relationship, and the object tracking results can be updated.

[0108] Specifically, the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object may be determined; and then the object tracking result may be updated based on the determined similarity.

[0109] In this embodiment, the correspondence between the first object and the second object may be determined according to the similarity between the neighbor topological maps of the first object and the second object, and then the object tracking result may be updated according to the determined correspondence.

[0110] In a possible implementation, the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object may be determined by using the following steps:

[0111] When the Euclidean distance between the first object and the second object is less than or equal to a preset threshold, determining a first similarity between a node of the first object and a node of the second object based on a neighbor topology graph of the first object and a neighbor topology graph of the second object;

[0112] Determine, based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, a second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object;

[0113] Based on the first similarity and the second similarity, a similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is determined.

[0114] Since the time interval between the current frame image and the historical frame image is short, if a first object and a second object are the same object, the similarity between their neighbor topological maps should be high, and the Euclidean distance between them should be close. If the Euclidean distance is greater than a preset threshold, it is considered that they are not the same object, and the similarity is set to 0. If the Euclidean distance is less than or equal to the preset threshold, it can be considered that the first object and the second object may be in a matching relationship. At this time, the first similarity between the node of the first object and the node of the second object can be determined based on the neighbor topological map of the first object and the second object. Exemplarily, the feature vector difference between the position feature of the first object and the position feature of the second object can be calculated, the determined difference is modulo processed, and then multiplied by -1 to obtain the first similarity.

[0115] At the same time, the second similarity between the node of the neighbor object of the first object and the node of the neighbor object of the second object can be determined according to the neighbor topology map of the first object and the neighbor topology map of the second object. and the neighbor topology of the second object Only the neighbor objects are considered (i.e., and ), we can assume There are x neighbors, There are y neighbors, and the two sets of neighbor objects form a bipartite graph.

[0116] By calculating the first similarity, a similarity matrix is ​​determined to obtain an x*y neighbor similarity matrix neighbor_matrix, each element in the neighbor similarity matrix is ​​the third similarity between the nodes of the neighbor object of the first object and the neighbor object of the second object corresponding to it, and the neighbor similarity matrix is ​​solved by the Hungarian matching algorithm or other matching algorithms to obtain a set of optimal matching relationships neighbor_match. Exemplarily, a pair of neighbor nodes with the highest third similarity can be taken as neighbor_match, and neighbor_match includes the neighbor object of the first object and the neighbor object of the second object matched therewith, that is, it is determined whether the neighbor object of the first object and the neighbor object of the second object are the same object. After obtaining neighbor_match, the third similarity between neighbor_matches can be determined, and each similarity that can be obtained is added to obtain the second similarity.

[0117] After obtaining each first similarity and each second similarity, the obtained first similarity and second similarity may be added together to obtain similarity_matrix(i,j) between the neighbor topology graphs of the i-th first object and the j-th second object.

[0118] The above similarities can form a similarity matrix similarity_matrix. By solving the similarity matrix, the corresponding relationship between each first object and the second object can be obtained. After that, the object tracking result can be updated according to the determined corresponding relationship.

[0119] The similarity matrix similarity_matrix obtained can be solved by using algorithms such as greedy nearest neighbor and Hungarian matching, so as to obtain the corresponding relationship between the current object and the historical object. The specific method can be the same as the method of determining neighbor_match.

[0120] Finally, after determining the correspondence between the first object and the second object, the obtained correspondence may be used as a matching result between the first object and the second object. Different methods may be used to update the object tracking result for different matching results.

[0121] Specifically, for the first object that matches the second object (i.e., the corresponding relationship is matched), the position information of the first object can be used to update the object tracking result corresponding to the matched second object; for the first object that does not match the second object (i.e., the corresponding relationship is added), a new object tracking for the first object can be created, and the position information of the first object can be used as its corresponding object tracking result; if a second object that does not match the first object is detected (i.e., the corresponding result is disappeared), it can be determined based on the time when the second object was last detected and the current time whether the duration for which the second object has not been detected reaches a preset time threshold, thereby determining whether to retain or clear the object tracking result of the second object.

[0122] After obtaining the updated object tracking results, the intelligent driving device equipped with the image acquisition device for acquiring the above-mentioned current frame image and historical frame image can be controlled based on the updated object tracking results. For example, the driving route, driving speed, etc. can be adjusted when there are detected objects on the expected driving route.

[0123] In this way, the neighbor topology map of the first object and the neighbor topology map of the second object are determined using the location information of the first object and the location information of the second object, thereby realizing object tracking using the neighbor topology map of the first object and the neighbor topology map of the second object, thereby improving the accuracy of object tracking.

[0124] Corresponding to the above object tracking method, the present disclosure also discloses an object tracking device, each module in the device can implement each step in the object tracking method of each embodiment above, and can achieve the same beneficial effect, so the same parts will not be repeated here. Specifically, Figure 3 As shown, the object tracking device includes:

[0125] The acquisition module 310 is used to acquire the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold;

[0126] A generating module 320, configured to generate predicted position information of the historical object in the current frame image based on the position information of the historical object;

[0127] A determination module 330 is used to determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and to determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map includes a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connection edge between the first node and the second node;

[0128] The updating module 340 is configured to update the object tracking result based on the neighbor topology map of the first object and the neighbor topology map of the second object.

[0129] In a possible implementation manner, the generating module 320 is specifically configured to:

[0130] Based on the position information of the image acquisition device when acquiring the current frame image and the position information of the image acquisition device when acquiring the historical frame image, determine the position offset vector generated by the image acquisition device from the time when the historical frame image is acquired to the time when the current frame image is acquired;

[0131] Based on the position offset vector, the position of the historical object is offset to obtain predicted position information of the historical object in the current frame image.

[0132] In a possible implementation manner, when determining a neighbor topology map of a first object in the current object based on the location information of the current object, the determination module 330 is configured to:

[0133] Determine a neighbor object of the first object based on the location information of the current object;

[0134] Determining position features of each object in the current object based on the position information of the current object;

[0135] generating a first node of the first object based on a position feature of the first object;

[0136] generating a second node of a neighbor object of the first object based on a position feature of a neighbor object of the first object;

[0137] A connection edge connecting the first node and the second node is generated to obtain a neighbor topology graph of the first object.

[0138] In a possible implementation, the updating module 340 is specifically configured to:

[0139] For each first object, determining a similarity between a neighbor topology map of the first object and a neighbor topology map of each second object;

[0140] The object tracking result is updated based on the determined similarity.

[0141] In a possible implementation, when determining the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object, the updating module 340 is configured to:

[0142] When the Euclidean distance between the first object and the second object is less than or equal to a preset threshold, determining a first similarity between a node of the first object and a node of the second object based on a neighbor topology graph of the first object and a neighbor topology graph of the second object;

[0143] Determine, based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, a second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object;

[0144] Based on the first similarity and the second similarity, a similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is determined.

[0145] In a possible implementation, when determining the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object, the updating module 340 is configured to:

[0146] When the Euclidean distance between the first object and the second object is greater than a preset threshold, it is determined that the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is 0.

[0147] In a possible implementation, when determining the first similarity between a node of the first object and a node of the second object based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, the updating module 340 is configured to:

[0148] The first similarity is determined based on a difference between a feature vector corresponding to the position feature of the first object and a feature vector corresponding to the position feature of the second object.

[0149] In a possible implementation, when determining the second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, the updating module 340 is configured to:

[0150] For a first neighbor object, based on a neighbor topological map of the first object and a neighbor topological map of the second object, determining a third similarity between the first neighbor object and each neighbor object of the second object; the first neighbor object is any one of the neighbor objects of the first object;

[0151] Based on the determined third similarity, selecting a target neighbor object matching the first neighbor object from each neighbor object of the second object;

[0152] Based on the third similarity between each first neighbor object and the target neighbor object matched therewith, a second similarity between the node of the neighbor object of the first object and the node of the neighbor object of the second object is determined.

[0153] In a possible implementation, when updating the object tracking result based on the determined similarity, the updating module 340 is configured to:

[0154] For a first object, based on the determined similarity, determining a matching result between the first object and each of the second objects;

[0155] When the first object matches the second object, the object tracking result corresponding to the matched second object is updated using the position information of the first object.

[0156] In a possible implementation, when updating the object tracking result based on the determined similarity, the updating module 340 is further configured to:

[0157] In the case that the first object is not matched to the second object, the position information of the first object is used to establish an object tracking result corresponding to the first object.

[0158] In a possible implementation, when updating the object tracking result based on the determined similarity, the updating module 340 is further configured to:

[0159] In response to the existence of a second object that is not matched to the first object, it is determined whether to retain or clear the object tracking result of the second object based on the time when the second object was last detected and the current time.

[0160] In a possible implementation manner, the device further includes a control module, configured to:

[0161] Based on the updated object tracking result, an intelligent driving device equipped with an image acquisition device for acquiring the current frame image and the historical frame images is controlled.

[0162] Corresponding to the above object tracking method, the embodiment of the present disclosure further provides an electronic device 400, such as Figure 4 FIG. 4 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present disclosure, including:

[0163] Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including internal memory 421 and external memory 422; the internal memory 421 is also called internal memory, which is used to temporarily store the operation data in the processor 41 and the data exchanged with the external memory 422 such as a hard disk. The processor 41 exchanges data with the external memory 422 through the internal memory 421. When the electronic device 400 is running, the processor 41 communicates with the memory 42 through the bus 43, so that the processor 41 executes the following instructions:

[0164] Acquire the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold;

[0165] Based on the position information of the historical object, generating predicted position information of the historical object in the current frame image;

[0166] Determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map includes a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connecting edge between the first node and the second node;

[0167] The object tracking result is updated based on the neighbor topology map of the first object and the neighbor topology map of the second object.

[0168] The present disclosure also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the object tracking method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0169] The computer program product of the object tracking method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the object tracking method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0170] The present disclosure also provides a computer program, which implements any one of the methods of the aforementioned embodiments when executed by a processor. The computer program product can be implemented in hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0171] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0174] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0175] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An object tracking method, It is characterized in that include: Acquire the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold; Based on the position information of the historical object, generating predicted position information of the historical object in the current frame image; Determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map includes a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connecting edge between the first node and the second node; updating an object tracking result based on a neighbor topology map of the first object and a neighbor topology map of the second object; The updating of the object tracking result based on the neighbor topology map of the first object and the neighbor topology map of the second object includes: For each first object, determining a similarity between a neighbor topology map of the first object and a neighbor topology map of each second object; The object tracking result is updated based on the determined similarity.

2. The method according to claim 1, It is characterized in that The generating, based on the position information of the historical object, predicted position information of the historical object in the current frame image comprises: Based on the position information of the image acquisition device when acquiring the current frame image and the position information of the image acquisition device when acquiring the historical frame image, determine the position offset vector generated by the image acquisition device from the time when the historical frame image is acquired to the time when the current frame image is acquired; Based on the position offset vector, the position of the historical object is offset to obtain predicted position information of the historical object in the current frame image.

3. The method according to claim 1 or 2, It is characterized in that Determining a neighbor topology map of a first object in the current object based on the location information of the current object includes: Determine a neighbor object of the first object based on the location information of the current object; Determining position features of each object in the current object based on the position information of the current object; generating a first node of the first object based on a position feature of the first object; generating a second node of a neighbor object of the first object based on a position feature of a neighbor object of the first object; A connection edge connecting the first node and the second node is generated to obtain a neighbor topology graph of the first object.

4. The method according to claim 1, It is characterized in that Determining a similarity between a neighbor topology map of a first object and a neighbor topology map of a second object includes: When the Euclidean distance between the first object and the second object is less than or equal to a preset threshold, determining a first similarity between a node of the first object and a node of the second object based on a neighbor topology graph of the first object and a neighbor topology graph of the second object; Determine, based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, a second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object; Based on the first similarity and the second similarity, a similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is determined.

5. The method according to claim 1, It is characterized in that Determining a similarity between a neighbor topology map of a first object and a neighbor topology map of a second object includes: When the Euclidean distance between the first object and the second object is greater than a preset threshold, it is determined that the similarity between the neighbor topology map of the first object and the neighbor topology map of the second object is 0.

6. The method according to claim 4, It is characterized in that The determining, based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, a first similarity between a node of the first object and a node of the second object comprises: The first similarity is determined based on a difference between a feature vector corresponding to the position feature of the first object and a feature vector corresponding to the position feature of the second object.

7. The method according to claim 4 or 6, It is characterized in that The determining, based on the neighbor topology graph of the first object and the neighbor topology graph of the second object, a second similarity between a node of a neighbor object of the first object and a node of a neighbor object of the second object comprises: For a first neighbor object, based on a neighbor topological map of the first object and a neighbor topological map of the second object, determining a third similarity between the first neighbor object and each neighbor object of the second object; the first neighbor object is any one of the neighbor objects of the first object; Based on the determined third similarity, selecting a target neighbor object matching the first neighbor object from each neighbor object of the second object; Based on the third similarity between each first neighbor object and the target neighbor object matched therewith, a second similarity between the node of the neighbor object of the first object and the node of the neighbor object of the second object is determined.

8. The method according to claim 1, It is characterized in that The updating of the object tracking result based on the determined similarity includes: For a first object, based on the determined similarity, determining a matching result between the first object and each of the second objects; When the first object matches the second object, the object tracking result corresponding to the matched second object is updated using the position information of the first object.

9. The method according to claim 8, It is characterized in that The method further comprises: In the case that the first object is not matched to the second object, the position information of the first object is used to establish an object tracking result corresponding to the first object.

10. The method according to claim 8, It is characterized in that The method further comprises: In response to the existence of a second object that is not matched to the first object, it is determined whether to retain or clear the object tracking result of the second object based on the time when the second object was last detected and the current time.

11. The method according to claim 1, It is characterized in that The method further comprises: Based on the updated object tracking result, an intelligent driving device equipped with an image acquisition device for acquiring the current frame image and the historical frame images is controlled.

12. An object tracking device, It is characterized in that include: An acquisition module, used to acquire the position information of the current object detected in the current frame image, and the position information of the historical object detected in the historical frame image before the current frame image; wherein the time interval between the acquisition of the historical frame image and the current frame image is less than or equal to a preset threshold; A generating module, configured to generate predicted position information of the historical object in the current frame image based on the position information of the historical object; A determination module, configured to determine a neighbor topology map of at least one first object in the current object based on the location information of the current object, and to determine a neighbor topology map of at least one second object in the historical object based on the predicted location information of the historical object; wherein the neighbor topology map comprises a first node representing a location feature of a corresponding object, a second node representing a location feature of a neighbor object of the corresponding object, and a connection edge between the first node and the second node; An updating module, configured to update an object tracking result based on a neighbor topology map of the first object and a neighbor topology map of the second object; The update module is specifically used for: For each first object, determining a similarity between a neighbor topology map of the first object and a neighbor topology map of each second object; The object tracking result is updated based on the determined similarity.

13. An electronic device, It is characterized in that include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the object tracking method as described in any one of claims 1 to 11 are performed.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the object tracking method according to any one of claims 1 to 11 are executed.

Images