Target matching method, device, equipment and storage medium

Abstract

Embodiments of the present disclosure relate to a target matching method, apparatus, device, and storage medium. The method includes: obtaining a detection target and a prediction target corresponding to an image to be processed; performing multi-stage matching based on the detection target and the prediction target, where different matching strategies are used in different stages. The embodiments of the present disclosure can preferably improve problems such as incomplete matching that are prone to occur in existing single-stage matching, and can effectively improve the matching accuracy.

Description

[0001] Cross - reference to related applications

[0002] This disclosure claims the priority of a Chinese patent application with the application number 2021114046049, titled "Target Matching Method, Device, Equipment and Storage Medium", filed with the Chinese Patent Office on November 24, 2021, the entire content of which is incorporated herein by reference. Technical field

[0003] This disclosure relates to the field of image processing technologies, and particularly to a target matching method, device, equipment and storage medium. Background art

[0004] In technologies such as target tracking, in order to determine the same target object in a video, target matching is usually required. That is, by matching the detected target with the predicted target, if the match is successful, it is confirmed that they correspond to the same target object, so as to achieve the tracking of the same target object. The accuracy of target matching will directly affect the target tracking result, and the accuracy of the target matching algorithms adopted in the related technologies still needs to be improved. Summary of the invention

[0005] In order to solve the above - mentioned technical problems or at least partially solve the above - mentioned technical problems, this disclosure provides a target matching method, device, equipment and storage medium.

[0006] In a first aspect, an embodiment of this disclosure provides a target matching method, including: obtaining a detected target and a predicted target corresponding to an image to be processed; performing multi - stage matching based on the detected target and the predicted target, where different stages adopt different matching strategies.

[0007] In a second aspect, an embodiment of this disclosure provides a target matching device, including: an obtaining module, configured to obtain a detected target and a predicted target corresponding to an image to be processed; a multi - stage matching module, configured to perform multi - stage matching based on the detected target and the predicted target; where different stages adopt different matching strategies.

[0008] In a third aspect, an embodiment of this disclosure provides an electronic device, where the electronic device includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the target matching method provided in the embodiment of this disclosure.

[0009] In a fourth aspect, an embodiment of this disclosure further provides a computer - readable storage medium, where the storage medium stores a computer program, and the computer program is used to execute the target matching method provided in the embodiment of this disclosure.

[0010] The above technical solution provided by the embodiments of the present disclosure may first obtain a detection target and a prediction target corresponding to an image to be processed; then perform multi-stage matching based on the detection target and the prediction target. Since multi-stage matching is adopted and different matching strategies are used in different stages, it can better improve problems such as incomplete matching that are prone to occur in existing single-stage matching, and can effectively improve the matching accuracy.

[0011] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0013] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0014] Figure 1 It is a schematic flowchart of a target matching method provided by an embodiment of the present disclosure;

[0015] Figure 2 It is a schematic diagram of a common relationship between a detection target and a prediction target provided by an embodiment of the present disclosure;

[0016] Figure 3 It is a schematic diagram of a boundary distance metric provided by an embodiment of the present disclosure;

[0017] Figure 4 It is a schematic diagram of a combined attribute metric provided by an embodiment of the present disclosure;

[0018] Figure 5 It is a schematic flowchart of a two-stage target matching method provided by an embodiment of the present disclosure;

[0019] Figure 6 It is a schematic diagram of a two-stage target matching provided by an embodiment of the present disclosure;

[0020] Figure 7 It is a schematic diagram of a multi-target tracking structure provided by an embodiment of the present disclosure;

[0021] Figure 8 It is a schematic diagram of the structure of a target matching device provided by an embodiment of the present disclosure;

[0022] Figure 9 Schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0023] In order to more clearly understand the above-mentioned objects, features, and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0024] Many specific details are set forth in the following description in order to provide a thorough understanding of the present disclosure, but the present disclosure may be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the present disclosure, rather than all embodiments.

[0025] Figure 1 Flowchart of a target matching method provided by an embodiment of the present disclosure. This method can be executed by a target matching device, where the device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 1 shown, the method mainly includes the following steps S102 to step S104:

[0026] Step S102, obtain a detection target and a prediction target corresponding to the image to be processed.

[0027] In some implementation manners, the above-mentioned image to be processed may be a frame image in a video to be processed. For example, each frame image in the video to be processed may be sequentially used as the image to be processed, or frame images may be extracted from the video to be processed at a specified interval, and the extracted frame images are used as the images to be processed. There may be only one target object in the image to be processed. In this case, both the detection target and the prediction target are one; there may also be multiple target objects in the image to be processed. In this case, both the detection target and the prediction target are multiple, and pairwise target matching needs to be performed between any detection target and any prediction target to achieve multi-target matching. The embodiments of the present disclosure do not limit the number of targets, nor do they limit the target categories. For example, the target may be a person or a vehicle, etc.

[0028] In practical applications, a target detection algorithm may be used to perform target detection on the image to be processed to obtain a detection target; and the target trajectory before the acquisition time of the image to be processed is obtained, and the image to be processed is predicted based on the target trajectory to obtain a prediction target. Exemplarily, a 3D target detector may be used to implement target detection, and a Kalman filter may be used to predict based on the target trajectory at the previous moment of the image to be processed to obtain a prediction target.

[0029] Step S104, perform multi-stage matching based on the detection target and the prediction target, where different matching strategies are used in different stages.

[0030] The purpose of matching is to determine the detection target and the prediction target that belong to the same target object. In some embodiments, the above multi-stage matching may be as follows: the objects to be matched in the first stage are all detection targets and all prediction targets, and the objects to be matched in non-first stages are the detection targets and prediction targets that did not match successfully in the previous stage. The similarity measurement algorithms and / or similarity screening thresholds in the matching strategies adopted in different stages are different; among them, the similarity measurement algorithm is used to calculate the similarity between the detection target to be matched and the prediction target, and the similarity between the successfully matched detection target and prediction target is higher than the similarity screening threshold. Through the above method, different matching strategies can be adopted in stages for target matching, thus greatly reducing problems such as incorrect target matching and incomplete target matching. Specifically, in a single stage, not only is it easy to have incorrect matching situations, but it is also easy for detection targets and prediction targets that should correspond to the same target object not to be successfully matched, resulting in incomplete target matching (which can also be simply referred to as incomplete target matching or missing matching targets). The multi-stage matching method provided by the embodiments of the present disclosure can effectively improve the above problems and comprehensively improve the accuracy of target matching.

[0031] In addition, in some embodiments, the strictness of the matching strategy in the previous stage is higher than that of the matching strategy in the subsequent stage. For ease of understanding, taking two-stage matching as an example, for detection target A and prediction target a, assuming the probability of successful matching in the first stage is 60%, then the probability of successful matching in the second stage may be 75%; in other words, for a certain detection target and prediction target, assuming they did not match successfully in the first stage, they may match successfully in the second stage because the strictness of the matching strategy in the second stage is lower than that of the matching strategy in the first stage. In the embodiments of the present disclosure in multi-stage matching, the most likely matching detection targets and prediction targets can be first screened out by adopting a strict matching strategy in the previous stage, and the accuracy of the matching result is also the highest, fully ensuring the matching accuracy. Then, on this basis, other matching strategies are used to match the remaining unmatched detection targets and prediction targets, thereby improving problems such as incomplete matching.

[0032] In summary, the embodiments of the present disclosure can better improve problems such as incomplete matching that are prone to occur in the existing single-stage matching by adopting multi-stage matching and different matching strategies in different stages, and can effectively improve the matching accuracy.

[0033] On the basis of adopting multi-stage target matching, in order to further improve the matching accuracy rate of each stage, the embodiments of the present disclosure also innovatively propose a boundary similarity measurement algorithm and a combined attribute measurement algorithm. For ease of understanding, the following will be described separately:

[0034] (I) Boundary similarity measurement algorithm

[0035] The boundary similarity metric algorithm is a new similarity metric algorithm proposed in the embodiments of the present disclosure. It can measure the similarity between a detection target and a prediction target based on the boundaries of the two. That is, in one stage, the boundary similarity metric algorithm is used to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity. Among them, the similarity between the successfully matched detection target and prediction target is greater than a preset first similarity screening threshold. The first similarity screening threshold can be set according to actual needs and is not limited here.

[0036] In some embodiments, the boundary similarity metric algorithm mainly includes the following steps: obtaining the boundary distance between the detection target and the prediction target; obtaining the coincidence degree between the detection target and the prediction target; and determining the similarity between the detection target and the prediction target according to the boundary distance and the coincidence degree. The above coincidence degree can also be understood as the coincidence degree between the detection target area formed by the boundary of the detection target and the prediction target area formed by the boundary of the prediction target. It can be understood that the coincidence degree is positively correlated with the similarity. The greater the coincidence degree, the greater the similarity between the detection target and the prediction target; while the boundary distance is negatively correlated with the similarity. The greater the boundary distance, the smaller the similarity between the detection target and the prediction target. In the boundary similarity metric algorithm, the coincidence degree and the boundary distance between the detection target and the prediction target can be comprehensively considered, and the similarity between the detection target and the prediction target can be jointly measured based on the two factors of the coincidence degree and the boundary distance, so as to comprehensively improve the accuracy of similarity measurement. In the related art, when measuring the similarity between a detection target and a prediction target, most often only the traditional IOU (Intersection-over-Union) similarity metric algorithm is used, but for the case where there is no intersection between the detection target and the prediction target, this algorithm cannot be used to measure the similarity. For example, referring to Figure 2 the common relationship diagram between a detection target and a prediction target shown, in Figure 2 example a in is the case where there is an intersection between the detection target and the prediction target (that is, at least partially overlapping), in Figure 2Example b in [the above] is a case where there is no intersection between the detection target and the prediction target (that is, there is no overlap at all). Obviously, the existing IOU similarity metric algorithm cannot calculate the similarity between the detection target and the prediction target in Example b, and the IOU similarity metric values obtained due to no intersection are all zero. That is, the IOU similarity metric algorithm cannot evaluate the similarity for non-intersecting targets. However, the above-mentioned boundary similarity metric algorithm provided by the embodiments of the present disclosure comprehensively considers the boundary distance and the degree of overlap between the detection target and the prediction target. Even for Example b, it can reasonably evaluate its similarity. Therefore, the applicability of similarity evaluation is also wider, and by comprehensively evaluating the boundary distance and the degree of overlap, the accuracy of the obtained similarity is higher than that of a single evaluation of IOU similarity.

[0037] In some specific embodiments, the detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box; the forms of the detection box and the prediction box can also refer to Figure 2 , which will not be elaborated here.

[0038] On the basis that the detection target is represented in the form of a detection box and the prediction target is represented in the form of a prediction box, in some embodiments, the step of obtaining the degree of overlap between the detection target and the prediction target includes: for the detection box and the prediction box to be matched, determining the intersection over union (IoU) between the detection box and the prediction box, and taking the obtained IoU value as the degree of overlap between the detection box and the prediction box. That is, the ratio of the area of the overlapping part of the two boxes, the detection box and the prediction box, to the area of the combination of the two boxes is the above-mentioned IoU, and the degree of overlap between the detection box and the prediction box is measured by the IoU value. In some other embodiments, the degree of overlap can also be determined only based on the overlapping area between the detection box and the prediction box. For example, normalizing the overlapping area and taking the normalization result as the degree of overlap between the detection box and the prediction box. In addition, other methods can also be used to calculate the degree of overlap between the detection box and the prediction box. The embodiments of the present disclosure do not limit the method for calculating the degree of overlap.

[0039] On the basis that the detection target is represented in the form of a detection box and the prediction target is represented in the form of a prediction box, in some embodiments, the step of obtaining the boundary distance between the detection target and the prediction target includes: for the detection box and the prediction box to be matched, obtaining the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box. In some embodiments, the box boundaries can be determined based on the box vertices, and then the boundary distance between the detection box and the prediction box can be obtained. The embodiments of the present disclosure do not limit the specific box vertices, nor the specific calculation method of the boundary distance, as long as it can reasonably and objectively reflect the distance between the two boxes.

[0040] In some embodiments of obtaining the boundary distance between the detection box and the prediction box based on the vertices of the detection box and the vertices of the prediction box, the vertex distance between the designated vertex of the detection box and the designated vertex of the prediction box may be obtained first, and then the boundary distance between the detection box and the prediction box may be determined according to the vertex distance.

[0041] In some embodiments, if the detection box and the prediction box are two parallel boxes, the boundary distance may be determined only based on the vertices at the same positions in the two boxes. For example, the distance between the upper left vertices of the detection box and the prediction box may be used as the boundary distance between the detection box and the prediction box. It should be noted that the above upper left vertices are only examples, and in practical applications, they may also be the upper right vertices, lower left vertices, lower right vertices, etc., which can be specified by oneself and are not limited here. In some embodiments, at least two different designated vertices may be used to determine the boundary distance. For example, the upper left vertex and the lower right vertex may be used to determine the box boundary, or the upper right vertex and the lower left vertex may be used to determine the box boundary, and then the boundary distance between the two boxes may be jointly determined according to the distance between the upper left vertices and the distance between the lower right vertices of the detection box and the prediction box. This method is applicable regardless of whether the detection box and the prediction box are parallel.

[0042] On the basis of the foregoing, the steps of obtaining the vertex distance between the designated vertex of the detection box and the designated vertex of the prediction box and determining the boundary distance between the detection box and the prediction box according to the vertex distance may be performed with reference to the following steps a to d:

[0043] Step a, obtaining a first distance between the first vertex of the detection box and the first vertex of the prediction box;

[0044] Step b, obtaining a second distance between the second vertex of the detection box and the second vertex of the prediction box; where the first vertex is the upper left vertex and the second vertex is the lower right vertex; or the coordinates of the first vertex are the upper right vertex and the coordinates of the second vertex are the lower left vertex;

[0045] Step c, obtaining a minimum bounding box containing the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box, and determining the diagonal length of the minimum bounding box;

[0046] Step d, determining the boundary distance between the detection box and the prediction box according to the first distance, the second distance, and the diagonal length of the minimum bounding box.

[0047] For ease of understanding, reference may also be made to Figure 3 a schematic diagram of a boundary distance metric shown in Figure 3Among them, d0 is the distance between the upper left vertex of the detection target and the prediction target; d1 is the distance between the lower right vertex of the detection target and the prediction target, and C is the diagonal length of the minimum bounding box (dashed box).

[0048] In some specific embodiments, the square value of the first distance and the square value of the second distance can be summed first to obtain a sum value; then, the ratio between the sum value and the square value of the diagonal length of the minimum bounding box is used as the boundary distance between the detection box and the prediction box.

[0049] In some other embodiments of obtaining the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box, the to-be-measured side line of the detection box and the to-be-measured side line of the prediction box can be determined according to the vertices of the detection box and the vertices of the prediction box; the boundary distance between the detection box and the prediction box is determined according to the to-be-measured side line of the detection box and the to-be-measured side line of the prediction box. For example, according to the vertex coordinates A1 (upper left vertex), A2 (lower left vertex), A3 (lower right vertex), A4 (upper right vertex) of the detection box and the vertex coordinates B1 (upper left vertex), B2 (lower left vertex), B3 (lower right vertex), B4 (upper right vertex) of the prediction box, if it is confirmed that the detection box and the prediction box are parallel, then in some embodiments, the to-be-measured side line of the detection box can be set as A1A2 (indicating the side line formed by A1 and A2), and the to-be-measured side line of the prediction box can be set as B1B2, and the distance between A1A2 and B1B2 is directly used as the boundary distance between the detection box and the prediction box. In some other embodiments, it is also possible to calculate multiple distances such as the distance between A1A2 and B1B2, the distance between A1A4 and B1B4, the distance between A1A4 and B2B3, the distance between A2A3 and B1B4, and the distance between A2A3 and B2B3, and then determine the boundary distance between the detection box and the prediction box according to the maximum distance and the minimum distance. Exemplarily, the mean value of the minimum distance and the maximum distance is used as the boundary distance; among them, the two side lines corresponding to the minimum distance and the two side lines corresponding to the maximum distance can both be regarded as the to-be-measured side lines. The above is only an example, and specific settings can be made according to actual needs, and the side lines used can all be used as the to-be-measured side lines.

[0050] As described above, after obtaining the boundary distance and the overlap degree between the detection target and the prediction target, in an implementation manner of determining the similarity between the detection target and the prediction target by using the boundary distance and the overlap degree, the similarity between the detection target and the prediction target can be determined according to the difference between the overlap degree and the boundary distance. Exemplarily, both the overlap degree and the boundary distance can be represented as values between 0 and 1 through normalization, and then the similarity can be determined by the difference between the two. For example, the difference can be directly used as the similarity between the detection target and the prediction target. In some specific implementation manners, the weight coefficient of the overlap degree and the weight coefficient of the boundary distance can also be obtained, and then the difference between the product of the overlap degree and its weight coefficient and the product of the boundary distance and its weight coefficient is used as the similarity between the detection target and the prediction target.

[0051] On the basis described above, an embodiment of the present disclosure gives a specific implementation example of the boundary similarity measurement algorithm. In this implementation example, the boundary similarity measurement algorithm can also be referred to as the BIOU (Boundary Intersection-over-Union) measurement algorithm. Whether there is an intersection between the detection target and the prediction target, and whether the detection target and the prediction target are parallel, this algorithm can be used to achieve reasonable and objective similarity measurement. Specifically, the calculation formula of the BIOU measurement algorithm is as follows:

[0052]

[0053] BIOU = IOU - R BIOU

[0054] Where R BIOU is the boundary distance; C 2 is the square of the diagonal distance of the minimum closed envelope of the two bounding boxes (that is, the minimum bounding box of the detection box and the prediction box). The aforementioned b0 is the upper left vertex of the detection target (that is, the detection box), b0 p is the upper left vertex of the prediction target (that is, the prediction box), b1 is the lower right vertex of the detection target, b1 p is the lower right vertex of the prediction target, ρ 2 (b0b0 p ) represents the square of the distance between the upper left vertices of the detection target and the prediction target respectively, and ρ 2 (b1b1 p ) is the square of the distance between the lower right vertices of the detection target and the prediction target respectively. Exemplarily, taking Figure 3 as an example, d0 2 = ρ 2 (b0b0 p ), d1 2 = ρ 2 (b1b1 p ).

[0055] Among them, IOU = S1 / S2, where S1 is the intersection area between the detected target and the predicted target, and S2 is the union area of the detected target and the predicted target. In the BIOU algorithm, the IOU value is directly used as the coincidence degree between the detected target and the predicted target.

[0056] Finally, the difference between IOU and R BIOU is used as the BIOU value. Among them, R BIOU can also be regarded as a penalty term designed by combining the boundary distance of the bounding box on the basis of the IOU similarity metric. Through the above formula, the two factors of the coincidence degree and the boundary distance between the detected target and the predicted target can be comprehensively considered, so as to more reasonably and effectively realize the similarity metric between the two targets. Moreover, even if there is no intersection between the two targets and IOU is 0, the similarity between the two targets can still be described by R BIOU For example, the greater the distance between the two targets, the corresponding R BIOU is greater, and the smaller the similarity obtained after taking the negative value.

[0057] Since the above boundary similarity metric algorithm measures the similarity between the detected target and the predicted target based on both the coincidence degree and the boundary distance, and its strictness is also higher than the traditional IOU similarity metric, in some embodiments, when performing multi-stage matching on the detected target and the predicted target, the matching strategy in the first stage can adopt the above boundary similarity metric algorithm, so that the accuracy of the detected target and the predicted target corresponding to the same target object selected by this method is the highest.

[0058] In summary, the boundary similarity metric algorithm provided by the embodiments of the present disclosure can comprehensively consider the boundary distance and the coincidence degree between two targets, and preferably solves the problem that the traditional IOU algorithm cannot measure the similarity between non-intersecting targets, and has a wider applicability.

[0059] (2) Combined attribute metric algorithm

[0060] In order to be able to more comprehensively evaluate the similarity between two targets, the embodiments of the present disclosure also provide a combined attribute metric algorithm, which is used to comprehensively realize similarity metric based on attributes such as the position of the target center point, the scale of the bounding box, and the target orientation. When performing multi-stage matching on the detected target and the predicted target, in one stage, the combined attribute metric algorithm can be used to determine the similarity between the detected target and the predicted target, and the detected target and the predicted target are matched according to the similarity; among them, the similarity between the detected target and the predicted target that is successfully matched is greater than a preset second similarity screening threshold. The second similarity screening threshold can be flexibly set according to the actual situation and will not be limited here.

[0061] On the basis that the detection target is represented in the form of a detection box and the predicted target is represented in the form of a prediction box, the steps of using the combined attribute measurement algorithm to determine the similarity between the detection target and the predicted target in one stage include the following steps A and B:

[0062] Step A, for the detection box and the prediction box to be matched, determine the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box. It should be noted that the above orientation angle can be based on a preset coordinate system, and the detection box and the prediction box are located in the same preset coordinate system. Exemplarily, the angle of the detection box relative to the X-axis is used as the orientation angle of the detection box, and the angle of the prediction box relative to the X-axis is used as the orientation angle of the prediction box.

[0063] Step B, based on the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box, determine the similarity between the detection box and the prediction box. In this step, the similarity between the detection target and the predicted target is jointly measured by the differences in the three target attributes of the center point distance, the diagonal length (representing the aforementioned border scale), and the orientation angle, so as to more comprehensively and objectively achieve the similarity measurement.

[0064] In some embodiments, the above step B can be implemented with reference to the following steps B1 to B5:

[0065] Step B1, based on the vertex coordinates of the detection box and the vertex coordinates of the prediction box, determine the smallest enclosing box containing the detection box and the prediction box, and obtain the diagonal length of the smallest enclosing box. Exemplarily, reference can be made to Figure 4 a schematic diagram of combined attribute measurement as shown. The dashed box is the smallest enclosing box. In some embodiments, the box with the smallest area that contains both the detection box and the prediction box can be directly used as the smallest enclosing box. In other embodiments, the box with the smallest perimeter that contains both the detection box and the prediction box can be directly used as the smallest enclosing box, and it can be flexibly set according to the actual situation. On this basis, assuming that there are two enclosing boxes with the same area (both are the smallest area), the enclosing box with the smallest perimeter can be selected as the smallest enclosing box; assuming that there are two enclosing boxes with the same perimeter (both are the smallest perimeter), the enclosing box with the smallest area can be selected as the smallest enclosing box.

[0066] Step B2, based on the center point distance between the detection box and the prediction box and the diagonal length of the smallest enclosing box, determine the central difference evaluation value between the detection box and the prediction box.

[0067] In some specific embodiments, the square value of the center point distance between the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box can be calculated, and the first ratio therebetween is used as the center difference evaluation value between the detection box and the prediction box. Specifically, the following formula can be referred to:

[0068]

[0069] Where C 2 is the square of the diagonal distance of the minimum closure of the two bounding boxes (that is, the minimum bounding box of the detection box and the prediction box), c represents the center point of the detection target (detection box), and c p represents the center point of the prediction target (prediction box), and ρ 2 (c, c p ) represents the square of the center point distance between the detection target and the prediction target.

[0070] Step B3: Based on the difference between the diagonal lengths of the detection box and the prediction box and the diagonal length of the minimum bounding box, determine the diagonal difference evaluation value between the detection box and the prediction box.

[0071] In some specific embodiments, the square value of the difference between the diagonal lengths of the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box can be calculated, and the second ratio therebetween is used as the diagonal difference evaluation value between the detection box and the prediction box. Specifically, the following formula can be referred to:

[0072]

[0073] Where C 2 is the square of the diagonal distance of the minimum closure of the two bounding boxes (that is, the minimum bounding box of the detection box and the prediction box), t0 is the diagonal length of the detection box, t1 is the diagonal length of the prediction box, and ρ 2 (t0, t1 p ) is the square of the difference between the diagonal lengths of the detection box and the prediction box.

[0074] Step B4: Based on the difference in the orientation angles and the preset trigonometric functions, determine the angle difference evaluation value between the detection box and the prediction box.

[0075] In some specific embodiments, based on the sine trigonometric function, the sine trigonometric function value of the difference in the orientation angles can be calculated; the square value of the sine trigonometric function value is used as the angle difference evaluation value between the detection box and the prediction box. Specifically, the following formula can be referred to:

[0076] ΔY = sin 2 (θ0 - θ1)

[0077] Among them, θ0 is the angle of the detection target relative to the x-axis, and θ1 is the angle of the predicted target relative to the x-axis.

[0078] Step B5: Determine the similarity between the detection box and the prediction box according to the central difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value.

[0079] In some embodiments, the similarity JM between the detection box and the prediction box is determined according to the following formula:

[0080] JM = α×(1 - ΔD) + β×(1 - ΔS) + (1 - α - β)×(1 - ΔY)

[0081] Among them, ΔD represents the central difference evaluation value; ΔS represents the diagonal difference evaluation value; ΔY represents the angle difference evaluation value; α represents a preset first weight coefficient, and β represents a preset second weight coefficient. In practical applications, the values of α and β can be flexibly set according to requirements, and the importance of the three attributes can be adjusted by different coefficients to obtain a better similarity description.

[0082] In summary, the above-mentioned combined attribute measurement algorithm provided by the embodiments of the present disclosure can comprehensively consider attributes such as the center point, size, and orientation of the target, so as to more accurately describe the similarity of the target. In the case of a small target, compared with traditional similarity evaluation algorithms such as IOU, the above-mentioned combined attribute measurement algorithm has a better similarity evaluation effect.

[0083] In some embodiments, the combined attribute measurement algorithm is used to determine the similarity between the detection target and the prediction target that failed to match successfully in the previous stage. Exemplarily, taking the two-stage matching target as an example, in the first stage, the foregoing boundary similarity measurement algorithm is used to screen out the detection target and the prediction target that match successfully in a relatively strict manner, that is, to screen out the optimal one-to-one matching result, fully ensuring the matching accuracy. Then, in the second stage, the foregoing combined attribute measurement algorithm is used to match the detection target and the prediction target that failed to match successfully in the first stage, fully ensuring the comprehensiveness of the matching, and comprehensively reducing the situations of incorrect matching and missing matching.

[0084] In addition to the foregoing boundary similarity measurement algorithm and combined attribute measurement algorithm, in practical applications, the intersection over union algorithm (IOU algorithm) can also be used in one stage to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; among them, the similarity between the detection target and the prediction target that match successfully is greater than a preset third similarity screening threshold. Exemplarily, in multi-stage matching, at least two of the boundary similarity measurement algorithm, the combined attribute measurement algorithm, and the intersection over union algorithm can be selected.

[0085] After obtaining the similarity between the detection target and the prediction target through any one of the above algorithms such as the boundary similarity measurement algorithm, the combined attribute measurement algorithm, the intersection over union algorithm, etc., the target matching can be further performed based on the similarity. In some embodiments, the matching strategy adopted in any stage includes the following steps:

[0086] (1) For at least one detection target and at least one prediction target to be matched, calculate the similarity between any detection target and any prediction target through the similarity algorithm adopted in this stage, and obtain the similarity between any two detection / prediction targets. Exemplarily, if the detection target set includes two detection targets X1 and X2, and the prediction target set includes two prediction targets Y1 and Y2, then the similarities of X1Y1, X1Y2, X2Y1, and X2Y2 need to be calculated respectively. For the convenience of processing, all the obtained similarities can also be formed into a similarity matrix such as 2*2.

[0087] (2) According to the similarity between any two detection / prediction targets, use a preset assignment algorithm to determine the optimal one-to-one matching relationship, and obtain the matching result of the detection target and the prediction target. In a specific implementation example, the assignment algorithm is the Hungarian algorithm. Exemplarily, through the assignment algorithm, it is considered that X1 is matched with Y2, and X2 is matched with Y1.

[0088] (3) Screen the matching result of the detection target and the prediction target through the similarity screening threshold adopted in this stage, and regard the detection target and the prediction target with a similarity greater than the similarity screening threshold in the one-to-one matching relationship as successfully matched. Exemplarily, assume that the similarity screening threshold is 80%, the similarity between X1 and Y2 is 75%, and the similarity between X2 and Y1 is 89%, then it is considered that X2 and Y1 are successfully matched in this stage.

[0089] If there are detection targets and prediction targets that are not successfully matched, such as X1 and Y2 are not successfully matched, then the matching strategy of the next stage can be used to match them again. Execute the above steps (1) to (3) using the matching strategy of the next stage. Since the matching strategies adopted in different stages are different, that is, the similarity algorithms and / or similarity screening thresholds adopted in different stages are different, the detection targets and prediction targets that were not successfully matched in the previous stage may be successfully matched in the next stage. Through the multi-stage matching method, the accuracy and comprehensiveness of the matching can be effectively guaranteed. For example, in the previous stage, the matching accuracy can be guaranteed through a relatively strict matching strategy, and in the subsequent stage, the detection targets and prediction targets that are not successfully matched can be matched through other strategies, so as to avoid the situation of target loss and comprehensively guarantee the matching accuracy.

[0090] Based on the above, an embodiment of the present disclosure gives a specific implementation example of multi-stage matching. In this implementation example, it is mainly two-stage matching. For details, please refer to Figure 5 the flowchart of a two-stage object matching method shown in

[0091] Step S502: Obtain the detection object and the prediction object to be matched.

[0092] Step S504: Use the boundary similarity metric algorithm to measure the similarity between all detection objects and prediction objects to be matched, and obtain the first similarity measurement result. Among them, the first similarity measurement result can be represented in the form of a similarity matrix.

[0093] Step S506: Based on the first similarity measurement result, use the Hungarian algorithm to solve the assignment and obtain a one-to-one matching relationship.

[0094] Step S508: Screen the above matching relationship based on the first similarity screening threshold, and obtain the detection object and the prediction object that are successfully matched in the first stage; among them, the similarity between the detection object and the prediction object that are successfully matched in the first stage is greater than the first similarity screening threshold.

[0095] Step S510: Determine whether all detection objects and prediction objects are successfully matched; if so, execute step S518, if not, execute step S512;

[0096] Step S512: Use the combined attribute similarity metric algorithm to measure the similarity between the above unmatched detection objects and prediction objects, and obtain the second similarity measurement result. Among them, the second similarity measurement result can be represented in the form of a similarity matrix.

[0097] Step S514: Based on the second similarity measurement result, use the Hungarian algorithm to solve the assignment and obtain a one-to-one matching relationship.

[0098] Step S516: Screen the above matching relationship (the matching relationship obtained in step S514) based on the second similarity screening threshold, and obtain the detection object and the prediction object that are successfully matched in the second stage; among them, the similarity between the detection object and the prediction object that are successfully matched in the second stage is greater than the second similarity screening threshold.

[0099] Step S518: Combine all the successfully matched detection objects and prediction objects together as the two-stage object matching result. That is, combine the detection objects and prediction objects that are successfully matched in the first stage and the second stage together as the two-stage object matching result.

[0100] For easy understanding, please also refer to Figure 6As shown in the schematic diagram of two-stage object matching, in Figure 6 , the BIOU is directly used to represent the boundary similarity metric algorithm, and the JM represents the aforementioned combined attribute similarity metric algorithm. For the remaining specific matching processes, reference can be made to Figure 5 , which will not be elaborated here.

[0101] Through the above Figure 5 and Figure 6 way, in the first stage, the aforementioned boundary similarity metric algorithm is adopted to screen out the successfully matched detection targets and prediction targets in a relatively strict manner, fully ensuring the matching accuracy. Then, in the second stage, the aforementioned combined attribute metric algorithm is adopted to match the detection targets and prediction targets that failed to match successfully in the first stage, fully ensuring the comprehensiveness of the matching, and comprehensively reducing the cases of incorrect matching and matching loss.

[0102] For the detection targets and prediction targets that still failed to match successfully in the last stage, they can be processed separately. For example, they can be saved at a specified location and saved for a preset duration to prevent possible application within the preset duration. If the unmatched detection targets and prediction targets have not been applied within the preset duration, they can be discarded.

[0103] As mentioned above, through multi-stage matching of detection targets and prediction targets, the detection targets and prediction targets that match successfully in each stage can be obtained. Then, the detection targets and prediction targets that match successfully in each stage can be jointly used as the multi-stage matching result. It should be noted that if all the detection targets and prediction targets to be detected have been successfully matched in a certain stage, the multi-stage object matching ends and the subsequent stages are no longer executed. Such as Figure 5 shown, assuming that all detection targets and prediction targets have been successfully matched in the first stage, the matching process of the second stage will no longer be executed.

[0104] It should be noted that the two-stage matching method such as Figure 5 and Figure 6 shown is only an example and should not be regarded as a limitation. In actual applications, the similarity metric algorithms in the matching strategies adopted in different stages can be set according to requirements. Moreover, it can be extended to multi-stage cascaded matching such as three-stage, four-stage, five-stage, etc., and finally the multi-stage matching result can be obtained.

[0105] After obtaining the multi-stage matching results, target tracking can be performed based on the multi-stage matching results. For example, for the image to be processed, multi-stage matching is performed on the detected target and the predicted target in the image to be processed, and the detected target and the predicted target corresponding to the same target object are obtained. However, the positions of the detected target and the predicted target may be different. Therefore, based on the positions of the detected target and the predicted target, the position of the target object corresponding to them can be determined. In this way, the positions of the target objects in each acquired image can be marked, and based on the acquisition time of each image, target trajectory tracking can be achieved. Target tracking can be preferably applied to scenarios such as autonomous driving and is an important part of the autonomous driving environment perception system.

[0106] On the basis described above, the embodiments of the present disclosure provide a schematic diagram of a multi-target tracking structure as shown in Figure 7 which schematically shows a 3D multi-target tracking framework. In Figure 7 a 3D object detector can be used to detect objects in the image to be processed to obtain detected targets. In the actual application process, the 3D object detector can obtain the 3D detected targets in the current scene in real time and represent them in the form of 3D bounding boxes (detection boxes). The Kalman filter can predict the objects in the image to be processed based on the target trajectory at the previous moment to obtain predicted targets. That is, the Kalman filter can use the detection information of the historical frames to predict the object positions in the current frame to obtain predicted targets and represent them in the form of prediction boxes. The predicted targets and the detected targets are subjected to object matching (one-to-one association) to obtain the object matching results. The object matching results can return the detected targets and the predicted targets that have completed the matching (that is, the detected targets and the predicted targets that have successfully matched) to the Kalman filter for posterior estimation, and trajectory management is performed based on the posterior estimation results. Among them, the main steps included in the above posterior estimation are: for the positions of the detected targets and the predicted targets that have successfully matched, determine the position of the target object corresponding to them, and use this as the recorded position of the target object in the image to be processed, so as to be used in subsequent trajectory management. In addition, a life cycle management process can be performed for the unmatched objects (unmatched detections / unmatched tracks). In the life cycle management process, the unmatched objects can be retained for a preset duration. If the unmatched objects are not used for more than the preset duration, the unmatched objects will be discarded. If the unmatched objects are applied again within the preset duration (for example, first detect target A, then target A is occluded, and then target A appears again), the trajectory management can continue for them.

[0107] In summary, the target matching method provided by the embodiments of the present disclosure can better improve problems such as incomplete matching prone to occur in the existing single-stage matching through multi-stage matching with different matching strategies adopted in different stages, and can effectively improve the matching accuracy. To further improve the matching accuracy rate of each stage, the embodiments of the present disclosure also innovatively propose a boundary similarity measurement algorithm and a combined attribute measurement algorithm. The boundary similarity measurement algorithm can comprehensively consider the boundary distance and coincidence degree between two targets, solving the problem that the traditional IOU algorithm cannot measure the similarity between targets without intersection. The combined attribute measurement algorithm can comprehensively consider attributes such as the center point, size, and orientation of the target, so as to more accurately describe the similarity of the target, and is also better applicable to the similarity evaluation of smaller targets.

[0108] Furthermore, the embodiments of the present disclosure also propose a way in which the strictness of the matching strategies in multiple stages decreases in turn. For example, in the previous stage, a more strict matching strategy is preferentially adopted for target matching to fully ensure the matching accuracy, and in the subsequent stage, a not very strict matching strategy is adopted to match the detected targets and predicted targets that failed to match successfully in the previous stage, so as to avoid target loss. In some specific embodiments, the foregoing boundary similarity measurement algorithm is used in the first stage to screen out the detected targets and predicted targets that match successfully in a more strict manner, fully ensuring the matching accuracy. Then, the foregoing combined attribute measurement algorithm is used in the second stage to match the detected targets and predicted targets that failed to match successfully in the first stage, fully ensuring the comprehensiveness of the matching, and comprehensively reducing the cases of incorrect matching and matching loss. Finally, the detected targets and predicted targets that match successfully in each stage are jointly used as the multi-stage matching result, comprehensively improving the matching accuracy rate, and also helping to further improve the tracking reliability of target tracking based on the matching result.

[0109] In addition, in the related art, a deep learning model or a correlation prediction network is used to extract features of the detected target and the predicted target, and similarity calculation is performed based on the features, but sufficient data is required for network training, and the training cost is relatively high; moreover, the above method is sensitive to the scene. Once the scene is changed, the network often needs to be retrained to calculate a more accurate similarity. In addition, in 3D target tracking based on lidar, due to the sparsity of the point cloud, it is not easy to extract features; and when the target is too small, it is even more difficult to extract an effective point cloud sequence for similarity calculation. Compared with the related art, the above method provided by the embodiments of the present disclosure has a lower required cost, can directly use a more convenient algorithm for target matching, and is not restricted by the scene, having universal applicability.

[0110] Corresponding to the foregoing target matching method, the embodiments of the present disclosure also provide a target matching device. Figure 8Schematic structural diagram of a target matching device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is generally integrated in an electronic device, such as Figure 8 shown, including:

[0111] A target acquisition module 802, configured to acquire a detection target and a prediction target corresponding to an image to be processed;

[0112] A multi-stage matching module 804, configured to perform multi-stage matching based on the detection target and the prediction target; wherein, different matching strategies are adopted in different stages.

[0113] By adopting multi-stage matching in the above manner, and different matching strategies are adopted in different stages, it can better improve problems such as incomplete matching that are prone to occur in existing single-stage matching, and can effectively improve the matching accuracy. In order to further improve the matching accuracy rate of each stage.

[0114] In some embodiments, a boundary similarity metric algorithm is used in one stage to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; wherein, the similarity between the successfully matched detection target and the prediction target is greater than a preset first similarity screening threshold.

[0115] In some embodiments, the multi-stage matching module includes:

[0116] A distance acquisition unit, configured to acquire the boundary distance between the detection target and the prediction target;

[0117] An overlap degree acquisition unit, configured to acquire the overlap degree between the detection target and the prediction target;

[0118] A first similarity determination unit, configured to determine the similarity between the detection target and the prediction target according to the boundary distance and the overlap degree.

[0119] In some embodiments, the detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box;

[0120] The distance acquisition unit is specifically configured to: for a detection box and a prediction box to be matched, acquire the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box;

[0121] The overlap degree acquisition unit is specifically configured to: determine the intersection over union ratio between the detection box and the prediction box, and use the obtained intersection over union ratio value as the overlap degree between the detection box and the prediction box.

[0122] In some embodiments, the distance acquisition unit is specifically configured to: acquire the vertex distance between the specified vertex of the detection box and the specified vertex of the prediction box, and determine the boundary distance between the detection box and the prediction box according to the vertex distance.

[0123] In some embodiments, the distance acquisition unit is specifically configured to: acquire a first distance between a first vertex of the detection box and a first vertex of the prediction box; acquire a second distance between a second vertex of the detection box and a second vertex of the prediction box; wherein, the first vertex is the upper left vertex, and the second vertex is the lower right vertex; alternatively, the first vertex coordinate is the upper right vertex, and the second vertex coordinate is the lower left vertex; obtain a minimum bounding box containing the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box, and determine the diagonal length of the minimum bounding box; determine the boundary distance between the detection box and the prediction box according to the first distance, the second distance, and the diagonal length of the minimum bounding box.

[0124] In some embodiments, the distance acquisition unit is specifically configured to: sum the squared value of the first distance and the squared value of the second distance to obtain a sum value; use the ratio between the sum value and the squared value of the diagonal length of the minimum bounding box as the boundary distance between the detection box and the prediction box.

[0125] In some embodiments, the distance acquisition unit is specifically configured to: determine the to-be-measured side line of the detection box and the to-be-measured side line of the prediction box according to the vertices of the detection box and the vertices of the prediction box; determine the boundary distance between the detection box and the prediction box according to the to-be-measured side line of the detection box and the to-be-measured side line of the prediction box.

[0126] In some embodiments, the first similarity determination unit is specifically configured to: determine the similarity between the detection target and the prediction target according to the difference between the coincidence degree and the boundary distance.

[0127] In some embodiments, a combination attribute measurement algorithm is used in one stage to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; wherein, the similarity between the detection target and the prediction target that are successfully matched is greater than a preset second similarity screening threshold.

[0128] In some embodiments, the combination attribute measurement algorithm is used to determine the similarity between the detection target and the prediction target that were not successfully matched in the previous stage.

[0129] In some embodiments, the detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box; the multi-stage matching module includes:

[0130] A multi-attribute determination unit, configured to determine the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box to be matched;

[0131] A second similarity determination unit, configured to determine the similarity between the detection box and the prediction box based on the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box.

[0132] In some embodiments, the second similarity determination unit is specifically configured to: determine a minimum bounding box containing the detection box and the prediction box based on the vertex coordinates of the detection box and the vertex coordinates of the prediction box, and obtain the diagonal length of the minimum bounding box; determine a center difference evaluation value between the detection box and the prediction box based on the center point distance between the detection box and the prediction box and the diagonal length of the minimum bounding box; determine a diagonal difference evaluation value between the detection box and the prediction box based on the difference in diagonal length between the detection box and the prediction box and the diagonal length of the minimum bounding box; determine an angle difference evaluation value between the detection box and the prediction box based on the difference in orientation angle and a preset trigonometric function; and determine the similarity between the detection box and the prediction box according to the center difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value.

[0133] On the basis of the foregoing, in some embodiments, the second similarity determination unit is specifically configured to: calculate a first ratio between the square value of the center point distance between the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box, and use the first ratio as the center difference evaluation value between the detection box and the prediction box.

[0134] On the basis of the foregoing, in some embodiments, the second similarity determination unit is specifically configured to: calculate a second ratio between the square value of the difference in diagonal length between the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box, and use the second ratio as the diagonal difference evaluation value between the detection box and the prediction box.

[0135] On the basis of the foregoing, in some embodiments, the second similarity determination unit is specifically configured to: calculate the sine trigonometric function value of the difference in orientation angle based on the sine trigonometric function; and use the square value of the sine trigonometric function value as the angle difference evaluation value between the detection box and the prediction box.

[0136] On the basis of the foregoing, in some embodiments, the second similarity determination unit is specifically configured to determine the similarity JM between the detection box and the prediction box according to the following formula:

[0137] JM = α×(1 - ΔD) + β×(1 - ΔS) + (1 - α - β)×(1 - ΔY)

[0138] Wherein, ΔD represents the central difference evaluation value; ΔS represents the diagonal difference evaluation value; ΔY represents the angle difference evaluation value; α represents a preset first weight coefficient, and β represents a preset second weight coefficient.

[0139] In some embodiments, the intersection over union (IoU) algorithm is used in one stage to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; wherein, the similarity between the successfully matched detection target and the prediction target is greater than a preset third similarity screening threshold.

[0140] In some embodiments, the target acquisition module is configured to: perform target detection on the to-be-processed image by using a target detection algorithm to obtain a detection target; acquire a target trajectory before the acquisition time of the to-be-processed image, and perform target prediction on the to-be-processed image according to the target trajectory to obtain a prediction target.

[0141] In some embodiments, the apparatus further includes: a tracking module, configured to use the successfully matched detection targets and prediction targets in each stage as multi-stage matching results; and perform target tracking according to the multi-stage matching results.

[0142] The target matching apparatus provided by the embodiments of the present disclosure can execute the target matching method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0143] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the apparatus embodiments described above can refer to the corresponding process in the method embodiments, and will not be described herein again.

[0144] The embodiments of the present disclosure provide an electronic device, which includes: a processor; a memory for storing processor-executable instructions; and the processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the above target matching methods.

[0145] Figure 9 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. As Figure 9 shown, the electronic device 900 includes one or more processors 901 and a memory 902.

[0146] The processor 901 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 900 to perform desired functions.

[0147] The memory 902 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 901 may run the program instructions to implement the target matching method of the embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage media.

[0148] In one example, the electronic device 900 may further include: an input device 903 and an output device 904, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0149] In addition, the input device 903 may further include, for example, a keyboard, a mouse, and so on.

[0150] The output device 904 may output various information to the outside, including the determined distance information, direction information, etc. The output device 904 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0151] Of course, for simplicity, Figure 9 only some of the components related to the present disclosure in the electronic device 900 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 900 may further include any other appropriate components.

[0152] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, and when the computer program instructions are run by a processor, the processor is caused to execute the target matching method provided by the embodiments of the present disclosure.

[0153] The computer program product may be written in any combination of one or more programming languages for executing the program code of the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are run by a processor, the processor is caused to execute the target matching method provided by the embodiments of the present disclosure.

[0155] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] An embodiment of the present disclosure also provides a computer program product, including a computer program / instructions, which when executed by a processor implement the target matching method in the embodiments of the present disclosure.

[0157] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0158] In summary, for the target matching method, device, equipment, and storage medium provided by the embodiments of the present disclosure, the following can be referred to for implementation:

[0159] A1. A target matching method includes:

[0160] Obtain the detection target and prediction target corresponding to the image to be processed;

[0161] Perform multi-stage matching based on the detection target and the prediction target, where different matching strategies are adopted in different stages.

[0162] A2. According to the method described in A1, in one stage, a boundary similarity metric algorithm is used to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; among them, the similarity between the successfully matched detection target and the prediction target is greater than a preset first similarity screening threshold.

[0163] A3. According to the method described in A2, the step of using a boundary similarity metric algorithm to determine the similarity between the detection target and the prediction target includes:

[0164] Obtain the boundary distance between the detection target and the prediction target;

[0165] Obtain the coincidence degree between the detection target and the prediction target;

[0166] Determine the similarity between the detection target and the prediction target according to the boundary distance and the coincidence degree.

[0167] A4. According to the method described in A3, the detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box;

[0168] The step of obtaining the boundary distance between the detection target and the prediction target includes:

[0169] For the detection box and the prediction box to be matched, obtain the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box;

[0170] The step of obtaining the coincidence degree between the detection target and the prediction target includes:

[0171] Determine the intersection over union ratio between the detection box and the prediction box, and use the obtained intersection over union ratio value as the coincidence degree between the detection box and the prediction box.

[0172] A5. According to the method described in A4, the step of obtaining the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box includes:

[0173] Obtain the vertex distance between the specified vertex of the detection box and the specified vertex of the prediction box, and determine the boundary distance between the detection box and the prediction box according to the vertex distance.

[0174] A6. According to the method described in A5, the steps of obtaining the vertex distance between the specified vertex of the detection box and the specified vertex of the prediction box, and determining the boundary distance between the detection box and the prediction box include:

[0175] Obtain the first distance between the first vertex of the detection box and the first vertex of the prediction box;

[0176] Obtain the second distance between the second vertex of the detection box and the second vertex of the prediction box; wherein, the first vertex is the upper left vertex, and the second vertex is the lower right vertex; or, the first vertex coordinate is the upper right vertex, and the second vertex coordinate is the lower left vertex;

[0177] According to the vertices of the detection box and the vertices of the prediction box, obtain the smallest bounding box containing the detection box and the prediction box, and determine the diagonal length of the smallest bounding box;

[0178] Determine the boundary distance between the detection box and the prediction box according to the first distance, the second distance, and the diagonal length of the smallest bounding box.

[0179] A7. According to the method described in A6, the steps of determining the boundary distance between the detection box and the prediction box according to the first distance, the second distance, and the diagonal length include:

[0180] Sum the squared value of the first distance and the squared value of the second distance to obtain a sum value;

[0181] Take the ratio between the sum value and the squared value of the diagonal length of the smallest bounding box as the boundary distance between the detection box and the prediction box.

[0182] A8. According to the method described in A4, the steps of obtaining the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box include:

[0183] Determine the side to be measured of the detection box and the side to be measured of the prediction box according to the vertices of the detection box and the vertices of the prediction box; determine the boundary distance between the detection box and the prediction box according to the side to be measured of the detection box and the side to be measured of the prediction box.

[0184] A9. According to the method described in A3, the steps of determining the similarity between the detection target and the prediction target according to the boundary distance and the coincidence degree include:

[0185] Determine the similarity between the detection target and the prediction target according to the difference between the degree of overlap and the boundary distance.

[0186] A10. According to the method described in any one of A1 to A9, in one stage, a combined attribute metric algorithm is used to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; wherein, the similarity between the detection target and the prediction target with successful matching is greater than a preset second similarity screening threshold.

[0187] A11. According to the method described in A10, the combined attribute metric algorithm is used to determine the similarity between the detection target and the prediction target that was not successfully matched in the previous stage.

[0188] A12. According to the method described in A10 or A11, the detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box;

[0189] The steps of using a combined attribute metric algorithm to determine the similarity between the detection target and the prediction target in one stage include:

[0190] For the detection box and the prediction box to be matched, determine the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box;

[0191] Based on the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box, determine the similarity between the detection box and the prediction box.

[0192] A13. According to the method described in A12, the steps of determining the similarity between the detection box and the prediction box based on the center point distance, the difference in diagonal length, and the difference in orientation angle include:

[0193] Based on the vertex coordinates of the detection box and the vertex coordinates of the prediction box, determine the smallest enclosing box containing the detection box and the prediction box, and obtain the diagonal length of the smallest enclosing box;

[0194] Based on the center point distance between the detection box and the prediction box and the diagonal length of the smallest enclosing box, determine the center difference evaluation value between the detection box and the prediction box;

[0195] Based on the difference in diagonal length between the detection box and the prediction box and the diagonal length of the smallest enclosing box, determine the diagonal difference evaluation value between the detection box and the prediction box;

[0196] Determine an angle difference evaluation value between the detection box and the prediction box based on the difference in the orientation angles and a preset trigonometric function;

[0197] Determine the similarity between the detection box and the prediction box according to the center difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value.

[0198] A14. The step of determining the center difference evaluation value between the detection box and the prediction box based on the distance between the center points of the detection box and the prediction box and the diagonal length of the minimum bounding box according to the method described in A13 includes:

[0199] Calculate a first ratio between the square value of the distance between the center points of the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box, and use the first ratio as the center difference evaluation value between the detection box and the prediction box.

[0200] A15. The step of determining the diagonal difference evaluation value between the detection box and the prediction box based on the difference in the diagonal lengths of the detection box and the prediction box and the diagonal length of the minimum bounding box according to the method described in A13 includes:

[0201] Calculate a second ratio between the square value of the difference in the diagonal lengths of the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box, and use the second ratio as the diagonal difference evaluation value between the detection box and the prediction box.

[0202] A16. The step of determining the angle difference evaluation value between the detection box and the prediction box based on the difference in the orientation angles and a preset trigonometric function according to the method described in A13 includes:

[0203] Based on the sine trigonometric function, calculate the sine trigonometric function value of the difference in the orientation angles;

[0204] Use the square value of the sine trigonometric function value as the angle difference evaluation value between the detection box and the prediction box.

[0205] A17. The step of determining the similarity between the detection box and the prediction box according to the center difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value according to the method described in A13 includes:

[0206] Determine the similarity JM between the detection box and the prediction box according to the following formula:

[0207] JM = α×(1 - ΔD) + β×(1 - ΔS) + (1 - α - β)×(1 - ΔY)

[0208] Wherein, ΔD represents the central difference evaluation value; ΔS represents the diagonal difference evaluation value; ΔY represents the angle difference evaluation value; α represents a preset first weight coefficient, and β represents a preset second weight coefficient.

[0209] A18. According to the method described in A1, in one stage, the intersection over union algorithm is used to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; wherein, the similarity between the detection target and the prediction target with successful matching is greater than a preset third similarity screening threshold.

[0210] A19. According to the method described in A1, the steps of obtaining the detection target and the prediction target corresponding to the image to be processed include:

[0211] Performing object detection on the image to be processed by using an object detection algorithm to obtain a detection target;

[0212] Obtaining the target trajectory before the acquisition moment of the image to be processed, and performing target prediction on the image to be processed according to the target trajectory to obtain a prediction target.

[0213] A20. According to the method described in A1, the method further includes:

[0214] Regarding the detection targets and prediction targets with successful matching in each stage as the multi-stage matching result;

[0215] Performing object tracking according to the multi-stage matching result.

[0216] B21. An object matching device, including:

[0217] An acquisition module, configured to acquire a detection target and a prediction target corresponding to an image to be processed;

[0218] A multi-stage matching module, configured to perform multi-stage matching based on the detection target and the prediction target; wherein, different matching strategies are adopted in different stages.

[0219] C22. An electronic device, the electronic device includes:

[0220] A processor;

[0221] A memory for storing executable instructions of the processor;

[0222] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the object matching method described in any one of A1 - A20 above.

[0223] D23. A computer-readable storage medium stores a computer program for executing the target matching method described in any one of A1 - A20 above.

[0224] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but rather will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target matching method, characterized in that, Including: Obtain a detection target and a prediction target corresponding to an image to be processed; Perform multi-stage matching based on the detection target and the prediction target, where different matching strategies are adopted in different stages; Wherein, in one stage, a combined attribute metric algorithm is used to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; the similarity between the successfully matched detection target and the prediction target is greater than a preset second similarity screening threshold; The detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box; the step of using a combined attribute metric algorithm to determine the similarity between the detection target and the prediction target in one stage includes: for a detection box and a prediction box to be matched, determine the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box; based on the vertex coordinates of the detection box and the vertex coordinates of the prediction box, determine the smallest bounding box containing the detection box and the prediction box, and obtain the diagonal length of the smallest bounding box; based on the center point distance between the detection box and the prediction box and the diagonal length of the smallest bounding box, determine the center difference evaluation value between the detection box and the prediction box; based on the difference in diagonal length between the detection box and the prediction box and the diagonal length of the smallest bounding box, determine the diagonal difference evaluation value between the detection box and the prediction box; based on the difference in orientation angle and a preset trigonometric function, determine the angle difference evaluation value between the detection box and the prediction box; according to the center difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value, determine the similarity between the detection box and the prediction box.

2. The method according to claim 1, characterized in that In one stage, a boundary similarity metric algorithm is used to determine the similarity between the detection target and the prediction target, and the detection target and the prediction target are matched according to the similarity; wherein, the similarity between the successfully matched detection target and the prediction target is greater than a preset first similarity screening threshold.

3. The method according to claim 2, wherein The step of using a boundary similarity metric algorithm to determine the similarity between the detection target and the prediction target in one stage includes: Obtain the boundary distance between the detection target and the prediction target; Obtain the overlap degree between the detection target and the prediction target; Determine the similarity between the detection target and the prediction target according to the boundary distance and the overlap degree.

4. The method according to claim 3, characterized in that, The detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box; The step of obtaining the boundary distance between the detection target and the prediction target includes: For a detection box and a prediction box to be matched, obtain the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box; The step of obtaining the overlap degree between the detection target and the prediction target includes: Determine the intersection over union ratio between the detection box and the prediction box, and use the obtained intersection over union ratio value as the overlap degree between the detection box and the prediction box.

5. The method according to claim 4, wherein The steps of obtaining the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box include: Obtain the vertex distance between the specified vertex of the detection box and the specified vertex of the prediction box, and determine the boundary distance between the detection box and the prediction box according to the vertex distance.

6. The method according to claim 5, characterized in that, The steps of obtaining the vertex distance between the specified vertex of the detection box and the specified vertex of the prediction box, and determining the boundary distance between the detection box and the prediction box according to the vertex distance include: Obtain the first distance between the first vertex of the detection box and the first vertex of the prediction box; Obtain the second distance between the second vertex of the detection box and the second vertex of the prediction box; where the first vertex is the upper left vertex, and the second vertex is the lower right vertex; or, the first vertex coordinate is the upper right vertex, and the second vertex coordinate is the lower left vertex; According to the vertices of the detection box and the vertices of the prediction box, obtain the smallest bounding box containing the detection box and the prediction box, and determine the diagonal length of the smallest bounding box; Determine the boundary distance between the detection box and the prediction box according to the first distance, the second distance, and the diagonal length of the smallest bounding box.

7. The method according to claim 6, characterized in that, The steps of determining the boundary distance between the detection box and the prediction box according to the first distance, the second distance, and the diagonal length include: Sum the squared value of the first distance and the squared value of the second distance to obtain a sum value; Use the ratio between the sum value and the squared value of the diagonal length of the smallest bounding box as the boundary distance between the detection box and the prediction box.

8. The method according to claim 4, characterized in that The steps of obtaining the boundary distance between the detection box and the prediction box according to the vertices of the detection box and the vertices of the prediction box include: Determine the to-be-measured side line of the detection box and the to-be-measured side line of the prediction box according to the vertices of the detection box and the vertices of the prediction box; determine the boundary distance between the detection box and the prediction box according to the to-be-measured side line of the detection box and the to-be-measured side line of the prediction box.

9. The method according to claim 3, characterized in that, The steps of determining the similarity between the detection target and the prediction target according to the boundary distance and the coincidence degree include: Determine the similarity between the detection target and the prediction target according to the difference between the coincidence degree and the boundary distance.

10. According to the method described in claim 1, the combined attribute measurement algorithm is used to determine the similarity between the detection target and the prediction target that was not successfully matched in the previous stage.

11. The method according to claim 1, wherein The steps of determining the central difference evaluation value between the detection box and the prediction box based on the center point distance between the detection box and the prediction box and the diagonal length of the smallest bounding box include: Calculate the first ratio between the squared value of the center point distance between the detection box and the prediction box and the squared value of the diagonal length of the smallest bounding box, and use the first ratio as the central difference evaluation value between the detection box and the prediction box.

12. The method according to claim 1, characterized in that, The steps of determining the diagonal difference evaluation value between the detection box and the prediction box based on the difference in diagonal length between the detection box and the prediction box and the diagonal length of the smallest bounding box include: Calculate the second ratio between the square value of the difference in the diagonal lengths between the detection box and the prediction box and the square value of the diagonal length of the minimum bounding box, and use the second ratio as the diagonal difference evaluation value between the detection box and the prediction box.

13. The method according to claim 1, wherein The step of determining the angle difference evaluation value between the detection box and the prediction box based on the difference in the orientation angles and a preset trigonometric function includes: Based on the sine trigonometric function, calculate the sine trigonometric function value of the difference in the orientation angles; Use the square value of the sine trigonometric function value as the angle difference evaluation value between the detection box and the prediction box.

14. The method according to claim 1, wherein The step of determining the similarity between the detection box and the prediction box according to the center difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value includes: Determine the similarity JM between the detection box and the prediction box according to the following formula: Among them, represents the central difference evaluation value; represents the diagonal difference evaluation value; represents the angle difference evaluation value; represents a preset first weight coefficient, represents a preset second weight coefficient.

15. The method according to claim 1, wherein In one stage, use the intersection over union algorithm to determine the similarity between the detection target and the prediction target, and match the detection target and the prediction target according to the similarity; wherein, the similarity between the successfully matched detection target and the prediction target is greater than a preset third similarity screening threshold.

16. The method according to claim 1, wherein The step of obtaining the detection target and the prediction target corresponding to the image to be processed includes: Use an object detection algorithm to perform object detection on the image to be processed to obtain a detection target; Obtain the target trajectory before the acquisition time of the image to be processed, and perform target prediction on the image to be processed according to the target trajectory to obtain a prediction target.

17. The method according to claim 1, wherein The method further includes: Take the successfully matched detection targets and prediction targets in each stage together as the multi-stage matching result; Perform object tracking according to the multi-stage matching result.

18. A target matching device, characterized in that, Includes: An acquisition module, configured to acquire a detection target and a prediction target corresponding to the image to be processed; A multi-stage matching module, configured to perform multi-stage matching based on the detection target and the prediction target; wherein, different stages adopt different matching strategies; Wherein, in one stage, use a combined attribute measurement algorithm to determine the similarity between the detection target and the prediction target, and match the detection target and the prediction target according to the similarity; the similarity between the successfully matched detection target and the prediction target is greater than a preset second similarity screening threshold; The detection target is represented in the form of a detection box, and the prediction target is represented in the form of a prediction box; the multi-stage matching module includes: a multi-attribute determination unit, which is used to determine the center point distance, the difference in diagonal length, and the difference in orientation angle between the detection box and the prediction box to be matched; a second similarity determination unit, which is used to determine the minimum bounding box containing the detection box and the prediction box based on the vertex coordinates of the detection box and the vertex coordinates of the prediction box, and obtain the diagonal length of the minimum bounding box; based on the center point distance between the detection box and the prediction box and the diagonal length of the minimum bounding box, determine the center difference evaluation value between the detection box and the prediction box; based on the difference in diagonal length between the detection box and the prediction box and the diagonal length of the minimum bounding box, determine the diagonal difference evaluation value between the detection box and the prediction box; based on the difference in orientation angle and a preset trigonometric function, determine the angle difference evaluation value between the detection box and the prediction box; according to the center difference evaluation value, the diagonal difference evaluation value, and the angle difference evaluation value, determine the similarity between the detection box and the prediction box.

19. An electronic device, characterized in that, The electronic device includes: a processor; a memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the target matching method according to any one of claims 1-17 above.

20. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the target matching method according to any one of claims 1-17 above.

Images