Object tracking method and system for dynamic scene

Abstract

The invention discloses an object tracking method and system for a dynamic scene, and the system comprises the steps: recognizing one or more objects frame by frame after continuous frames are obtained, selecting a target object in a first frame, calculating the distance between the target object and each object in the one or more objects recognized in a second frame, and obtaining the distance between the one or more objects; identifying character strings on the target object and each object of the second frame, and calculating character string similarity between the character string of the target object and the character string in each object of the second frame; the object distance between the target object and each object of the second frame and the character string similarity are used for calculating a comprehensive score, the comprehensive score is used for judging whether the target object of the first frame is any object in the second frame, and when it is judged that the target object also appears in the second frame and a subsequent frame, object tracking can be carried out.

Description

Technical Field

[0001] The specification discloses an object tracking method, specifically a method and system for tracking an object in a dynamic scene based on the object distance and character string similarity in previous and next frames. Background Art

[0002] In the current multi-object tracking algorithm implemented in continuous images, in order to determine whether some objects between the previous and next frames belong to the same object or some are new objects, it is necessary to calculate the matching degree between the objects in the previous and next frames. Through the algorithm calculation, if the matching degree between the two objects in the previous and next frames is high enough, they can be determined to be the same object and given the same object number; on the contrary, if the matching degree between the objects in the previous and next frames is lower than a threshold, the object appearing in the next frame is determined to be a new object and given a new object number.

[0003] Among the commonly known object matching degree calculation methods, for example, an intersection over union (IoU) method for judging the object matching degree based on the degree of position overlap between two objects in previous and next frames, such as the Sort algorithm, and an appearance metric method for calculating the similarity of appearance features of two objects, such as the DeepSort algorithm.

[0004] However, when the conventional algorithm is used in the scene of dynamic license plate recognition, it will face several problems. The first problem is that due to the high relative speed of the approaching vehicle relative to the photographic device used to calculate the object matching, the license plate of the same vehicle in the two frames captured has a large movement distance, which often results in the same license plate position in the previous and next frames not overlapping at all, so it is impossible to judge simply by the degree of overlap of the object position; the second problem is that the appearance of common license plates is mostly light-colored background and dark-colored characters, and the contours and aspect ratios are also similar. The difference in letters is not enough to serve as a sufficient appearance feature, so it is impossible to use only appearance features as a basis for judging whether two license plates belong to the same license plate; the third problem is that in the situation where there are dense parking on the roadside or on the highway where the oncoming lane can be seen, it is often the case that multiple license plates appear in the same position in the picture in a short period of time. Therefore, even if there are objects appearing in the same position in the previous and next frames, it is still impossible to directly determine that they are the same object.

[0005] To solve the above problems, a further license plate feature extraction algorithm is needed. However, if the device that performs object matching is an embedded device, it cannot provide sufficient computing power to extract the features of each license plate in each picture. Summary of the invention

[0006] Based on the problem that it is impossible to judge the same license plate in different images based on the license plate position alone in dynamic scenes, the present disclosure proposes a method and system for object tracking in dynamic scenes. When considering the license plate position, it is not required that the license plates in consecutive frames overlap, nor is it determined to be the same license plate because the license plates in the previous and next frames overlap. When the method is used in a changing scene, the relative distance between two objects (such as license plates) in the previous and next frames is obtained and a distance score is assigned; further, the similarity between each two objects in the previous and next frames is calculated. For license plate recognition, the result of license plate recognition is used to calculate the string similarity between the license plates in the previous and next frames and assign a similarity score. The method performs object recognition and tracking based on the distance score and the similarity score.

[0007] According to an embodiment of the object tracking system for dynamic scenes, the system includes a microcontroller, an image processor and a camera module, wherein the image processor is used to perform object recognition and positioning, and the camera module is used to shoot a video. The microcontroller executes the object tracking method.

[0008] In the object tracking method of dynamic scenes, a video is first shot with a camera module to obtain continuous frames, and the previous and next frames are represented by the first frame and the second frame, and then an image processor is used to identify one or more objects frame by frame, and one of them is selected as the target object. Next, the distance between the target object in the first frame and each of the one or more objects identified in the second frame is calculated to obtain one or more object distances; and then the target object in the first frame and the character string in the one or more objects in the second frame are identified to calculate the character string similarity between the character string in the target object in the first frame and the character string in each object in the second frame.

[0009] In the method, an integrated score is calculated using the object distance and string similarity between the target object in the first frame and each object in the second frame, so that the object tracking system can determine whether the target object in the first frame is any object appearing in the second frame based on the integrated score.

[0010] Preferably, the object distance is the distance between the geometric center of the target object in the first frame and the geometric center of each object in the second frame.

[0011] Preferably, the string in the target object of the first frame and the string in each object of the second frame are license plate numbers composed of multiple English letters and / or numbers, and the string similarity between the strings in the two objects is calculated based on this, which is an edit distance, such as a Levenshtein edit distance.

[0012] Furthermore, when the comprehensive score between the target object in the first frame and each object in the second frame is obtained, the comprehensive score is compared with a score threshold, and it can be determined whether the target object in the first frame is any object appearing in the second frame based on the comparison result with the score threshold.

[0013] Therefore, when it is determined based on the comprehensive score that the same target object appears in multiple consecutive frames, a unique number is assigned to the target object appearing in the multiple consecutive frames to perform object tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1A and Figure 1B A schematic diagram showing a scenario of an object tracking method using a dynamic scene;

[0015] Figure 2 A schematic diagram showing the relationship between the license plates in the previous and next frames in the object tracking method for dynamic scenes;

[0016] Figure 3 An embodiment diagram showing the main functional modules of an object tracking system for dynamic scenes;

[0017] Figure 4 A flow chart showing an embodiment of a method for tracking an object in a dynamic scene;

[0018] Figure 5 Display a license plate tracking embodiment flow chart;

[0019] Figure 6 A schematic diagram showing an embodiment of determining the same license plate in consecutive frames;

[0020] Figure 7 A flowchart of an embodiment of determining the same license plate from consecutive frames in a method for tracking an object in a dynamic scene; and

[0021] Figure 8 A flowchart showing an embodiment of character recognition in a license plate string. DETAILED DESCRIPTION

[0022] In order to solve the problem that conventional algorithms are not suitable for tracking specific objects in dynamically changing scenes, the present disclosure proposes a method and system for tracking objects in dynamic scenes. The proposed method does not require that license plates in consecutive frames overlap, nor does it determine that the license plates are the same just because the license plates in the previous and next frames overlap. Instead, the relative distance between two objects (such as license plates) in the previous and next frames is obtained and a distance score is assigned. On the other hand, the similarity between every two objects in the previous and next frames is calculated, such as identifying the character string on the license plate, and then calculating the character string similarity between the license plates in the previous and next frames, assigning a similarity score. Object recognition and tracking are then performed based on the distance score and the similarity score.

[0023] For scenes where object tracking methods in dynamic scenes can be used, please refer to Figure 1A and Figure 1B Schematic diagram of the situation shown.

[0024] Figure 1AThe display shows the first frame 101 of the continuous frames taken by the camera module in the terminal device (such as a driving recorder). The first frame 101 shows that there are vehicles 11, 12 and 13 in front of the rear vehicle where the terminal device is installed. The object detection model running in the terminal device determines the license plate 110 of the vehicle 11, the license plate 120 of the vehicle 12 and the license plate 130 of the vehicle 13 in the first frame 101. The vehicle 11 is set as the target vehicle, and the license plate 110 of the vehicle 11 is set as the target license plate.

[0025] then Figure 1B The second frame 102 of the continuous frames is shown as a schematic diagram. The second frame 102 still includes the vehicle 11 set as the target vehicle, and the license plate 110 is continuously captured. In the automatic license plate recognition edge computing method, the license plate recognition technology is used to track the vehicle 11 with the same license plate 110 in the continuous frames. In this example, according to the license plate recognition result, the second frame 102 no longer has the vehicles 12 and 13 parked on the roadside. The license plate recognition result determines that another vehicle 14 (license plate 140) parked on the roadside is present, and the vehicle 11 continues to appear.

[0026] Based on the technology of the above system obtaining continuous frames through a photographic module and tracking the same license plate therein in real time, and specifically the photographic module can be a driving recorder in a moving vehicle, the object tracking system proposed in the present disclosure is suitable for electronic devices in moving vehicles, and can achieve the purpose of object tracking in dynamic scenes.

[0027] Figure 2 Schematic diagram showing the relationship between the distance between the license plates in the front and rear frames in the object tracking method for dynamic scenes.

[0028] The first frame 21 of the continuous frames is shown. After license plate recognition, several license plates (211, 212, 213) are obtained. In the object tracking method, one of the license plates is selected as the target license plate 211. In this example, the license plate number is 0001. There are also license plates 212 and 213.

[0029] Next, the object tracking system obtains another frame in the continuous frame, which is shown as the second frame 22 in this example. After object recognition, it is found that the second frame 22 includes multiple license plates (221, 222, 223). In the object tracking method, the distance between the target license plate 211 in the first frame 21 and each license plate in the second frame 22 is calculated. This example shows that according to the position of the target license plate 211 in the first frame 21, the corresponding position of the second frame 22 is marked as the target license plate 211' (license plate 0001). Because in a dynamically changing scene, the target license plate 211 may have changed its position to one of the license plates (221, 222, 223).

[0030] To determine whether the target license plate 211 in the first frame 21 is any of the license plates in the second frame 22, the distance between the target license plate 211 and each of the license plates (221, 222, 223) identified in the second frame 22 is first calculated, and based on the target license plate 211' marked in the second frame 21 (corresponding to the position of the target license plate 211 in the first frame 21), the first object distance d1 between the target license plate 211' and the license plate 221, the second object distance d2 between the target license plate 211' and the license plate 222, and the third object distance d3 between the target license plate 211' and the license plate 223 are calculated. The distance can be calculated as the straight-line distance between the two objects in the image, and the object distance can be the distance between the geometric center of the target object marked in the second frame and the geometric center of each of the other objects.

[0031] By obtaining one or more object distances between the selected target license plate 211 and one or more license plates identified in another frame (which may be the next frame or any frame before or after), the object distance becomes one of the bases for determining whether the target license plate 211 appears in another frame. Figure 2 In the example shown, in the continuous frames captured in a rapidly changing scene, the target license plate 211 (license plate number 0001) selected in the first frame 21 has moved to the license plate 221 in the second frame 22, and the license plate number of the license plate 221 has been identified as 0001 in this example. This example shows that the recognition and tracking of the target license plate 211 in the continuous frames are completed. According to the embodiment, the object tracking system can simultaneously perform recognition and tracking of multiple target license plates.

[0032] It is worth mentioning that for the purpose of object tracking, when identifying the same object in a changing scene, in addition to calculating the distance between the target object (such as a target license plate) and each of one or more objects in different frames as shown in the above embodiment, the feature similarity between the target object and objects in different frames, such as string similarity, will also be considered.

[0033] The license plate is used as the main embodiment. Since the appearance of the license plates of general vehicles is similar, it is impossible to judge whether the same object exists in the previous and next frames from the appearance of the object. Therefore, the feature similarity between objects (such as license plates) is further considered, such as the string similarity. The license plate string is generally a license plate number composed of multiple English letters and / or numbers. The system will consider the string similarity between the target object in the first frame and the string similarity in each object in the second frame. One implementation method can adopt a Levenshtein edit distance. The Levenshtein edit distance is an edit distance. The Levenshtein edit distance refers to the minimum number of edit operations required to convert one string into another between two strings (minimum number of single-character edits between two strings), wherein the minimum number of edit operations can be interpreted as the edit distance of the two strings, which is used as the similarity of the license plate strings in the previous and next frames in the object tracking method proposed in the present disclosure. When calculating the string similarity, the edit distance is 0, that is, the two strings are exactly the same; the edit distance is 1, which means that the two strings are completely different. Therefore, after calculating the minimum number of edit operations and normalizing, an edit distance between 0 and 1 is obtained as the string similarity. Using license plate string similarity can eliminate the problem of being unable to accurately identify license plate content due to light and shadow and various factors.

[0034] Levenshtein edit distance (lev a,b (i,j)) is calculated as in formula 1, where |a| and |b| represent the lengths of the two strings a and b, and the Levenshtein edit distance between the two strings is lev a,b (|a|,|b|), can be obtained from formula 1, where lev a,b (i,j) represents the Levenshtein edit distance between the first i characters of a and the first j characters of b.

[0035] Formula 1:

[0036]

[0037] Therefore, the object tracking method for dynamic scenes proposed in the present disclosure will simultaneously refer to the two calculation results of object distance and edit distance (string similarity), and according to the actual situation of the dynamic scene, such as the speed of the vehicle installed in the electronic device running the object tracking system, the two calculation results of object distance and edit distance can be given a specific ratio (such as 4:6; and can also include a 1:1 ratio) to calculate a comprehensive score that can evaluate whether the objects in the previous and next frames are the same object, and then compare it with the judgment threshold set by the object tracking system to determine the same object in consecutive frames and give an object number.

[0038] Continued reference Figure 3 1 is a diagram showing an embodiment of the main functional modules of the object tracking system for dynamic scenes, which describes various functional modules implemented by software and hardware circuits to implement object tracking using the above-mentioned object distance and edit distance. According to the embodiment, the main functional modules of the object tracking system include a photography module 31, a license plate recognition module 33, a calculation module 35, and a memory unit 37 for temporarily storing processed image data (such as continuous frame data and license plate images, etc.) and values ​​generated by various calculation processes (such as object distance, edit distance, comprehensive score, etc.).

[0039] According to an embodiment, the license plate recognition module 33 may be implemented by an image processor in an electronic device, wherein firmware or software and hardware are used to perform object recognition and positioning, and an embodiment includes executing a license plate recognition program 331 and a string recognition program 333. The operation module 35 may also be implemented by a microcontroller of an electronic device, wherein the object distance calculation program 351, the string similarity calculation program 353 and the license plate tracking program 355 are mainly executed.

[0040] according to Figure 3 The functional module embodiment of the object tracking system shown, wherein the operation process can also refer to Figure 2 A schematic diagram showing the relationship between the license plates in the front and rear frames, and Figure 4 A flow chart of an embodiment of a method for tracking an object in a dynamic scene is shown.

[0041] According to the embodiment process of the object tracking method for dynamic scenes executed by the object tracking system, the camera module 31 in the object tracking system first shoots a video and obtains continuous frames (step S401), and then the license plate recognition program 331 running in the license plate recognition module 33 recognizes one or more license plates frame by frame (step S403), and implements, for example, Figure 2 The license plates in the first frame 21 are displayed, including the target license plate 211 and other license plates 212 and 213, and the license plates 221, 222 and 223 in the second frame 22. It should be noted that the first frame 21 is the currently received image for determining the target license plate, and the second frame 22 may refer to any other frame in the continuous frames, which may be one of the frames before the first frame 21, or may refer to other frames after the first frame 21.

[0042] according to Figure 2The schematic diagram shown shows that after selecting the target object (such as the target license plate 211) in the first frame 21, the object distance calculation program 351 in the computing module 35 implemented by the central processing unit and related circuits in the object tracking system can calculate one or more object distances between the target license plate 211 and one or more objects in the second frame 22 (such as the license plates 221, 222 and 223 in the second frame 22) according to the position of the target license plate 211 in the image (step S405).

[0043] Afterwards, the target object (e.g. Figure 2 The target license plate 211 in the first frame 21) and one or more objects in the second frame (such as Figure 2 The character string of the license plate 221, 222 and 223 in the second frame 22 (step S407). The technology of character recognition in the character string can be referred to Figure 8 A flow chart of an embodiment of character recognition in a license plate string is shown.

[0044] Next, the string similarity calculation program 353 in the operation module 35 can be used to calculate the string similarity between the two objects. In an embodiment, the Levenshtein edit distance is calculated to obtain a string similarity between the string in the target object of the first frame and the string in each object of the second frame, that is, the string similarity between the two license plates (step S409).

[0045] The object distance between the target object of the first frame and one or more objects of the second frame and the string similarity between the two objects are obtained, a distance score is assigned to the object distance between the target object of the first frame and each object in the second frame, and a string similarity score is assigned to the string of the target object of the first frame and the string of each object in the second frame, and then a comprehensive score is calculated according to the distance score and the string similarity score (step S411). According to one embodiment, when calculating the comprehensive score, an average score of the distance score and the string similarity score can be used as the comprehensive score; or a ratio can be set between the distance score and the string similarity score considering the actual dynamic scene, and then the comprehensive score is calculated according to the ratio.

[0046] Afterwards, the result of comparing the comprehensive score with a score threshold set by the system can be used to determine whether the target object in the first frame is any object that appears in the second frame, wherein objects with a comprehensive score greater than or equal to the score threshold can be regarded as being equal to the target object in the first frame and assigned the same number (step S413). After determining that there are identical target objects in consecutive frames, the license plate tracking program 355 in the computing module 35 can be run to perform license plate tracking. For related processes, please refer to Figure 5 The flowchart of the license plate tracking embodiment is shown.

[0047] When the system recognizes the license plates of each frame, it can number them one by one, and then determine whether the same license plate is recognized in consecutive frames based on the comprehensive score obtained by the distance score between the two license plates of different frames and the string similarity score (step S501). If the comprehensive score is less than the score threshold set by the system, it is determined to be a different license plate (no), and if it is determined to be a new license plate of a vehicle, the system will assign a new number (step S503).

[0048] If the comprehensive score is greater than the score threshold, it means that the target license plate in the first frame appears in other frames (yes), including judging that the same target license plate appears in multiple consecutive frames based on the comprehensive score, that is, assigning (including adding and modifying) the same number (step S505), and then performing license plate tracking for the license plates with the same number in multiple consecutive frames (step S507).

[0049] The same and unique number can be assigned to the same license plate in multiple consecutive frames, and the unique number can be marked on the display of the electronic device. The recognized license plate number can also be marked, so that the user can see the images of the same vehicle or license plate in consecutive frames on the display, which is convenient for tracking the vehicle.

[0050] In addition to the above description, several situations that the method may face are proposed here. According to one of the situations, when a single comprehensive score between the target license plate of the first frame and all the license plates of the second frame is obtained, before comparing the score threshold, if it is found that the comprehensive scores calculated from multiple license plates in the second frame or other frames and the target license plate are greater than or equal to the score threshold, the one with the highest comprehensive score can be regarded as equal to the target license plate of the first frame and assigned the same number.

[0051] However, in another case, when multiple license plates are identified in both the first frame and the second frame, there will be exceptions if only the highest comprehensive score is used as the determination of the same license plate. The method for identifying characters in a license plate string proposed in the present disclosure is described in the following implementation scheme.

[0052] Reference first Figure 6 The first frame 61 and the second frame 62 in the continuous frames are respectively used to determine multiple objects (taking multiple license plates as an example), and the object tracking method of the dynamic scene is further used to determine the same license plate from the continuous frames to perform license plate tracking. Figure 7 Flow chart of the embodiment shown.

[0053] Figure 6 The schematic diagram shows that three license plates 1A, 1B and 1C are recognized by using license plate recognition technology in the first frame 61, and three license plates 2D, 2E and 2F are also recognized in another second frame 62 in the continuous frame. Then, the object distance (in this case, the license plate distance) and the character string similarity between different frames are calculated to obtain a comprehensive score to determine the same license plate in different frames.

[0054] At the beginning Figure 7 In the first frame, one of the license plates is selected as the target license plate (step S701), and then Figure 4 The described method calculates the license plate distance and string similarity between the target license plate and all the license plates in the second frame, and a single comprehensive score between the target license plate and each license plate in the second frame can be obtained (step S703). After completing the calculation at this stage, it will be determined whether the calculation is completed (step S705). If there are other license plates in the first frame that have not been processed, steps S701 and S703 are repeated until the calculation of a single comprehensive score is completed for all the license plates in the first frame.

[0055] The calculation results of the individual comprehensive scores among all license plates can refer to the example shown in Table 1, which shows that the individual comprehensive scores of license plate 1A in the first frame and license plates 2D, 2E and 2F in the second frame are 100, 90 and 10 respectively; the individual comprehensive scores of license plate 1B in the first frame and license plates 2D, 2E and 2F in the second frame are 95, 60 and 15 respectively; the individual comprehensive scores of license plate 1C in the first frame and license plates 2D, 2E and 2F in the second frame are 20, 35 and 25 respectively.

[0056] Table 1:

[0057]

[0058] After the calculation of the single comprehensive scores between the multiple license plates in different frames is completed, the overall comprehensive score after the first frame license plate and the second frame license plate are paired one by one is then calculated (step S707). That is to say, in this embodiment process, the basis for judging the same license plate in different frames is not only the highest single comprehensive score, but the overall comprehensive score after the license plates in different frames are paired one by one is considered, for example, the single comprehensive score of the first frame license plate 1A and the second frame license plate 2D is 100, the single comprehensive score of the first frame license plate 1B and the second frame license plate 2E is 60, and the single comprehensive score of the first frame license plate 1C and the second frame license plate 2F is 25, and the overall comprehensive score obtained after this pairing is 185 (100+60+25). However, when such a pairing is not the one with the highest overall comprehensive score, it is not the result of the best pairing.

[0059] In this way, the process of this embodiment will calculate the overall comprehensive scores of all license plate pairs. There are multiple overall comprehensive scores, and the license plate pair with the highest overall comprehensive score is used as the final license plate pairing result (object pairing result) (step S709). Then, the score threshold is used to filter out the same license plates in different frames (step S711). For example, the score threshold is set to 50. In the license plate pairing results, if the comprehensive score is lower than 50, it will be determined to be a different license plate. After that, the license plates of each frame are assigned numbers according to the license plate pairing results (step S701). Among them, those judged to be the same license plate are assigned the same number, and those judged to be different license plates are represented as newly appeared license plates and will be assigned new numbers.

[0060] The above embodiment describes a method for identifying characters in a license plate string. Figure 8 The flowchart of the embodiment of character recognition in the license plate character string is shown.

[0061] According to an embodiment, the object tracking system proposed in the present disclosure may be provided in an electronic device such as a driving recorder, and a camera module (such as Figure 3 The camera module 31) obtains a video composed of continuous frames (step S801), and then uses the software and hardware collaborative license plate recognition module (such as Figure 3 The license plate recognition technology (such as the license plate recognition module 33) executed in Figure 3 The license plate recognition program 331 of the embodiment of the present invention identifies the position of one or more license plates in the image (step S803). For example, an intelligent method can be used to obtain objects in the image that match the characteristics of the license plate, such as identifying the bounding box of the license plate of each vehicle in each frame. After obtaining the image of the license plate, a character string detection technology (such as Figure 3 The character recognition program 333) performs character recognition on each license plate (step S805).

[0062] According to the embodiment of recognizing license plate characters, the computing module in the object tracking system can execute a confidence algorithm for the character string recognition result in the license plate, wherein the confidence level of each character is calculated for each character obtained by recognizing each license plate frame by frame (step S807). The confidence level of the character is about the probability of recognizing each character on the license plate by an intelligent model (a character recognition model that can perform recognition on the letters and numbers that make up the license plate number) and calculating it as a certain letter or number by an intelligent algorithm. The system can set a probability threshold (or confidence threshold), and the one with the highest probability greater than this probability threshold is the recognition result of the character.

[0063] Next, after the confidence of all recognizable letters (all or more than a certain number of letters) in the license plate is calculated, the confidence score of the entire license plate character recognition can be obtained frame by frame according to the confidence of all characters in each license plate (step S809). The confidence score of the license plate is a statistical value of the confidence of multiple characters in the license plate, such as calculating the average value of the confidence of all characters, or adding up the sum, or a comprehensive score through weight setting (for example, giving a higher weight to those with higher confidence). It should be mentioned here that the statistical method of the confidence score of all or multiple characters in the license plate is not limited to the above.

[0064] The above steps S807 and S809 can be used to calculate the confidence score of each license plate in multiple consecutive frames. When the confidence scores of each license plate can be calculated multiple times within a period of time (step S811), the one with the highest confidence score within this period of time can be continuously counted (step S813), or the confidence score of the license plate can be calculated at a time interval in the consecutive frames, and the frame image with the highest confidence score can be obtained before the license plate disappears. At this time, when the same license plate appears in multiple consecutive frames, the confidence score of each frame is calculated, and the one with the highest confidence score can be obtained. The recognition result of the license plate with the highest confidence score can be used as the recognition result (step S815), and the license plate string recognition is completed.

[0065] To summarize, one of the purposes of the object tracking method and system for dynamic scenes described in the above embodiments is to be able to perform multi-object tracking in various scenarios, specifically for changing scenes, and the method specifically calculates the object distance between objects in different frames in continuous frames and the string similarity between objects, and calculates a comprehensive score based on the object distance and the string similarity. Based on this, the same object in the continuous frames is judged and object tracking can be performed. This can solve the problem that the conventional license plate tracking method cannot effectively track objects in dynamic scenes.

[0066] The contents disclosed above are only preferred feasible embodiments of the present invention, and are not intended to limit the scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention's specification and drawings are included in the scope of the present invention.

Claims

1. A method for tracking an object in a dynamic scene, characterized in that: The method comprises: Shoot a video and obtain continuous frames; Identify one or more objects frame by frame, and select a target object in a first frame; Calculating the distance between the target object in the first frame and each of the one or more objects identified in a second frame to obtain one or more object distances; Recognize the target object in the first frame and the character string in the one or more objects in the second frame; Calculating a string similarity between the string in the target object in the first frame and the string in each object in the second frame; Calculating a comprehensive score using the object distance between the target object in the first frame and each object in the second frame and the character string similarity; and It is determined according to the comprehensive score whether the target object in the first frame is any object appearing in the second frame.

2. The object tracking method in a dynamic scene according to claim 1, characterized in that: The object distance is the distance between a geometric center of the target object in the first frame and a geometric center of each object in the second frame.

3. The object tracking method in a dynamic scene according to claim 2, characterized in that: The character string in the target object of the first frame and the character string in each object of the second frame are license plate numbers composed of a plurality of English letters and / or numbers, and the character string similarity between the character strings in the two objects is an edit distance.

4. The object tracking method in a dynamic scene according to claim 3, characterized in that: The edit distance is a Levenshtein edit distance, which is a minimum number of edit operations required to convert strings in two objects from one string to another string.

5. The object tracking method in a dynamic scene according to claim 1, characterized in that: The one or more objects identified frame by frame are one or more license plates, and the one or more license plates obtained by identification are numbered one by one.

6. The object tracking method for a dynamic scene according to any one of claims 1 to 5, characterized in that: When the comprehensive score between the target object in the first frame and each object in the second frame is obtained, the comprehensive score is compared with a score threshold, and it is determined whether the target object in the first frame is any object appearing in the second frame based on the comparison result with the score threshold.

7. The object tracking method in a dynamic scene according to claim 6, characterized in that: When it is determined according to the comprehensive score that the same target object appears in multiple consecutive frames, a unique number is assigned to the target object appearing in the multiple consecutive frames to perform object tracking.

8. The object tracking method in a dynamic scene according to claim 6, characterized in that: The object distance between the target object in the first frame and each object in the second frame is assigned a distance score, and the string similarity between the string of the target object in the first frame and the string of each object in the second frame is assigned a string similarity score, and then the comprehensive score is calculated based on the distance score and the string similarity score.

9. The object tracking method in a dynamic scene according to claim 8, characterized in that: When calculating the comprehensive score, an average score of the distance score and the string similarity score is used as the comprehensive score; or a ratio is set between the distance score and the string similarity score, and the comprehensive score is calculated according to the ratio.

10. The object tracking method in a dynamic scene according to claim 9, characterized in that: The object whose comprehensive score is greater than or equal to the score threshold is regarded as being equal to the target object in the first frame and is assigned the same number.

11. The object tracking method in a dynamic scene according to claim 9, characterized in that: When the first frame includes a plurality of objects and the second frame includes a plurality of objects, the plurality of objects in the first frame are selected one by one as the target object, and a single comprehensive score between the target object in the plurality of objects in the first frame and each object in the second frame is calculated; A plurality of overall comprehensive scores are calculated by summing up the objects in the first frame after pairing them one by one with the objects in the second frame, and the object pairing result with the highest overall comprehensive score is adopted. The same objects in different frames are then determined based on the score threshold, and the same number is assigned to the same objects.

12. A dynamic scene object tracking system, applied in an electronic device, characterized in that: The system comprises: A microcontroller; an image processor, electrically connected to the microcontroller, for performing object recognition and positioning; a camera module, electrically connected to the image processor, for shooting a video; The microcontroller executes an object tracking method, including: Using the camera module to shoot the video to obtain consecutive frames, including a first frame and a second frame; Using the image processor to identify one or more objects frame by frame, and selecting a target object in the first frame; Calculating the distance between the target object in the first frame and each of the one or more objects identified in the second frame to obtain one or more object distances; Recognize the target object in the first frame and the character string in the one or more objects in the second frame; Calculating a string similarity between the string in the target object in the first frame and the string in each object in the second frame; Calculating a comprehensive score using the object distance between the target object in the first frame and each object in the second frame and the character string similarity; and It is determined according to the comprehensive score whether the target object in the first frame is any object appearing in the second frame.

13. The object tracking system for dynamic scenes according to claim 12, characterized in that: The object distance is the distance between a geometric center of the target object in the first frame and a geometric center of each object in the second frame.

14. The object tracking system for dynamic scenes according to claim 12, characterized in that: The character string in the target object of the first frame and the character string in each object of the second frame are license plate numbers composed of a plurality of English letters and / or numbers, and the character string similarity between the character strings in the two objects is an edit distance.

15. The object tracking system for dynamic scenes according to claim 12, characterized in that: The one or more objects identified frame by frame are one or more license plates, and the one or more license plates obtained by identification are numbered one by one.

16. The object tracking system for dynamic scenes according to claim 15, characterized in that: The license plate numbers of the one or more license plates identified frame by frame are displayed on a display of the electronic device.

17. The object tracking system for dynamic scenes according to any one of claims 12 to 16, characterized in that: In the object tracking method, when the comprehensive score between the target object in the first frame and each object in the second frame is obtained, the comprehensive score is compared with a score threshold, and based on the comparison result with the score threshold, it is determined whether the target object in the first frame is any object appearing in the second frame; wherein, when it is determined based on the comprehensive score that the same target object appears in multiple consecutive frames, a unique number is assigned to the target object appearing in the multiple consecutive frames for object tracking.

18. The object tracking system for dynamic scenes according to claim 17, characterized in that: The object distance between the target object in the first frame and each object in the second frame is assigned a distance score, and the string similarity between the string of the target object in the first frame and the string of each object in the second frame is assigned a string similarity score, and then the comprehensive score is calculated based on the distance score and the string similarity score.

19. The object tracking system for dynamic scenes according to claim 18, characterized in that: When calculating the comprehensive score, an average score of the distance score and the string similarity score is used as the comprehensive score; or a ratio is set between the distance score and the string similarity score, and the comprehensive score is calculated according to the ratio.

20. The object tracking system for dynamic scenes according to claim 19, characterized in that: The object whose comprehensive score is greater than or equal to the score threshold is regarded as being equal to the target object of the first frame and is assigned the same number; when the first frame includes a plurality of objects and the second frame includes a plurality of objects, the plurality of objects of the first frame are selected one by one as the target object, and a single comprehensive score between the target object of the plurality of objects of the first frame and each object of the second frame is calculated; A plurality of overall comprehensive scores are calculated by summing up the objects in the first frame after pairing them one by one with the objects in the second frame, and the object pairing result with the highest overall comprehensive score is adopted. The same objects in different frames are then determined based on the score threshold, and the same number is assigned to the same objects.

Images