A target tracking and object recognition method based on angle interval

Abstract

The application discloses a target tracking and object identification method based on an angle interval, identifies objects in a camera shooting picture based on a YOLO model, records object categories and coordinates of the objects in the shooting picture, calculates an angle interval of the identified objects, extracts and records feature points of each object saved in each angle interval, compares a result recorded by a new frame of picture with a last frame after a new frame of picture is acquired, judges objects in adjacent angle intervals as the same object when the objects have the same category, an area difference value is within a preset threshold range and a distance between feature points is within a preset distance threshold range, and updates a motion track of the object. The scheme uses the angle interval to identify the object and judge whether the motion track belongs to the same object, occupies less resources, runs faster, and is more suitable for scenes with limited computing power and real-time requirements.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a target tracking and object recognition method based on angle range. Background Technology

[0002] Traditional target tracking algorithms typically rely on IOU matching and Kalman filtering to determine whether moving trajectories belong to the same object. When used on devices with limited computing power, this can lead to high resource consumption, slow operation, or even crashes, making them unsuitable for scenarios with limited computing power and real-time requirements. For example, in a falling object alarm scenario, an object or camera may disappear from the frame captured by one camera and then reappear in the frame captured by another. In such scenarios, traditional target tracking algorithms cannot determine whether they are the same object. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a target tracking and object recognition method based on angular intervals.

[0004] This invention is achieved using the following technical solution:

[0005] A target tracking and object recognition method based on angular intervals includes the following steps:

[0006] Step S1: Identify objects in the camera's captured image based on the YOLO model, and record the object category and the object's coordinates in the captured image;

[0007] Step S2: Calculate the angle intervals of the identified objects. Record one or more categories of objects in each angle interval. For a single angle interval where multiple objects of the same category appear, record only the object with the largest area.

[0008] Step S3: Extract and record the feature points of each object stored within each angle interval;

[0009] Step S4: After acquiring a new frame, repeat steps S1 to S3 to obtain the new frame data; compare the results of the new frame with the previous frame, and determine that objects with the same category, area difference within a preset threshold range, and distance between feature points within a preset distance threshold range appearing in adjacent angle intervals are the same object, and update the motion trajectory of the object.

[0010] Specifically, step S1 includes the following sub-steps:

[0011] Step S11: Preprocess the single-frame image data, load the YOLO model using the ncnn deep learning inference framework, input the preprocessed image data and obtain the output feature map;

[0012] Step S12: Define the downsampling rate of the multi-scale feature map to cover target detection at different scales, and generate the coordinates and corresponding step size of each grid point;

[0013] Step S13: Decode the bounding box parameters output by the model, filter out recognition results with confidence scores below a preset threshold range, obtain candidate box data, sort them according to confidence scores, and remove overlapping candidate boxes using NMS.

[0014] Step S14: Convert the coordinates and dimensions of the filtered candidate boxes from the model input space to the original image space to perform coordinate space transformation, and finally generate object bounding box data.

[0015] Specifically, the preprocessing of the image data includes image scaling, image format conversion, and normalization.

[0016] Specifically, the candidate box data includes the object category, confidence level, and the coordinates and dimensions of the candidate box.

[0017] Specifically, the calculation of the angle interval in step S2 includes:

[0018] Set the reference angle for the image captured by the camera: set the leftmost angle to 0°, the middle angle to 90°, and the rightmost angle to 180°.

[0019] Select a reference camera and record the camera's deflection angle when the reference camera captures the first frame as the original angle;

[0020] For subsequent images captured by the reference camera, the actual angle range of the object is set to the angle of the object in the captured image plus the deflection angle of the camera.

[0021] Specifically, the selection of the reference camera includes: when there is only one camera, that camera is the reference camera; when there are multiple cameras, the camera directly in front is selected as the reference camera, and the other cameras are non-reference cameras.

[0022] Specifically, in the non-reference camera, the actual angle range of the object is set as the angle of the object in the shooting frame plus the deflection angle of the camera and the deviation angle between the shooting frames of the non-reference camera and the reference camera.

[0023] Specifically, step S3, feature point extraction and recording, includes: using OpenCV image processing to convert the image within the object's bounding box into a grayscale image, running the SIFT algorithm to extract and record feature points in the grayscale image.

[0024] Specifically, step S4 further includes: saving the information of each object for a preset time, deleting the information of the object after the preset time has expired; and marking the object as a new object if it reappears in the screen.

[0025] The beneficial effects of this invention are as follows: The target tracking and object recognition method proposed in this invention uses an angle range to determine whether the motion trajectory belongs to the same object, which consumes less resources, runs faster, and is more suitable for scenarios with limited computing power and real-time requirements, especially falling object alarm scenarios; it can acquire information such as the movement speed and deflection angle of the camera in real time and can identify the images captured by multiple moving cameras. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the target tracking and object recognition method based on angle intervals in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0030] The following is in conjunction with the appendix Figure 1 The following describes some embodiments of the present invention in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0031] This invention proposes a target tracking and object recognition method based on angle ranges. It identifies objects appearing in videos and continuously records their motion trajectories, making it suitable for scenarios involving falling objects, such as... Figure 1 As shown, the specific steps include:

[0032] Step S1: Identify objects in the camera's captured image based on the YOLO model, and record the object category and the object's coordinates in the captured image;

[0033] Step S2: Calculate the angle intervals of the identified objects. Record one or more categories of objects in each angle interval. For a single angle interval where multiple objects of the same category appear, record only the object with the largest area.

[0034] Step S3: Extract and record the feature points of each object stored within each angle interval;

[0035] Step S4: After acquiring a new frame, repeat steps S1 to S3 to obtain the new frame data; compare the results of the new frame with the previous frame, and determine that objects of the same category, similar area (area difference within a preset threshold range), and similar feature points (distance between feature points within a preset distance threshold range) appearing in adjacent angle intervals are the same object, and update the motion trajectory of the object.

[0036] In one embodiment, target tracking and object recognition based on angle intervals includes the following:

[0037] 1. Use YOLO to identify objects in the captured image, and record the categories of these objects and their coordinates in the captured image.

[0038] In this embodiment, recording the object's category and its coordinates in the captured image includes the following steps:

[0039] (1) Preprocessing of single-frame image data, including image size scaling, image format conversion, normalization and other operations;

[0040] (2) Load the YOLO model using ncnn, input the preprocessed image data and obtain the output feature map;

[0041] (3) Define the downsampling rate (8x / 16x / 32x) of the multi-scale feature map to cover target detection at different scales, and generate the coordinates and corresponding step size of each grid point;

[0042] (4) Decode the bounding box parameters output by the model, filter out recognition results with low confidence (confidence is below the preset threshold range), and obtain candidate box data (including object category, confidence, coordinates and length and width of candidate box).

[0043] (5) Sort according to confidence level and remove overlapping candidate boxes by NMS;

[0044] (6) Coordinate space transformation: The coordinates and dimensions of the filtered candidate boxes are transformed from the model input space to the original image space, and finally the object bounding box data is generated.

[0045] In this embodiment, the object bounding box is a rectangular box that can just select the object. The object bounding box data includes the object category (e.g., person, basketball, etc.), confidence score (the probability that the object belongs to the category), the coordinates and length and width of the bounding box in the image; the coordinates and length and width of the object bounding box are in pixels; the object center point used in subsequent processes is the center point of the object bounding box.

[0046] 2. Calculate the angular intervals in which the center points of these objects are located. Multiple types of objects can be recorded in each angular interval, but if multiple objects of the same type appear in a single angular interval, only the object with the largest area is recorded.

[0047] In this embodiment, the angle interval is calculated as follows:

[0048] The leftmost angle of the image captured by the camera is 0°, the middle angle is 90°, and the rightmost angle is 180° (these angles refer to the angles of the center point of the object in the image captured by the camera, not the angles calculated at the end).

[0049] The original angle is recorded as the deflection angle of the reference camera when it takes the first frame (if there are multiple cameras working, take the camera directly in front; if there is only one camera working, take that camera).

[0050] In the subsequent images captured by the reference camera, the actual angle range of the object is the sum of the angle of the object in the captured image and the camera's deflection angle. For example, if the angle of the object in the image is 70°~80°, and the camera's shooting angle is rotated 30° to the right, then the actual angle range of the object is 100°~110°.

[0051] For images captured by other cameras, the actual angle range of an object is the angle of the object in the captured image plus the camera's deflection angle and the deviation angle between the captured image of this camera and the image captured by the reference camera. For example, if the rear camera (located directly behind the reference camera, with a deviation angle of 180°) captures an object at an angle of 70°~80°, and the rear camera's shooting angle is rotated 30° to the right compared to the first frame, then the actual angle range of the object is 280°~290°.

[0052] Only the largest object is recorded because in a falling object alarm scenario, the larger the area occupied by the object in the captured image, the closer the object is to the camera and the higher the risk.

[0053] 3. Extract and record the feature points of each object within each angle range. In this embodiment, the feature points are extracted by converting the image within the object's bounding box to a grayscale image using OpenCV, and then running the SIFT algorithm to extract and record the feature points in the grayscale image.

[0054] 4. Each time a new frame is acquired, the recorded result of that frame is compared with the previous frame. Objects of the same category, similar area, and similar feature points appearing in adjacent angle intervals are identified as the same object, and the motion trajectory of the object is updated.

[0055] In this embodiment, the motion trajectory can be used to calculate the object's speed and the distance between it and the shooting device.

[0056] 5. Information about each object will be saved for a period of time. After the time expires, the object's information will be deleted. If the object reappears in the scene later, it will be marked as a new object.

[0057] This method acquires information such as the camera's motion speed and deflection angle in real time, enabling the identification of images captured by multiple moving cameras. For example, when an object or camera is moving, it may disappear from the image captured by one camera and then reappear in the image captured by another. Traditional target tracking algorithms cannot determine whether they are the same object, but this method can. Specific examples are provided below for illustration:

[0058] In this embodiment, two cameras are placed one in front of the other on the same horizontal plane, 30cm apart, with an elevation angle of 0° for both. The shooting angles of the center points of the two cameras differ by 180° in the horizontal direction. One camera captures the view in front and is called the front-facing camera, while the other captures the view behind and is called the rear-facing camera. The front-facing camera is used as the reference camera, with the leftmost angle of its captured image marked as 0° and the rightmost angle marked as 180°. Similarly, the leftmost angle of the rear-facing camera's captured image is marked as 180° and the rightmost angle as 360°.

[0059] The 30cm gap between the two cameras is a blind zone in the shooting scene. When testing the situation where an object disappears and reappears in the shooting scene, the object needs to stay in the blind zone. The resolution of the video shot by both cameras is 1920*1080, and the video frame rate is 25.

[0060] The test personnel were instructed to move clockwise from the center of the front-facing camera's view to the rear-facing camera's view, and then back to the front-facing camera's view. The first time they passed through the blind spot, they did not stop, but the second time they passed through the blind spot, they stopped for 10 seconds.

[0061] The first blind spot the testers passed through was the area between the far right of the front camera's view and the far left of the rear camera's view. The testers did not stop here because they needed to test whether an object could be marked as the same object after it briefly disappeared and reappeared in the view.

[0062] The blind spot the testers passed through for the second time was the area between the far right of the rear camera's view and the far left of the front camera's view. The testers stayed here for 10 seconds because they needed to test whether an object could be marked as a new object after disappearing from the view for a long time and then reappearing.

[0063] The image data captured by the two cameras is preprocessed, including image scaling, image format conversion, and normalization.

[0064] Use ncnn to load the YOLO model, input preprocessed image data, and obtain the output feature map.

[0065] Define the downsampling rate (8x / 16x / 32x) of the multi-scale feature map to cover target detection at different scales, and generate the coordinates and corresponding step size of each grid point.

[0066] The bounding box parameters output by the decoding model are used to filter out low-confidence recognition results and obtain candidate box data (including object category, confidence level, coordinates and dimensions of the candidate box).

[0067] Sort by confidence level and remove overlapping candidate boxes using NMS.

[0068] Coordinate space transformation converts the coordinates and dimensions of the filtered candidate boxes from the model input space to the original image space, finally generating object bounding box data.

[0069] An object bounding box is a rectangular box that can just select the object. The object bounding box data includes the object category (e.g., person, basketball, etc.), confidence score (the probability that the object belongs to the category), and the coordinates and dimensions of the bounding box in the image. The coordinates and dimensions of the object bounding box are in pixels. The center point of the object used in subsequent processes is the center point of the object bounding box.

[0070] Record the bounding box data of each object appearing in each frame, the angle range of the center point of the bounding box, the object category, frame time, and other information. If there are multiple objects of the same category in the same angle range, only the data of the object with the largest bounding box area is saved.

[0071] In this embodiment, each 18° is an angle interval, and there are a total of 20 angle intervals. The leftmost angle interval of the image captured by the front camera is 0°~18°, and the rightmost angle interval is 162°~180°. The leftmost angle interval of the image captured by the rear camera is 180°~198°, and the rightmost angle interval is 342°~360°.

[0072] If there are multiple objects of the same type within the same angular range, only the data with the largest bounding box area is saved because this algorithm is currently mainly used in falling object alarm scenarios. The larger the area, the closer the object is to the camera and the higher the degree of danger.

[0073] Use OpenCV to convert the image within the bounding box of the object into a grayscale image, and then use OpenCV to run the SIFT algorithm to extract and record the feature points in the grayscale image.

[0074] Each time a new frame is acquired, the recorded result of that frame is compared with the previous frame. Objects of the same type, similar area, and similar feature points appearing in adjacent angle intervals are identified as the same object, and the motion trajectory of the object is updated.

[0075] The condition for determining whether the areas are close is that the area of ​​the object detection box in the second frame is between 70% and 130% of the area of ​​the object detection box in the first frame.

[0076] The condition for determining whether feature points are close is that the feature points in the object detection box of the second frame and the feature points in the object detection box of the first frame have an overlap rate of more than 70%.

[0077] According to calculations, the angle range of the tester in the first frame is 90°~108°. In the frame before the tester disappears from the front camera's view, the angle range of the tester is 162°~180°. In the first frame where the tester appears in the rear camera's view, the angle range of the tester is 180°~198°.

[0078] The key data related to the testers identified in the experiment are shown in the table below:

[0079] Table 1 Test Data Table

[0080]

[0081] In frames 15-72 of the experiment, the tester moved from the center of the front camera's view to the far left of the rear camera's view. In each adjacent frame, the area of ​​the tester's object detection box was close, the feature points were close, and the angle range of the center point of the object detection box was also close. In the end, they were identified as the same object, the detection box IDs remained consistent, the target tracking was successful, and the experimental results met expectations.

[0082] In frames 183-447, the detection box data with ID 3 was deleted because the disappearance time exceeded the threshold. When the tester reappeared in the shooting scene, it was identified as a new object, and the detection box ID changed. The experimental results were as expected.

[0083] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0084] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A target tracking and object recognition method based on angular intervals, characterized in that, Includes the following steps: Step S1: Identify objects in the camera's captured image based on the YOLO model, and record the object category and the object's coordinates in the captured image; Step S2: Calculate the angle intervals of the identified objects. Record one or more categories of objects in each angle interval. For a single angle interval where multiple objects of the same category appear, record only the object with the largest area. The calculation of the angle range includes: Set the reference angle for the image captured by the camera: set the leftmost angle to 0°, the middle angle to 90°, and the rightmost angle to 180°. A reference camera is selected, and the deflection angle of the camera when the reference camera captures the first frame is recorded as the original angle. The selection of the reference camera includes: when there is only one camera, that camera is the reference camera; when there are multiple cameras, the camera directly in front is selected as the reference camera, and the other cameras are non-reference cameras. Among the non-reference cameras, the actual angle range of the object is set as the angle of the object in the captured image plus the deflection angle of the camera and the deviation angle between the captured image of the camera and the reference camera. For subsequent images captured by the reference camera, the actual angle range of the object is set to the angle of the object in the captured image plus the deflection angle of the camera. Wherein, angle represents the angle of the object's center point in the camera's captured image, and the object's center point is the center point of the object's bounding box. Step S3: Extract and record the feature points of each object stored within each angle interval; Step S4: After acquiring a new frame, repeat steps S1 to S3 to obtain the new frame data; compare the results of the new frame with the previous frame, and determine that objects with the same category, area difference within a preset threshold range, and distance between feature points within a preset distance threshold range appearing in adjacent angle intervals are the same object, and update the motion trajectory of the object.

2. The target tracking and object recognition method based on angular intervals as described in claim 1, characterized in that, Step S1 specifically includes the following sub-steps: Step S11: Preprocess the single-frame image data, load the YOLO model using the ncnn deep learning inference framework, input the preprocessed image data and obtain the output feature map; Step S12: Define the downsampling rate of the multi-scale feature map to cover target detection at different scales, and generate the coordinates and corresponding step size of each grid point; Step S13: Decode the bounding box parameters output by the model, filter out recognition results with confidence scores below a preset threshold range, obtain candidate box data, sort them according to confidence scores, and remove overlapping candidate boxes using NMS. Step S14: Convert the coordinates and dimensions of the filtered candidate boxes from the model input space to the original image space to perform coordinate space transformation, and finally generate object bounding box data.

3. The target tracking and object recognition method based on angular intervals as described in claim 2, characterized in that, The preprocessing of the image data includes image scaling, image format conversion, and normalization.

4. The target tracking and object recognition method based on angular intervals as described in claim 2, characterized in that, The candidate box data includes the object category, confidence level, and the coordinates, length, and width of the candidate box.

5. The target tracking and object recognition method based on angular intervals as described in claim 1, characterized in that, The extraction and recording of feature points in step S3 includes: using OpenCV image processing to convert the image within the object's bounding box into a grayscale image, running the SIFT algorithm to extract and record the feature points in the grayscale image.

6. The target tracking and object recognition method based on angular intervals as described in claim 1, characterized in that, Step S4 further includes: saving the information of each object for a preset saving time, deleting the information of the object after the preset saving time has expired; if the object reappears in the screen, it is marked as a new object.

Images