Region tracking method and apparatus, electronic device, and storage medium

Abstract

The application provides a region tracking method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: performing feature map extraction on a reference frame image and a to-be-detected frame image respectively to obtain a reference frame feature map and a to-be-detected frame feature map; performing feature point matching on the reference frame feature map and the to-be-detected frame feature map; determining the position of a to-be-detected frame calibration point according to the position of a reference frame calibration point in the reference frame feature map, the positions of a plurality of reference frame feature points in a set range around the reference frame calibration point, and the positions of a plurality of matched to-be-detected frame feature points in the to-be-detected frame feature map, and calibrating a to-be-detected frame target region in the to-be-detected frame image. The method can track the target region even if the target region has a moving pedestrian, moving goods or the like when a shooting device such as a camera moves.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to a region tracking method, apparatus, electronic device, and storage medium. Background Technology

[0002] Region tracking tasks refer to defining a region within an image captured by a camera or other imaging device, and then performing certain applications within that region, such as counting the number of people in that region. These applications pre-mark a polygonal region within the image captured by the camera. However, the camera may sometimes move for various reasons, causing the marked region to change its position within the image, necessitating remarking.

[0003] In related technologies, a similar task—object tracking—is mainly used instead. Object tracking first uses a model to detect specific objects and extract features. Then, it performs detection across multiple frames and compares the similarity of the extracted features to determine consistency. However, object tracking tracks objects, while region tracking tracks specific labeled regions. In real-world scenarios, regions are not static objects; pedestrians, goods, etc., may move. When the camera moves, object tracking cannot ignore the dynamic movement of objects and only capture the movement of the labeled region, thus failing to achieve region tracking. Summary of the Invention

[0004] This application discloses a method, apparatus, electronic device, and storage medium for area tracking.

[0005] The first aspect of this application proposes a region tracking method, comprising: acquiring a reference frame image and a frame to be detected image captured sequentially by an imaging device; extracting feature maps from the reference frame image and the frame to be detected image respectively to obtain a reference frame feature map corresponding to the reference frame image and a frame to be detected feature map corresponding to the frame to be detected image; performing feature point matching on the reference frame feature points in the reference frame feature map and the frame to be detected feature points in the frame to be detected feature map; determining the position of the frame to be detected calibration point corresponding to the reference frame calibration point in the frame to be detected feature map based on the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple frame to be detected feature points in the frame to be detected feature map that match the multiple reference frame feature points; and marking the target region of the frame to be detected in the frame to be detected image based on the position of the frame to be detected calibration point.

[0006] The region tracking method of this application embodiment extracts feature maps from a reference frame image and a frame to be detected image captured sequentially by the imaging device to obtain corresponding reference frame feature maps and frame to be detected feature maps. Feature point matching is performed on the reference frame feature points in the reference frame feature map and the frame to be detected feature points in the frame to be detected feature map. Based on the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, the position of multiple reference frame feature points within a set range around the reference frame calibration point, and the position of multiple frame to be detected feature points in the frame to be detected feature map that match multiple reference frame feature points, the position of the frame to be detected calibration point corresponding to the reference frame calibration point in the frame to be detected feature map is determined. The target region of the frame to be detected is marked in the frame to be detected image based on the position of the frame to be detected calibration point. This application embodiment extracts and matches feature points from a reference frame image and a frame image to be detected. Based on the position of the reference frame calibration point in the reference frame image and the positions of multiple matching feature points around the calibration point of the target area in the two frames, the position of the calibration point of the frame image to be detected is determined, thereby determining the position of the target area in the frame image to be detected. Even if pedestrians or goods move in the target area, the tracking of the target area can be achieved when the shooting device, such as a camera, moves.

[0007] A second aspect of this application provides a region tracking device, comprising: an acquisition module configured to acquire a reference frame image and a frame to be detected image captured sequentially by an imaging device; an extraction module configured to extract feature maps from the reference frame image and the frame to be detected image respectively, to obtain a reference frame feature map corresponding to the reference frame image and a frame to be detected feature map corresponding to the frame to be detected image; a matching module configured to perform feature point matching on the reference frame feature map and the frame to be detected feature map; a determination module configured to determine the position of a frame to be detected calibration point corresponding to the reference frame calibration point in the frame to be detected image based on the position of a reference frame calibration point of a target region of the reference frame in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple frame to be detected feature points in the frame to be detected image that match the multiple reference frame feature points; and a calibration module configured to calibrate the target region of the frame to be detected in the frame to be detected image based on the position of the frame to be detected calibration point.

[0008] The region tracking device of this application embodiment extracts feature maps from the reference frame image and the frame to be detected image captured sequentially by the shooting device to obtain corresponding reference frame feature maps and frame to be detected feature maps. It performs feature point matching on the reference frame feature points in the reference frame feature map and the frame to be detected feature points in the frame to be detected feature map. Based on the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, the position of multiple reference frame feature points within a set range around the reference frame calibration point, and the position of multiple frame to be detected feature points in the frame to be detected feature map that match the multiple reference frame feature points, it determines the position of the frame to be detected calibration point in the frame to be detected feature map that corresponds to the reference frame calibration point. Based on the position of the frame to be detected calibration point, it marks the target region of the frame to be detected in the frame to be detected image. This application embodiment extracts and matches feature points from a reference frame image and a frame image to be detected. Based on the position of the reference frame calibration point in the reference frame image and the positions of multiple matching feature points around the calibration point of the target area in the two frames, the position of the calibration point of the frame image to be detected is determined, thereby determining the position of the target area in the frame image to be detected. Even if pedestrians or goods move in the target area, the tracking of the target area can be achieved when the shooting device, such as a camera, moves.

[0009] A third aspect of this application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the region tracking method as described in the first aspect of the application above.

[0010] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to perform the region tracking method as described in the first aspect above.

[0011] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0012] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0013] Figure 1 A schematic flowchart illustrating a region tracking method provided in an embodiment of this application;

[0014] Figure 2 This is a schematic diagram of the reference frame image;

[0015] Figure 3This is a schematic diagram of the frame image to be detected;

[0016] Figure 4 A schematic diagram of the target region marked in the reference frame image;

[0017] Figure 5 This is a schematic diagram of the feature point matching map;

[0018] Figure 6 A schematic diagram illustrating the range defined around the reference frame calibration point;

[0019] Figure 7 A schematic diagram of the marked target region in the frame image to be detected;

[0020] Figure 8 A flowchart illustrating a region tracking method provided in another embodiment of this application;

[0021] Figure 9 This is a schematic diagram of the structure of an area tracking device provided in an embodiment of this application;

[0022] Figure 10 A block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0024] Image processing is the technology of using computers to analyze images to achieve desired results. It mainly includes image digitization, image enhancement and restoration, image data encoding, image segmentation, and image recognition.

[0025] The following description, with reference to the accompanying drawings, describes a region tracking method, apparatus, electronic device, and storage medium according to embodiments of this application.

[0026] Figure 1 This is a schematic flowchart of a region tracking method provided in an embodiment of this application.

[0027] like Figure 1 As shown, the region tracking method in this application embodiment may specifically include the following steps:

[0028] S101, acquire the reference frame image and the frame image to be detected captured sequentially by the imaging device.

[0029] Specifically, the execution entity of the area tracking method in this application embodiment can be the area tracking device provided in this application embodiment. The area tracking device can be a hardware device with data processing capabilities and / or the necessary software to drive the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other devices. The user terminal includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, and vehicle terminals.

[0030] In this embodiment, the capturing device may specifically include an image acquisition device such as a camera. The capturing device captures an image, designated as a reference frame image, and after a certain interval, captures another image, designated as the frame to be detected. The frame to be detected can be an image captured after the capturing device has moved, or an image captured when the capturing device has not moved. The region tracking device acquires the reference frame image and the frame to be detected image captured sequentially by the capturing device. The reference frame image is as follows: Figure 2 As shown, the frame image to be detected is as follows: Figure 3 As shown, by comparison Figure 2 and Figure 3 This confirms that the filming device has been moved.

[0031] The target region is marked in the reference frame image and is denoted as the reference frame target region, such as... Figure 4 As shown. The purpose of this application embodiment is to re-mark the target region in the image to be detected, i.e., the target region of the frame to be detected. If the image to be detected was captured after the capturing device moved, the position of the target region in the image to be detected also shifts compared to the position of the target region in the reference frame image. If the image to be detected was captured without the capturing device moving, the position of the target region in the image to be detected does not shift compared to the position of the target region in the reference frame image.

[0032] S102, feature maps are extracted from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map corresponding to the reference frame image and the frame image to be detected corresponding to the frame image to be detected.

[0033] Specifically, the region tracking device extracts feature maps from the reference frame image acquired in step S101 to obtain a feature map corresponding to the reference frame image, i.e., the reference frame feature map. The region tracking device also extracts feature maps from the frame image to be detected acquired in step S101 to obtain a feature map corresponding to the frame image to be detected, i.e., the frame to be detected feature map. The reference frame feature map includes feature points extracted from the reference frame image, i.e., reference frame feature points, and the frame to be detected feature map includes feature points extracted from the frame image to be detected, i.e., frame to be detected feature points.

[0034] S103, perform feature point matching between the reference frame feature points in the reference frame feature map and the detection frame feature points in the detection frame feature map.

[0035] Specifically, the region tracking device performs feature point matching on the reference frame feature points in the reference frame feature map obtained in step S102 and the detectable frame feature points in the detectable frame feature map to obtain a feature point matching map as shown below. Figure 5 As shown. Feature point matching involves finding the same feature point in two frames of images. Generally, the similarity between two feature points in the two frames is compared to determine if they match. Each feature point corresponds to a descriptor, usually represented by a vector. The similarity between two feature points can be obtained by calculating the similarity between the two vectors.

[0036] S104. Based on the position of the reference frame calibration point in the target region of the reference frame as indicated in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple target frame feature points in the target frame feature map that match multiple reference frame feature points, determine the position of the target frame calibration point in the target frame feature map that corresponds to the reference frame calibration point.

[0037] Specifically, the calibration point of the target area is the corner point of the target area, the calibration point of the reference frame is the corner point of the target area of ​​the reference frame, and the calibration point of the frame to be detected is the corner point of the target area of ​​the frame to be detected. The set range is a pre-defined range, which can specifically be a circular area with the reference frame calibration point as the center and a set pixel distance as the radius, such as... Figure 6 As shown. In practical applications, the set range can be adjusted according to the number of reference frame feature points within the set range. For example, if the number of reference frame feature points within the set range is 0-1, the set range can be appropriately expanded. The set pixel distance can be set to 3-5 pixels. Taking a 3-pixel distance as an example, assuming the reference frame calibration point is located in the 5th row and 5th column, then the reference frame feature point in the 1st row and 5th column does not belong to the feature points within its surrounding set range, while the reference frame feature point in the 3rd row and 5th column does belong to the feature points within its surrounding set range.

[0038] The region tracking device acquires the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, acquires the position of multiple reference frame feature points within a set range around the reference frame calibration point, acquires multiple detectable frame feature points and their positions in the detectable frame feature map that match the above multiple reference frame feature points based on the feature matching map obtained in step S103, and determines the position of the detectable frame calibration point corresponding to the reference frame calibration point in the detectable frame feature map based on the acquired positions.

[0039] S105, mark the target region of the frame to be detected in the image of the frame to be detected according to the position of the calibration point of the frame to be detected.

[0040] Specifically, the region tracking device re-marks the target region in the image of the frame to be detected based on the position of the calibration point of the frame to be detected determined in step S104, i.e., re-marks the target region of the frame to be detected. Figure 7 As shown.

[0041] In summary, the region tracking method of this application embodiment extracts feature maps from the reference frame image and the frame to be detected image captured sequentially by the imaging device to obtain corresponding reference frame feature maps and frame to be detected feature maps. Feature point matching is performed on the reference frame feature points in the reference frame feature map and the frame to be detected feature points in the frame to be detected feature map. Based on the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple frame to be detected feature points in the frame to be detected feature map that match multiple reference frame feature points, the position of the frame to be detected calibration point corresponding to the reference frame calibration point in the frame to be detected feature map is determined. The target region of the frame to be detected is marked in the frame to be detected image based on the position of the frame to be detected calibration point. This application embodiment extracts and matches feature points from a reference frame image and a target frame image. Based on the position of the reference frame calibration point in the reference frame image and the positions of multiple matching feature points around the calibration point of the target region in both frames, the position of the target frame calibration point in the target frame image is determined, thereby determining the position of the target region in the target frame image. Even if pedestrians or goods move within the target region, tracking of the target region can be achieved when the capturing device, such as a camera, moves. Predicting the position of the target frame calibration point based on the positions of multiple reference frame feature points within a set range around the reference frame calibration point, using a multi-point comprehensive prediction method, avoids the risk of inaccurate predictions due to factors such as occlusion.

[0042] Figure 8 This is a flowchart illustrating a region tracking method provided in another embodiment of this application. Figure 8 As shown above, in the above Figure 1 Based on the illustrated embodiment, the region tracking method of this application embodiment may specifically include the following steps:

[0043] S801, acquire the reference frame image and the frame image to be detected captured sequentially by the imaging device.

[0044] Specifically, step S801 in this embodiment is the same as step S101 in the above embodiment, and will not be repeated here.

[0045] S802 performs image preprocessing on the reference frame image and the frame image to be detected, respectively.

[0046] Specifically, before extracting feature maps from the reference frame image and the frame image to be detected, image preprocessing can be performed on both images. Then, feature map extraction can be performed on the preprocessed images to improve the accuracy of feature map extraction. Image preprocessing may include, but is not limited to, grayscale conversion and normalization.

[0047] The step S102 in the above embodiment, "extracting feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map corresponding to the reference frame image and the frame image to be detected corresponding to the frame image to be detected", may specifically include the following step S803.

[0048] S803 uses a deep neural network to extract feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the feature map of the reference frame and the feature map of the frame to be detected.

[0049] Specifically, feature map extraction can be performed using non-deep learning methods, such as optical flow-based approaches, or using deep learning methods, such as deep neural networks. Using deep learning for feature map extraction can improve the quality of the extracted features and significantly save computational resources.

[0050] Step S103 in the above embodiment, "matching the feature points of the reference frame in the reference frame feature map and the feature points of the frame to be detected in the feature map of the frame to be detected", may specifically include the following steps S804-S805.

[0051] S804 calculates the Euclidean distance between feature points in the reference frame and feature points in the frame to be detected.

[0052] Specifically, each feature point corresponds to a descriptor, usually represented by a vector. The similarity between two vectors, i.e., the similarity between two feature points, can be obtained by calculating the Euclidean distance between them.

[0053] Euclidean distance, also known as Euclidean distance, is the "ordinary" (i.e., straight-line) distance between two points represented by vectors in Euclidean space.

[0054] S805 performs feature point matching between the reference frame feature points and the target frame feature points based on Euclidean distance.

[0055] Specifically, if the Euclidean distance between the reference frame feature point and the detection frame feature point calculated in step S804 is less than a preset Euclidean distance threshold, then the reference frame feature point and the detection frame feature point are determined to match. If the Euclidean distance between the reference frame feature point and the detection frame feature point calculated in step S804 is equal to or greater than the preset Euclidean distance threshold, then the reference frame feature point and the detection frame feature point are determined to not match.

[0056] Step S104 in the above embodiment, "determine the position of the target frame calibration point in the target frame map corresponding to the reference frame calibration point based on the position of the reference frame calibration point in the target frame map, the position of multiple reference frame feature points within a set range around the reference frame calibration point, and the position of multiple target frame feature points in the target frame map that match the multiple reference frame feature points", may specifically include the following steps S806-S809.

[0057] S806 calculates the position of the center point of multiple reference frame feature points based on the positions of multiple reference frame feature points.

[0058] S807 calculates the position vector from the reference frame calibration point to the center point of the reference frame feature point based on the position of the reference frame calibration point and the position of the center point of the reference frame feature point.

[0059] Specifically, assuming the position of the reference frame calibration point A is (x1, y1) and the position of the center point B of the reference frame feature point is (x2, y2), then the position vector AB from the reference frame calibration point A to the center point B of the reference frame feature point is (x2-x1, y2-y1).

[0060] S808 calculates the position of the center point of multiple feature points in multiple frames to be detected based on their positions.

[0061] S809, determine the position of the calibration point of the frame to be detected based on the position vector and the position of the center point of the feature points of the frame to be detected.

[0062] Specifically, assuming the position of the center point B' of the feature points of the frame to be detected is (x2', y2'), then the position of the calibration point A' of the frame to be detected is determined as (x2', y2')-(x2-x1, y2-y1)=(x2'-x2+x1, y2'-y2+y1).

[0063] S810 marks the target region of the frame to be detected in the image of the frame to be detected based on the position of the calibration point of the frame to be detected.

[0064] Specifically, step S810 in this embodiment is the same as step S105 in the above embodiment, and will not be repeated here.

[0065] In summary, the region tracking method of this application embodiment extracts feature maps from the reference frame image and the frame to be detected image captured sequentially by the imaging device to obtain corresponding reference frame feature maps and frame to be detected feature maps. Feature point matching is performed on the reference frame feature points in the reference frame feature map and the frame to be detected feature points in the frame to be detected feature map. Based on the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple frame to be detected feature points in the frame to be detected feature map that match multiple reference frame feature points, the position of the frame to be detected calibration point corresponding to the reference frame calibration point in the frame to be detected feature map is determined. The target region of the frame to be detected is marked in the frame to be detected image based on the position of the frame to be detected calibration point. This application embodiment extracts and matches feature points from a reference frame image and a target frame image. Based on the position of the reference frame calibration point in the reference frame image and the positions of multiple matching feature points surrounding the calibration point of the target region in both frames, the position of the target frame calibration point in the target frame image is determined, thereby determining the position of the target region in the target frame image. Even if pedestrians or goods move within the target region, tracking of the target region is still possible when the capturing device (e.g., a camera) moves. The position of the target frame calibration point is predicted based on the positions of multiple reference frame feature points within a set range around the reference frame calibration point. This multi-point comprehensive prediction method avoids the risk of inaccurate predictions due to factors such as occlusion. By calculating the position vector from the reference frame calibration point to the center point of multiple reference frame feature points, the relative positional relationship between the reference frame calibration point and surrounding reference frame feature points can be quickly and accurately determined. Based on this relative positional relationship and the positions of the corresponding center points of multiple target frame feature points, the position of the target frame calibration point is quickly and accurately determined. Using Euclidean distance for feature point matching improves matching accuracy. Image preprocessing and the use of neural networks for feature map extraction both further improve the accuracy of feature map extraction.

[0066] To achieve the above embodiments, this application also proposes a region tracking device. Figure 9 This is a schematic diagram of the structure of an area tracking device according to an embodiment of this application. Figure 9 As shown, the area tracking device 900 of this application embodiment may specifically include: an acquisition module 901, an extraction module 902, a matching module 903, a determination module 904, and a calibration module 905. Wherein:

[0067] The acquisition module 901 is configured to acquire reference frame images and frame images to be detected captured sequentially by the imaging device.

[0068] The extraction module 902 is configured to extract feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map corresponding to the reference frame image and the frame image to be detected feature map corresponding to the frame image to be detected.

[0069] The matching module 903 is configured to perform feature point matching between the reference frame feature points in the reference frame feature map and the detection frame feature points in the detection frame feature map.

[0070] The determination module 904 is configured to determine the position of the target frame calibration point in the target frame map corresponding to the reference frame calibration point based on the position of the reference frame calibration point in the target area of ​​the reference frame calibrated in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple target frame feature points in the target frame feature map that match the multiple reference frame feature points.

[0071] The calibration module 905 is configured to calibrate the target region of the frame to be detected in the image of the frame to be detected based on the position of the calibration point of the frame to be detected.

[0072] In one embodiment of this application, the determining module 904 is further configured to: calculate the position of the center point of the multiple reference frame feature points based on the positions of the multiple reference frame feature points; calculate the position vector from the reference frame calibration point to the center point of the reference frame feature points based on the position of the reference frame calibration point and the position of the center point of the reference frame feature points; calculate the position of the center point of the multiple frame feature points to be detected based on the positions of the multiple frame feature points to be detected; and determine the position of the frame calibration point to be detected based on the position vector and the position of the center point of the frame feature points to be detected.

[0073] In one embodiment of this application, the set range is a circular area with the reference frame calibration point as the center and the set pixel distance as the radius.

[0074] In one embodiment of this application, the pixel distance is set to 3-5 pixels.

[0075] In one embodiment of this application, the matching module 903 is further configured to: calculate the Euclidean distance between the feature points of the reference frame and the feature points of the frame to be detected; and perform feature point matching between the feature points of the reference frame and the feature points of the frame to be detected based on the Euclidean distance.

[0076] In one embodiment of this application, the matching module 903 is further configured to: if the Euclidean distance is less than a preset Euclidean distance threshold, then determine that the feature points of the reference frame match the feature points of the frame to be detected.

[0077] In one embodiment of this application, the extraction module 902 is further configured to perform image preprocessing on the reference frame image and the frame image to be detected before extracting feature maps from the reference frame image and the frame image to be detected, respectively.

[0078] In one embodiment of this application, the extraction module 902 is further configured to: use a deep neural network to extract feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map and the frame image to be detected feature map.

[0079] It should be noted that the above explanation of the embodiments of the area tracking method also applies to the area tracking device of the embodiments of this application, and the specific process will not be repeated here.

[0080] In summary, the region tracking device of this application extracts feature maps from the reference frame image and the frame to be detected image captured sequentially by the shooting device to obtain corresponding reference frame feature maps and frame to be detected feature maps. It performs feature point matching on the reference frame feature points in the reference frame feature map and the frame to be detected feature points in the frame to be detected feature map. Based on the position of the reference frame calibration point of the target region of the reference frame in the reference frame feature map, the position of multiple reference frame feature points within a set range around the reference frame calibration point, and the position of multiple frame to be detected feature points in the frame to be detected feature map that match the multiple reference frame feature points, it determines the position of the frame to be detected calibration point corresponding to the reference frame calibration point in the frame to be detected feature map. Based on the position of the frame to be detected calibration point, it marks the target region of the frame to be detected in the frame to be detected image. This application embodiment extracts and matches feature points from a reference frame image and a target frame image. Based on the position of the reference frame calibration point in the reference frame image and the positions of multiple matching feature points surrounding the calibration point of the target region in both frames, the position of the target frame calibration point in the target frame image is determined, thereby determining the position of the target region in the target frame image. Even if pedestrians or goods move within the target region, tracking of the target region is still possible when the capturing device (e.g., a camera) moves. The position of the target frame calibration point is predicted based on the positions of multiple reference frame feature points within a set range around the reference frame calibration point. This multi-point comprehensive prediction method avoids the risk of inaccurate predictions due to factors such as occlusion. By calculating the position vector from the reference frame calibration point to the center point of multiple reference frame feature points, the relative positional relationship between the reference frame calibration point and surrounding reference frame feature points can be quickly and accurately determined. Based on this relative positional relationship and the positions of the corresponding center points of multiple target frame feature points, the position of the target frame calibration point is quickly and accurately determined. Using Euclidean distance for feature point matching improves matching accuracy. Image preprocessing and the use of neural networks for feature map extraction both further improve the accuracy of feature map extraction.

[0081] like Figure 10The diagram shown is a block diagram of an electronic device for an area tracking method according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as intelligent voice interaction devices, personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0082] like Figure 10 As shown, the electronic device includes one or more processors 1001, a memory 1002, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processor 1001 can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 10 Take processor 1001 as an example.

[0083] The memory 1002 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause at least one processor to perform the region tracking method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the region tracking method provided in this application.

[0084] Memory 1002, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the region tracking method in the embodiments of this application (e.g., appendix). Figure 9 The acquisition module 901, extraction module 902, matching module 903, determination module 904, and calibration module 905 are shown. The processor 1001 executes various server functions and data processing by running non-transient software programs, instructions, and modules stored in the memory 1002, thereby implementing the region tracking method in the above method embodiments.

[0085] The memory 1002 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the electronic device according to the area tracking method. Furthermore, the memory 1002 may include high-speed random access memory and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1002 may optionally include memory remotely located relative to the processor 1001, and these remote memories can be connected to the electronic device of the area tracking method via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0086] The electronic device for the area tracking method may further include an input device 1003 and an output device 1004. The processor 1001, memory 1002, input device 1003, and output device 1004 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0087] Input device 1003 can receive input numerical or character information, as well as key signal input related to user settings and function control of the electronic device for the area tracking method, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1004 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0088] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0091] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0092] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability.

[0093] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0094] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0095] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A region tracking method, characterized in that, include: Acquire reference frame images and frame images to be detected captured sequentially by the imaging device; Feature maps are extracted from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map corresponding to the reference frame image and the frame image to be detected corresponding to the frame image to be detected; Feature point matching is performed between the reference frame feature points in the reference frame feature map and the detection frame feature points in the detection frame feature map. Based on the position of the reference frame calibration point in the target region of the reference frame marked in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple detectable frame feature points in the detectable frame feature map that match the multiple reference frame feature points, the position of the detectable frame calibration point corresponding to the reference frame calibration point in the detectable frame feature map is determined. The target region of the frame to be detected is marked in the image of the frame to be detected based on the position of the calibration point of the frame to be detected. The step of determining the position of the target frame calibration point in the target frame feature map corresponding to the reference frame calibration point based on the position of the reference frame calibration point in the target region of the reference frame in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple target frame feature points in the target frame feature map that match the multiple reference frame feature points, includes: Calculate the position of the center point of the multiple reference frame feature points based on their positions; Based on the position of the reference frame calibration point and the position of the center point of the reference frame feature point, calculate the position vector from the reference frame calibration point to the center point of the reference frame feature point; Calculate the position of the center point of the multiple feature points of the frame to be detected based on the positions of the multiple feature points of the frame to be detected; The position of the calibration point of the frame to be detected is determined based on the position vector and the position of the center point of the feature point of the frame to be detected.

2. The method according to claim 1, characterized in that, The set range is a circular area centered on the reference frame calibration point and with a set pixel distance as the radius.

3. The method according to claim 2, characterized in that, The set pixel distance is 3-5 pixels.

4. The method according to claim 1, characterized in that, The feature point matching between the reference frame feature points in the reference frame feature map and the target frame feature points in the target frame feature map includes: Calculate the Euclidean distance between the feature points of the reference frame and the feature points of the frame to be detected; Feature point matching is performed between the feature points of the reference frame and the feature points of the frame to be detected based on the Euclidean distance.

5. The method according to claim 4, characterized in that, The step of matching feature points of the reference frame and the target frame based on the Euclidean distance includes: If the Euclidean distance is less than a preset Euclidean distance threshold, then the feature point of the reference frame is determined to match the feature point of the frame to be detected.

6. The method according to claim 1, characterized in that, Before extracting feature maps from the reference frame image and the frame image to be detected, the method further includes: Image preprocessing is performed on the reference frame image and the frame image to be detected, respectively.

7. The method according to claim 1, characterized in that, The step of extracting feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map corresponding to the reference frame image and the frame image to be detected corresponding to the frame image to be detected, includes: A deep neural network is used to extract feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the feature map of the reference frame and the feature map of the frame to be detected.

8. A region tracking device, characterized in that, include: The acquisition module is configured to acquire reference frame images and frame images to be detected captured sequentially by the imaging device; The extraction module is configured to extract feature maps from the reference frame image and the frame image to be detected, respectively, to obtain the reference frame feature map corresponding to the reference frame image and the frame image to be detected feature map corresponding to the frame image to be detected; The matching module is configured to perform feature point matching between the reference frame feature points in the reference frame feature map and the detection frame feature points in the detection frame feature map. The determination module is configured to determine the position of the target frame calibration point in the target frame map corresponding to the reference frame calibration point based on the position of the reference frame calibration point in the target region of the reference frame in the reference frame feature map, the positions of multiple reference frame feature points within a set range around the reference frame calibration point, and the positions of multiple target frame feature points in the target frame feature map that match the multiple reference frame feature points. The calibration module is configured to calibrate the target region of the frame to be detected in the image of the frame to be detected based on the position of the calibration point of the frame to be detected; The module is defined as follows: Calculate the position of the center point of the multiple reference frame feature points based on their positions; Based on the position of the reference frame calibration point and the position of the center point of the reference frame feature point, calculate the position vector from the reference frame calibration point to the center point of the reference frame feature point; Calculate the position of the center point of the multiple feature points of the frame to be detected based on the positions of the multiple feature points of the frame to be detected; The position of the calibration point of the frame to be detected is determined based on the position vector and the position of the center point of the feature point of the frame to be detected.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method as described in any one of claims 1-7.

Images