Target tracking method and device based on monitoring system, processor and electronic device

Abstract

The application discloses a target tracking method and device based on a monitoring system, a processor and electronic equipment, and relates to the field of financial technology. The method comprises the following steps: acquiring an appearance template library of a target object; acquiring a target image frame to be tracked from the monitoring system, wherein the target image frame at least comprises image information of the target object; tracking a target position coordinate of the target object in the target image frame according to the appearance image and the target image frame through a twin neural network and a first preset algorithm; taking the target position coordinate as the center, performing frame selection on the target object in the target image frame through a second preset algorithm, obtaining a target appearance image of the target object in the target image frame, and storing the target appearance image into the appearance template library. Through the application, the problem that the accuracy of tracking is relatively low due to the use of a fixed appearance model to track the target object in the related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology, in particular, to a target tracking method and device based on a monitoring system, a processor and an electronic device. BACKGROUND

[0002] In recent years, with the improvement of computer hardware and software performance and the rapid development of the field of computer vision, intelligent monitoring has become a new direction of development in the field of monitoring. In order to improve the level of security and protection, it is essential to build an intelligent monitoring system for banks. Target tracking is the core technology of an intelligent monitoring system, so it is of practical significance to study target tracking methods.

[0003] According to the difference of algorithm model, the existing classical target tracking methods can be divided into two categories: generative and discriminative. There are also many studies that combine the two. The generative method uses a generative model to model the appearance of the target, and searches for the region that best matches the appearance model. In the construction process, the generative model focuses on how to mine the characteristics of the tracked target, and almost does not consider the background information. The discriminative method is more to regard tracking as a special classification problem, and collects samples by taking the background as negative samples and the image blocks containing the target as positive samples, so as to distinguish the target and the background to the greatest extent. The above methods are all based on fixed appearance model for modeling and tracking. However, in bank monitoring, there are many customers, and interference such as occlusion is easy to occur. At the same time, customers do not maintain the same posture when walking and conducting business. Therefore, using a fixed appearance model to track the target object in the related technology has the problem of low tracking accuracy.

[0004] In view of the problem that using a fixed appearance model to track the target object in the related technology leads to low tracking accuracy, no effective solution has been proposed so far. SUMMARY

[0005] The main purpose of the present application is to provide a target tracking method and device based on a monitoring system, a processor and an electronic device, so as to solve the problem that using a fixed appearance model to track the target object in the related technology leads to low tracking accuracy.

[0006] In order to achieve the above object, according to one aspect of the present application, a target tracking method based on a monitoring system is provided. The method comprises: obtaining an appearance template library of a target object, wherein the appearance template library comprises at least one appearance image of the target object; obtaining a target image frame to be tracked from the monitoring system, wherein the target image frame comprises at least image information of the target object; tracking a target position coordinate of the target object in the target image frame by a twin neural network and a first preset algorithm according to the appearance image and the target image frame; and performing frame selection on the target object in the target image frame by a second preset algorithm with the target position coordinate as the center to obtain a target appearance image of the target object in the target image frame, and storing the target appearance image into the appearance template library.

[0007] Further, obtaining the appearance template library of the target object comprises: detecting an appearance image of the target object by the second preset algorithm after detecting that the target object enters a monitoring range; and storing the appearance image into the appearance template library.

[0008] Further, tracking the target position coordinate of the target object in the target image frame by the twin neural network and the first preset algorithm according to the appearance image and the target image frame comprises: predicting an initial position coordinate of the target object in the target image frame by the twin neural network according to the appearance image and the target image frame; and correcting the initial position coordinate by the first preset algorithm to obtain the target position coordinate.

[0009] Further, predicting the initial position coordinate of the target object in the target image frame by the twin neural network according to the appearance image and the target image frame comprises: obtaining a target sample pair according to the appearance image and the target image frame; extracting features of the sample pair by the twin neural network to obtain an appearance feature map and a detection feature map; performing cross-correlation calculation on the appearance feature map and the detection feature map to obtain a response map, wherein the response map comprises a plurality of pixel points, and each pixel point corresponds to a similarity value; and taking a coordinate of a pixel point corresponding to a maximum similarity value in the response map as the initial position coordinate.

[0010] Further, obtaining the target sample pair according to the appearance image and the target image frame comprises: processing the appearance image to obtain an appearance image block of a first preset size; processing the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is greater than the first preset size; and assembling the appearance image block and the detection image block into the target sample pair.

[0011] Further, obtaining the target position coordinates by correcting the initial position coordinates using the first preset algorithm includes: obtaining the initial state variables of the filter of the first preset algorithm based on the size of the appearance image and the initial position coordinates; processing the initial state variables based on the process transition matrix and the prediction covariance matrix to obtain the target state variables; and correcting the initial position coordinates based on the target state variables to obtain the target position coordinates.

[0012] Further, processing the initial state variables based on the process transition matrix and the prediction covariance matrix to obtain the target state variables includes: setting the process transition matrix and updating the initial state variables based on the process transition matrix to obtain updated initial state variables; obtaining the prediction covariance matrix and obtaining the target state variables based on the prediction covariance matrix and the updated initial state variables.

[0013] To achieve the above objectives, according to another aspect of this application, a target tracking device based on a monitoring system is provided. The device includes: a first acquisition unit for acquiring a target object appearance template library, wherein the appearance template library includes at least one appearance image of the target object; a second acquisition unit for acquiring a target image frame to be tracked from the monitoring system, wherein the target image frame includes at least image information of the target object; a tracking unit for tracking the target object's target position coordinates in the target image frame using a Siamese neural network and a first preset algorithm based on the appearance image and the target image frame; and a selection unit for selecting the target object in the target image frame using a second preset algorithm, centered on the target position coordinates, to obtain a target appearance image of the target object in the target image frame, and storing the target appearance image in the appearance template library.

[0014] Furthermore, the first acquisition unit includes: a detection subunit, used to detect the monitoring screen through the second preset algorithm after detecting that the target object has entered the monitoring range, and obtain the appearance image of the target object; and a storage subunit, used to store the appearance image into the appearance template library.

[0015] Furthermore, the tracking unit includes: a prediction subunit, used to predict the initial position coordinates of the target object in the target image frame using the Siamese neural network based on the appearance image and the target image frame; and a correction subunit, used to correct the initial position coordinates using the first preset algorithm to obtain the target position coordinates.

[0016] Further, the prediction subunit comprises: a first obtaining module, configured to obtain a target sample pair according to the appearance image and the target image frame; an extraction module, configured to perform feature extraction on the sample pair by using the twin neural network to obtain an appearance feature map and a detection feature map; a calculation module, configured to perform cross-correlation calculation on the appearance feature map and the detection feature map to obtain a response map, wherein the response map comprises a plurality of pixel points, and each pixel point corresponds to a similarity value; and a determination module, configured to determine the coordinates of a pixel point corresponding to the maximum similarity value in the response map as the initial position coordinates.

[0017] Further, the first obtaining module comprises: a first processing submodule, configured to process the appearance image to obtain an appearance image block of a first preset size; a second processing submodule, configured to process the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is greater than the first preset size; and an assembly submodule, configured to assemble the appearance image block and the detection image block into the target sample pair.

[0018] Further, the correction subunit comprises: a second obtaining module, configured to obtain an initial state variable of a filter of the first preset algorithm according to the size of the appearance image and the initial position coordinates; a processing module, configured to process the initial state variable according to a process transition matrix and a prediction covariance matrix to obtain a target state variable; and a correction module, configured to correct the initial position coordinates according to the target state variable to obtain the target position coordinates.

[0019] Further, the processing module comprises: a setting submodule, configured to set the process transition matrix, and update the initial state variable according to the process transition matrix to obtain an updated initial state variable; and an obtaining submodule, configured to obtain the prediction covariance matrix, and obtain a target state variable according to the prediction covariance matrix and the updated initial state variable.

[0020] In order to achieve the above purpose, according to an aspect of the present application, a processor is provided, which is used to run a program, wherein the program performs the target tracking method based on the monitoring system as described in any one of the above aspects when running.

[0021] In order to achieve the above purpose, according to an aspect of the present application, an electronic device is provided, which comprises one or more processors and a memory, and the memory is used to store the one or more processors to implement the target tracking method based on the monitoring system as described in any one of the above aspects.

[0022] According to the application, the following steps are adopted: obtaining an appearance template library of a target object, wherein the appearance template library comprises at least one appearance image of the target object; obtaining a target image frame to be tracked from a monitoring system, wherein the target image frame at least comprises image information of the target object; tracking a target position coordinate of the target object in the target image frame by means of a twin neural network and a first preset algorithm according to the appearance image and the target image frame; and performing frame selection on the target object in the target image frame by means of a second preset algorithm with the target position coordinate as the center, obtaining a target appearance image of the target object in the target image frame, and storing the target appearance image in the appearance template library, thereby solving the problem that in the related art, a fixed appearance model is used to track the target object, resulting in relatively low tracking accuracy. The target object in the target image frame is tracked and positioned by means of the appearance template library of the target object, and the appearance template library is updated by means of the target appearance image of the target object in the target image frame, thereby avoiding tracking the target object by means of a fixed appearance template, and thus the effect of improving tracking accuracy is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their

[0024] Figure 1 is a flowchart of a target tracking method based on a monitoring system according to an embodiment of the application;

[0025] Figure 2 is a flowchart of obtaining a target position coordinate by means of a twin neural network and a Kalman filtering algorithm according to an embodiment of the application;

[0026] Figure 3 is a schematic diagram of a twin neural network according to an embodiment of the application;

[0027] Figure 4 is a schematic diagram of a target tracking device based on a monitoring system according to an embodiment of the application;

[0028] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0029] It should be noted that the embodiments and features in the application can be combined with each other without conflict. The application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, and obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should belong to the protection scope of the present application.

[0031] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those clearly listed steps or units, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties.

[0033] The present application will be described below in conjunction with preferred implementation steps, Figure 1 The flowchart of the target tracking method based on the monitoring system provided according to the embodiments of the present application is shown in Figure 1 The method comprises the following steps:

[0034] Step S101, obtaining an appearance template library of a target object, wherein the appearance template library comprises at least one appearance image of the target object.

[0035] Specifically, the appearance image of the target object is obtained through a monitoring video, and the appearance image is saved in the appearance template library. It should be noted that the appearance template comprises multiple appearance images of the target object. The target object generally refers to a customer who needs to handle business in a bank.

[0036] Step S102, obtaining a target image frame to be tracked from a monitoring system, wherein the target image frame comprises at least image information of the target object.

[0037] Specifically, a target image frame to be tracked is acquired from the monitoring system, the target image frame can include image information of the target object and image information of other objects, and the target object in the target image frame needs to be framed out.

[0038] In step S103, the target position coordinates of the target object in the target image frame are tracked by the twin neural network and the first preset algorithm according to the appearance image and the target image frame.

[0039] Specifically, the target position coordinates of the target object in the target image frame to be tracked are determined by using the twin neural network and the Kalman filtering algorithm (i.e., the first preset algorithm) according to the appearance image of the target object and the target image frame, so as to realize tracking of the target object.

[0040] In step S104, the target object in the target image frame is framed out by the second preset algorithm with the target position coordinates as the center, so as to obtain the target appearance image of the target object in the target image frame, and the target appearance image is stored in the appearance template library.

[0041] Specifically, the target object in the target image frame is framed out by using the YOLOv5 algorithm (i.e., the second preset algorithm) with the target position coordinates as the center point, so as to obtain the target appearance image of the target object, and the target appearance image is stored in the appearance template library of the target object. It should be noted that, in order to avoid occupying too much memory, the appearance image can be stored once every 20 frames, and the maximum storage capacity of the appearance template library is set. When the maximum storage capacity is exceeded, the earliest stored appearance image is deleted, and the appearance template library of the target object is deleted after the target object leaves the monitoring range.

[0042] In summary, the target object in the target image frame is tracked and positioned by the appearance template library of the target object, and the appearance template library is updated by the target appearance image of the target object in the target image frame, so as to avoid tracking the target object by using the fixed appearance template, and the tracking accuracy is improved.

[0043] Optionally, in the target tracking method based on the monitoring system provided in the embodiments of the present application, acquiring the appearance template library of the target object includes: detecting the target object entering the monitoring range, detecting the monitoring screen by the second preset algorithm to obtain the appearance image of the target object; and storing the appearance image in the appearance template library.

[0044] Specifically, when it is detected that the target object enters the monitoring unit of the monitoring system, the target object in the monitoring picture is monitored by using the YOLOv5 algorithm, an appearance image of the target object is obtained, and the appearance image is stored in the appearance template library. It should be noted that the monitoring picture may include images of multiple customers who need to handle business, and then the YOLOv5 algorithm will detect each customer's image and establish an appearance template library for each customer. Through the above steps, the appearance template library of the target object can be accurately obtained, which is helpful for subsequent accurate tracking of the target object.

[0045] Optionally, in the target tracking method based on the monitoring system provided in the embodiments of the present application, the target position coordinates of the target object in the target image frame are tracked by the Siamese neural network and the first preset algorithm according to the appearance image and the target image frame, including: predicting the initial position coordinates of the target object in the target image frame by the Siamese neural network according to the appearance image and the target image frame; and correcting the initial position coordinates by the first preset algorithm to obtain the target position coordinates.

[0046] Specifically, as shown in Figure 2 The Siamese neural network predicts the position information of the target object in the target image frame by the appearance image and the target image frame to obtain the initial position coordinates. Then, the initial position coordinates are corrected by using the Kalman filtering algorithm to obtain the final target position coordinates. The Siamese neural network (Siamese neural network) is a coupled architecture based on two artificial neural networks. The Siamese neural network takes two samples as input and outputs the representation of the two samples embedded in a high-dimensional space to compare the similarity of the two samples. The Siamese neural network is composed of two full convolutional networks with the same structure and shared weights, and the two full convolutional networks are called template branches and detection branches. The full convolutional network is composed of five convolutional layers and two pooling layers, and its structure is as shown in Figure 3 The Kalman filtering (Kalman filtering) is an algorithm that uses the state equation of a linear system to optimally estimate the state of the system through system input and output observation data. Since the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process. The Siamese neural network can accurately compare the similarity of the appearance image and the target image frame, and then determine the position information of the target object. Then, the position is corrected by using the Kalman filtering algorithm, which further ensures the accuracy of tracking the target object.

[0047] Optionally, in the target tracking method based on a monitoring system provided in the embodiments of the present application, the initial position coordinates of the target object in the target image frame are predicted by a twin neural network according to the appearance image and the target image frame, comprising: obtaining a target sample pair according to the appearance image and the target image frame; extracting features of the target sample pair by the twin neural network to obtain an appearance feature map and a detection feature map; performing cross-correlation calculation on the appearance feature map and the detection feature map to obtain a response map, wherein the response map includes a plurality of pixel points, and each pixel point corresponds to a similarity value; and the coordinates of the pixel point corresponding to the maximum similarity value in the response map are taken as the initial position coordinates.

[0048] Specifically, first, a target sample pair is obtained according to the appearance image and the target image frame. For example, there are n appearance images in the appearance template library, and then n sample pairs are obtained according to the n appearance images. The target sample pair is extracted to obtain an appearance feature map (corresponding to the appearance image) and a detection feature map (corresponding to the target image frame). The full convolution network is represented as , the extracted appearance feature map is , and the detection feature map is The cross-correlation operation is used to calculate the similarity between the feature maps, as shown in the following formula: , wherein represents the cross-correlation operation, r is the operation result, and it is a two-dimensional response map. The size of the response map is 17x17, and the value in the response map represents the similarity, which can also be called the probability of belonging to the target. Then, n response maps can be obtained for n sample pairs. The bicubic interpolation is used to convert the 17x17 response map to 272x272, and finally the position of the pixel point corresponding to the maximum similarity value in the response map is selected as the predicted position (that is, the initial position coordinates). The similarity between the appearance image and the target image frame can be accurately compared by the twin neural network, and the position information of the target object can be accurately predicted.

[0049] Optionally, in the target tracking method based on a monitoring system provided in the embodiments of the present application, obtaining a target sample pair according to the appearance image and the target image frame comprises: processing the appearance image to obtain an appearance image block of a first preset size; processing the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is greater than the first preset size; and assembling the appearance image block and the detection image block into a target sample pair.

[0050] Specifically, an MxM image block is cropped with the center point of the appearance image as the center, and the size of M is calculated by the following formula: (w+p)x(h+p)=M 2where w, h are the width and height of the appearance image, p = (w + h) / 2, and then the image block is scaled to an appearance image block with a size of 127 x 127 (i.e., the first preset size described above). Then the target image frame is cropped to obtain an image block with a size of 2M x 2M, and then the image block is scaled to a detection image block with a size of 255 x 255 (i.e., the second preset size described above). In the cropping process, if the target image frame boundary is exceeded, the average color of the target image frame is used as padding. By cropping and scaling the appearance image and the target image frame, the image is ensured not to be distorted, and the computational burden of the twin neural network is not increased.

[0051] Optionally, in the target tracking method based on the monitoring system provided in the embodiments of the present application, the initial position coordinates are corrected by the first preset algorithm to obtain the target position coordinates, including: obtaining an initial state variable of a filter of the first preset algorithm according to the size of the appearance image and the initial position coordinates; processing the initial state variable according to a process transition matrix and a prediction covariance matrix to obtain a target state variable; and correcting the initial position coordinates according to the target state variable to obtain the target position coordinates.

[0052] Specifically, the initial position coordinates are corrected by the Kalman filtering algorithm, and the process is as follows:

[0053] (1) An initial state variable S of a Kalman filter is obtained, as shown below:

[0054] S = [x, y, a, r, v x , v y , v a ] T

[0055] where x, y are the initial position coordinates, a is the area of the appearance image, r is the aspect ratio of the appearance image, v x , v y , and v a are the change rates of the initial position coordinates and the area, respectively.

[0056] (2) The initial state vector S is updated by a process transition matrix to obtain an updated initial state vector:

[0057] S t|t-1 = F t S t-1|t-1

[0058] where S t|t-1 is the updated initial state vector, F t is the process transition matrix, and the process transition matrix F is set as:

[0059]

[0060] (3) Obtain the prediction covariance matrix P t / t-1 :

[0061] P t|t-1 = F t P t-1|t-1 F t T + Q k

[0062] Wherein, Q k is the covariance of process excitation noise, which represents uncertain disturbance, the error between state transition matrix and actual process.

[0063] (4) The updated initial state variable S t|t-1 is processed by the prediction covariance matrix to obtain the target state variable S t|t :

[0064]

[0065] P t|t = (I-K t H) P t|t-1 ,

[0066] S t|t = S t|t-1 + K t (Z t -HS t|t-1 )

[0067] Wherein, Kt is the Kalman filter gain, Rt is the observation noise covariance at time t, that is, the noise inside the tracker, H is the matrix for converting state space to observation space, which is set as:

[0068]

[0069] (5) The initial position coordinates predicted by the twin network are converted into the same format as the target state variable, and the converted result is called observation result, which is written as Z = [x, y, a, r]. Finally, the observation result is corrected according to S t / t to obtain the target position coordinates.

[0070] Through the above way, the influence of noise and interference can be effectively eliminated, and the accuracy of target object tracking and positioning is improved.

[0071] Optionally, in the target tracking method based on the monitoring system provided in the embodiments of the present application, the initial state variable is processed according to the process transition matrix and the prediction covariance matrix to obtain the target state variable, including: setting the process transition matrix, and updating the initial state variable according to the process transition matrix to obtain the updated initial state variable; obtaining the prediction covariance matrix, and obtaining the target state variable according to the prediction covariance matrix and the updated initial state variable.

[0072] Specifically, the process transition matrix is set, the initial state variable is updated by using the process transition matrix, then the prediction covariance matrix is obtained, and the final target state vector is obtained by using the prediction covariance matrix.

[0073] The target tracking method based on the monitoring system provided in the embodiments of the present application obtains the appearance template library of the target object, wherein the appearance template library includes at least one appearance image of the target object; obtains the target image frame to be tracked from the monitoring system, wherein the target image frame includes at least image information of the target object; tracks the target position coordinates of the target object in the target image frame according to the appearance image and the target image frame by using the twin neural network and the first preset algorithm; and frames the target object in the target image frame by using the second preset algorithm with the target position coordinates as the center to obtain the target appearance image of the target object in the target image frame, and stores the target appearance image into the appearance template library, thereby solving the problem that the fixed appearance model is used to track the target object in the related art, resulting in relatively low tracking accuracy. The target object in the target image frame is tracked and positioned by using the appearance template library of the target object, and the appearance template library is updated by using the target appearance image of the target object in the target image frame, thereby avoiding tracking the target object by using the fixed appearance template, and thus the effect of improving the tracking accuracy is achieved.

[0074] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0075] The embodiments of the present application also provide a target tracking device based on a monitoring system. It should be noted that the target tracking device based on the monitoring system of the embodiments of the present application can be used to execute the target tracking method based on the monitoring system provided by the embodiments of the present application. The target tracking device based on the monitoring system provided by the embodiments of the present application is introduced as follows.

[0076] Figure 4 is a schematic diagram of the target tracking device based on the monitoring system according to the embodiments of the present application. As shown in Figure 4As shown, the device comprises a first acquisition unit 401, a second acquisition unit 402, a tracking unit 403 and a selection unit 404.

[0077] The first acquisition unit 401 is configured to acquire an appearance template library of a target object, wherein the appearance template library comprises at least one appearance image of the target object.

[0078] The second acquisition unit 402 is configured to acquire a target image frame to be tracked from a monitoring system, wherein the target image frame comprises at least image information of the target object.

[0079] The tracking unit 403 is configured to track a target position coordinate of the target object in the target image frame according to the appearance image and the target image frame by using a twin neural network and a first preset algorithm.

[0080] The selection unit 404 is configured to frame the target object in the target image frame by using the target position coordinate as a center and by using a second preset algorithm, to obtain a target appearance image of the target object in the target image frame, and to store the target appearance image into the appearance template library.

[0081] The device for tracking a target based on a monitoring system provided by the embodiments of the present application acquires an appearance template library of a target object by using the first acquisition unit 401, wherein the appearance template library comprises at least one appearance image of the target object; acquires a target image frame to be tracked from a monitoring system by using the second acquisition unit 402, wherein the target image frame comprises at least image information of the target object; tracks a target position coordinate of the target object in the target image frame according to the appearance image and the target image frame by using the tracking unit 403 and by using a twin neural network and a first preset algorithm; frames the target object in the target image frame by using the selection unit 404 and by using the target position coordinate as a center and by using a second preset algorithm, to obtain a target appearance image of the target object in the target image frame, and stores the target appearance image into the appearance template library, thereby solving the problem that the use of a fixed appearance model to track a target object in the related art leads to relatively low tracking accuracy. The appearance template library of the target object is used to track and locate the target object in the target image frame, and the target appearance image of the target object in the target image frame is used to update the appearance template library, thereby avoiding the tracking of the target object by using a fixed appearance template, and thus the accuracy of tracking is improved.

[0082] Optionally, in the device for tracking a target based on a monitoring system provided by the embodiments of the present application, the first acquisition unit 401 comprises a detection subunit configured to detect an appearance image of the target object by detecting a monitoring picture by using a second preset algorithm after detecting that the target object enters a monitoring range; and a storage subunit configured to store the appearance image into the appearance template library.

[0083] Optionally, in the target tracking device based on the monitoring system provided in the embodiments of this application, the tracking unit 403 includes: a prediction subunit, used to predict the initial position coordinates of the target object in the target image frame through a Siamese neural network based on the appearance image and the target image frame; and a correction subunit, used to correct the initial position coordinates through a first preset algorithm to obtain the target position coordinates.

[0084] Optionally, in the target tracking device based on a monitoring system provided in this application embodiment, the prediction subunit includes: a first acquisition module, used to obtain target sample pairs based on appearance images and target image frames; an extraction module, used to extract features from the target sample pairs through a Siamese neural network to obtain an appearance feature map and a detection feature map; a calculation module, used to perform cross-correlation calculation on the appearance feature map and the detection feature map to obtain a response map, wherein the response map includes multiple pixels and a similarity value corresponding to each pixel; and a determination module, used to take the coordinates of the pixel corresponding to the maximum similarity value in the response map as the initial position coordinates.

[0085] Optionally, in the target tracking device based on a monitoring system provided in this application embodiment, the first acquisition module includes: a first processing submodule, used to process the appearance image to obtain an appearance image block of a first preset size; a second processing submodule, used to process the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is larger than the first preset size; and a composition submodule, used to assemble the appearance image block and the detection image block into a target sample pair.

[0086] Optionally, in the target tracking device based on the monitoring system provided in the embodiments of this application, the correction subunit includes: a second acquisition module, used to obtain the initial state variables of the filter of the first preset algorithm based on the size of the appearance image and the initial position coordinates; a processing module, used to process the initial state variables based on the process transition matrix and the prediction covariance matrix to obtain the target state variables; and a correction module to correct the initial position coordinates based on the target state variables to obtain the target position coordinates.

[0087] Optionally, in the target tracking device based on the monitoring system provided in the embodiments of this application, the processing module includes: a setting submodule, used to set a process transition matrix and update the initial state variables according to the process transition matrix to obtain the updated initial state variables; and an acquisition submodule, used to acquire a prediction covariance matrix and obtain the target state variables according to the prediction covariance matrix and the updated initial state variables.

[0088] The target tracking device based on the monitoring system includes a processor and a memory. The first acquisition unit 401, the second acquisition unit 402, the tracking unit 403, and the selection unit 404 are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0089] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and the tracking of the target object can be achieved by adjusting the kernel parameters.

[0090] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0091] This invention provides a processor for running a program, wherein the program executes a target tracking method based on a monitoring system during runtime.

[0092] like Figure 5 As shown, this embodiment of the invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring a target object appearance template library, wherein the appearance template library includes at least one appearance image of the target object; acquiring a target image frame to be tracked from a monitoring system, wherein the target image frame includes at least image information of the target object; tracking the target position coordinates of the target object in the target image frame using a Siamese neural network and a first preset algorithm based on the appearance image and the target image frame; using the target position coordinates as the center, selecting the target object in the target image frame using a second preset algorithm to obtain the target appearance image of the target object in the target image frame, and storing the target appearance image in the appearance template library.

[0093] Optionally, obtaining the appearance template library of the target object includes: after detecting that the target object has entered the monitoring range, detecting the monitoring screen through a second preset algorithm to obtain the appearance image of the target object; and storing the appearance image in the appearance template library.

[0094] Optionally, tracking the target object's position coordinates in the target image frame using a Siamese neural network and a first preset algorithm based on the appearance image and the target image frame includes: predicting the initial position coordinates of the target object in the target image frame using a Siamese neural network based on the appearance image and the target image frame; and correcting the initial position coordinates using the first preset algorithm to obtain the target position coordinates.

[0095] Optionally, the initial position coordinates of the target object in the target image frame are predicted by the twin neural network according to the appearance image and the target image frame, and the method comprises: obtaining a target sample pair according to the appearance image and the target image frame; performing feature extraction on the target sample pair by the twin neural network to obtain an appearance feature map and a detection feature map; performing cross-correlation calculation on the appearance feature map and the detection feature map to obtain a response map, wherein the response map comprises a plurality of pixel points, and each pixel point corresponds to a similarity value; and taking the coordinates of the pixel point corresponding to the maximum similarity value in the response map as the initial position coordinates.

[0096] Optionally, the target sample pair is obtained according to the appearance image and the target image frame, and the method comprises: processing the appearance image to obtain an appearance image block of a first preset size; processing the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is greater than the first preset size; and assembling the appearance image block and the detection image block into the target sample pair.

[0097] Optionally, the target position coordinates are obtained by correcting the initial position coordinates by the first preset algorithm, and the method comprises: obtaining an initial state variable of a filter of the first preset algorithm according to the size of the appearance image and the initial position coordinates; processing the initial state variable according to a process transition matrix and a prediction covariance matrix to obtain target state variables; and correcting the initial position coordinates according to the target state variables to obtain the target position coordinates.

[0098] Optionally, the target state variables are obtained by processing the initial state variable according to the process transition matrix and the prediction covariance matrix, and the method comprises: setting the process transition matrix, and updating the initial state variable according to the process transition matrix to obtain an updated initial state variable; obtaining the prediction covariance matrix, and obtaining the target state variables according to the prediction covariance matrix and the updated initial state variable. The device in the present application can be a server, a PC, a PAD, a mobile phone, etc.

[0099] The present application also provides a computer program product adapted to execute the program of the following method steps when executed on a data processing device: obtaining an appearance template library of a target object, wherein the appearance template library comprises at least one appearance image of the target object; obtaining a target image frame to be tracked from a monitoring system, wherein the target image frame comprises at least image information of the target object; tracking the target position coordinates of the target object in the target image frame by a twin neural network and a first preset algorithm according to the appearance image and the target image frame; and performing frame selection on the target object in the target image frame by a second preset algorithm with the target position coordinates as the center to obtain a target appearance image of the target object in the target image frame, and storing the target appearance image into the appearance template library.

[0100] Optionally, the acquiring the appearance template library of the target object comprises: after detecting that the target object enters a monitoring range, detecting a monitoring picture by a second preset algorithm to obtain an appearance image of the target object; and storing the appearance image into the appearance template library.

[0101] Optionally, the tracking, according to the appearance image and the target image frame, to the target position coordinate of the target object in the target image frame by the twin neural network and the first preset algorithm comprises: predicting, according to the appearance image and the target image frame, an initial position coordinate of the target object in the target image frame by the twin neural network; and correcting the initial position coordinate by the first preset algorithm to obtain the target position coordinate.

[0102] Optionally, the predicting, according to the appearance image and the target image frame, to the initial position coordinate of the target object in the target image frame by the twin neural network comprises: obtaining a target sample pair according to the appearance image and the target image frame; extracting features of the target sample pair by the twin neural network to obtain an appearance feature map and a detection feature map; performing cross-correlation calculation on the appearance feature map and the detection feature map to obtain a response map, wherein the response map comprises a plurality of pixel points, and each pixel point corresponds to a similarity value; and taking a coordinate of a pixel point corresponding to a maximum similarity value in the response map as the initial position coordinate.

[0103] Optionally, the obtaining the target sample pair according to the appearance image and the target image frame comprises: processing the appearance image to obtain an appearance image block of a first preset size; processing the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is greater than the first preset size; and assembling the appearance image block and the detection image block into the target sample pair.

[0104] Optionally, the correcting the initial position coordinate by the first preset algorithm to obtain the target position coordinate comprises: obtaining an initial state variable of a filter of the first preset algorithm according to a size of the appearance image and the initial position coordinate; processing the initial state variable according to a process transition matrix and a prediction covariance matrix to obtain a target state variable; and correcting the initial position coordinate according to the target state variable to obtain the target position coordinate.

[0105] Optionally, the processing the initial state variable according to the process transition matrix and the prediction covariance matrix to obtain the target state variable comprises: setting the process transition matrix, and updating the initial state variable according to the process transition matrix to obtain an updated initial state variable; obtaining the prediction covariance matrix, and obtaining the target state variable according to the prediction covariance matrix and the updated initial state variable.

[0106] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0107] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0108] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0109] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0110] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0111] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as a read only memory (ROM), EPROM, EEPROM, or flash memory. The memory can be another form of computer-readable media.

[0112] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0113] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0114] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A target tracking method based on a monitoring system, characterized in that, include: Obtain an appearance template library for the target object, wherein the appearance template library includes at least one appearance image of the target object; The monitoring system acquires target image frames to be tracked, wherein the target image frames include at least image information of the target object; Based on the appearance image and the target image frame, the target object's target position coordinates in the target image frame are tracked using a Siamese neural network and a first preset algorithm; Using the target location coordinates as the center, the target object in the target image frame is selected by a second preset algorithm to obtain the target appearance image of the target object in the target image frame, and the target appearance image is stored in the appearance template library; The library for obtaining the appearance templates of the target object includes: After the target object is detected to have entered the monitoring range, the monitoring screen is detected by the second preset algorithm to obtain the appearance image of the target object; Store the appearance image in the appearance template library; The monitoring screen includes images of multiple customers who need to conduct business. The second preset algorithm detects the image of each customer and establishes an appearance template library for each customer. Specifically, based on the appearance image and the target image frame, tracking the target object's position coordinates in the target image frame using a Siamese neural network and a first preset algorithm includes: Based on the appearance image and the target image frame, the initial position coordinates of the target object in the target image frame are predicted by the Siamese neural network; The initial position coordinates are corrected using the first preset algorithm to obtain the target position coordinates; The process of obtaining the target sample pair based on the appearance image and the target image frame further includes: The appearance image is processed to obtain an appearance image block of a first preset size; The target image frame is processed to obtain a detection image block of a second preset size, wherein the second preset size is larger than the first preset size; The appearance image block and the detection image block are combined to form the target sample pair; The method also includes setting a maximum storage capacity for the appearance template library. When the maximum storage capacity is exceeded, the earliest stored appearance image is deleted, and the appearance template library of the target object is deleted after the target object leaves the monitoring range.

2. The method according to claim 1, characterized in that, Based on the appearance image and the target image frame, the initial position coordinates of the target object in the target image frame are predicted by the Siamese neural network, including: The sample pairs are subjected to feature extraction using the Siamese neural network to obtain an appearance feature map and a detection feature map; The appearance feature map and the detection feature map are cross-correlation calculated to obtain a response map, wherein the response map includes multiple pixels and a similarity value corresponding to each pixel; The coordinates of the pixel corresponding to the maximum similarity value in the response image are used as the initial position coordinates.

3. The method according to claim 2, characterized in that, The initial position coordinates are corrected using the first preset algorithm to obtain the target position coordinates, including: Based on the size of the appearance image and the initial position coordinates, the initial state variables of the filter of the first preset algorithm are obtained; The initial state variables are processed based on the process transition matrix and the prediction covariance matrix to obtain the target state variables; The initial position coordinates are corrected based on the target state variables to obtain the target position coordinates.

4. The method according to claim 3, characterized in that, The initial state variables are processed based on the process transition matrix and the prediction covariance matrix to obtain the target state variables, including: Set the process transition matrix, and update the initial state variables according to the process transition matrix to obtain the updated initial state variables; Obtain the predicted covariance matrix, and based on the predicted covariance matrix and the updated initial state variables, obtain the target state variables.

5. A target tracking device based on a monitoring system, characterized in that, include: The first acquisition unit is used to acquire an appearance template library of the target object, wherein the appearance template library includes at least one appearance image of the target object; The second acquisition unit is used to acquire a target image frame to be tracked from the monitoring system, wherein the target image frame includes at least image information of the target object; The tracking unit is used to track the target object's target position coordinates in the target image frame based on the appearance image and the target image frame, using a Siamese neural network and a first preset algorithm. The selection unit is used to select the target object in the target image frame with the target position coordinates as the center and a second preset algorithm to obtain the target appearance image of the target object in the target image frame, and store the target appearance image in the appearance template library; The first acquisition unit includes: a detection subunit, used to detect the monitoring screen through the second preset algorithm after the target object is detected to enter the monitoring range, and obtain the appearance image of the target object; and a storage subunit, used to store the appearance image into the appearance template library. The monitoring screen includes images of multiple customers who need to conduct business. The second preset algorithm detects the image of each customer and establishes an appearance template library for each customer. The tracking unit is further configured to predict the initial position coordinates of the target object in the target image frame using the Siamese neural network based on the appearance image and the target image frame; and to correct the initial position coordinates using the first preset algorithm to obtain the target position coordinates. The device is further configured to process the appearance image to obtain an appearance image block of a first preset size; process the target image frame to obtain a detection image block of a second preset size, wherein the second preset size is larger than the first preset size; assemble the appearance image block and the detection image block into a target sample pair; set a maximum storage capacity for the appearance template library, and when the maximum storage capacity is exceeded, delete the earliest stored appearance image, and delete the appearance template library of the target object after the target object leaves the monitoring range.

6. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the target tracking method based on a monitoring system as described in any one of claims 1 to 4.

Images