Camera target handover method and handover system

Abstract

The present invention discloses a camera target handover method, which is applied to a monitoring area including at least two cameras, and includes: generating a homography matrix between the two cameras according to the projective relationship between the field of view images in the two cameras; using the homography matrix to map the target objects captured by the two cameras to each other's camera, and matching the target objects in the two cameras with the mapped objects respectively to generate respective one-way matching matrices; merging the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the corresponding relationship between the target objects in the two cameras according to the element values ​​in the two-way matching matrix; using the above-mentioned corresponding relationship to find the known tracking target in the original camera in the target camera for handover. The technical solution provided by the present invention improves the accuracy of target handover, is easy to implement, and has good performance when applied to multi-target handover scenarios.

Description

Technical Field

[0001] The invention relates to the field of video monitoring, and in particular to a camera target handover method and a handover system. Background Art

[0002] In recent years, with the continuous development of smart cities, intelligent transportation and other fields, the single camera has been limited by its large blind spot and small perception field of view when performing monitoring tasks, which has limited the development of related monitoring technologies. Therefore, multi-camera tracking technology has emerged. In actual road scenes, multi-camera tracking technology can realize continuous tracking of suspicious targets in large scenes, and can assist monitoring personnel to better analyze the target's motion trajectory. Among them, target handover tracking is one of the most important links in multi-camera tracking. It aims to identify the same target between cameras through computer vision, image processing and other technologies, so that the target can maintain consistent identification when moving between multiple cameras, so as to greatly save manpower and material resources consumed in the target tracking process. Commonly used target handover tracking algorithms include those based on target feature matching, based on field of view demarcation line, and based on camera coordinate calibration. Among them, the algorithm based on target feature matching has large intra-class differences in the target due to the changes in viewing angle and brightness of the same target between different cameras, resulting in slightly poor matching robustness; the method based on the field of view boundary line relies more on the accuracy of the field of view boundary line position, and the error generated when establishing the field of view boundary line equation will be passed on to the subsequent handover tracking timing judgment and target handover tracking results; in the case of a large number of targets and complex scenes, it is likely to cause target handover tracking errors; the algorithm based on camera coordinate calibration realizes target tracking at all angles by mapping the monitoring target and the camera to the same coordinate system, but the difficulty of obtaining information and the complexity of calculation are relatively high, the real-time performance is poor, and it is difficult to implement. Therefore, further research is needed to realize the camera target handover function with high accuracy and low implementation difficulty. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide a camera target handover method and a handover system, thereby improving the accuracy of camera target handover.

[0004] According to a first aspect, a camera target handover method is applied to a monitoring area, wherein the monitoring area includes at least two cameras, and the method includes:

[0005] Generate a homography matrix between two cameras according to the projective relationship between the field of view images in the two cameras;

[0006] The target objects captured by the two cameras are mapped to each other's camera by using the homography matrix, and the target objects in the two cameras are matched with the mapped objects respectively to generate respective one-way matching matrices;

[0007] Combining the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the corresponding relationship between the target objects in the two cameras according to the element values ​​in the two-way matching matrix;

[0008] The corresponding relationship between the target objects in the two cameras is used to find the known tracking target in the original camera in the destination camera to continue tracking.

[0009] Optionally, generating a homography matrix between two cameras according to a projective relationship between field of view images in the two cameras includes:

[0010] Obtaining the road area of ​​interest in the field of view images of two cameras;

[0011] Matching feature points of the road area of ​​interest in the two images to generate feature point sets for the two images;

[0012] The homography matrix is ​​generated by using the projective relationship between the two feature point sets.

[0013] Optionally, matching the feature points of the road area of ​​interest in the two images to generate feature point sets of the two images respectively includes:

[0014] Extracting feature points of the road area of ​​interest in the two images;

[0015] Perform initial feature point matching based on the similarity between the feature points of the two images;

[0016] The mismatched points in the initial matching result of the feature points are eliminated, and the feature point sets of the two images are obtained.

[0017] Optionally, for the process of mapping the target objects captured by the two cameras to each other's camera using the homography matrix, and matching the target objects in the two cameras with the mapped objects to generate respective one-way matching matrices, the two cameras are controlled to perform the following steps simultaneously:

[0018] Acquire a target detection point representing a target object in the field of view image, wherein the target detection point includes at least a target ID of the target object;

[0019] Receiving the mapping points of the other party's camera, and establishing a one-to-one correspondence between the target detection points and the mapping points, wherein the mapping points are generated by mapping the target detection points of the other party's camera;

[0020] The target detection points and the target IDs of the mapping points are matched according to the one-to-one correspondence to generate a one-way matching matrix according to the matching results.

[0021] Optionally, in the process of establishing the one-to-one correspondence between the target detection points and the mapping points, for any single mapping point, the single mapping point is matched with each target detection point, wherein for any single mapping point, the step of matching the single mapping point with each target detection point includes:

[0022] Calculating the Euclidean distance from the single mapping point to each target detection point in the image coordinate system, and obtaining the minimum distance therebetween;

[0023] When the minimum distance is less than a preset threshold, matching the single mapping point with the target detection point corresponding to the minimum distance;

[0024] When the minimum distance is above the preset threshold, the matching of the single mapping point is abandoned.

[0025] Optionally, the dimension of the one-way matching matrix is ​​m×n, where m and n are the number of target objects in the two cameras respectively; the one-way matching matrix is ​​a binary matrix; the row and column positions of the elements represent the correspondence between the target detection points and the mapping points, and the value of the element represents the matching status of the target ID.

[0026] Optionally, the merging of the unidirectional matching matrices of the two cameras to obtain a bidirectional matching matrix, so as to obtain a correspondence between target objects in the two cameras according to element values ​​in the bidirectional matching matrix, comprises:

[0027] Adding corresponding elements of the one-way matching matrices of the two cameras to obtain a two-way matching matrix;

[0028] The maximum value element in the bidirectional matching matrix indicates that the match is accurate, so that the target IDs corresponding to the rows and columns where the maximum value element is located are matched;

[0029] The intermediate value element in the bidirectional matching matrix indicates that the match needs to be verified, so that the target objects corresponding to the rows and columns where the intermediate value elements are located are subjected to secondary appearance verification;

[0030] The minimum value element in the bidirectional matching matrix indicates that no matching is performed, that is, the target IDs corresponding to the rows and columns where the minimum value element is located are not matched.

[0031] Optionally, the intermediate value element in the bidirectional matching matrix indicates that the match needs to be verified, so that the target objects corresponding to the rows and columns where the intermediate value elements are located are subjected to secondary appearance verification, including:

[0032] Taking the target object image with a smaller size as a reference, unifying the sizes of the two target object images;

[0033] The target object image is converted to the HSV space, and the H, S, and V components of the converted image are non-uniformly quantized at 16-4-4 levels. The formula is as follows:

[0034]

[0035] synthesizing a one-dimensional color vector from the quantized three-channel components, and extracting a color histogram of the target object from the color vector;

[0036] The similarity between the two color histograms is measured. When the similarity is greater than a preset similarity value, the target IDs corresponding to the rows and columns where the intermediate value elements are located are matched, otherwise no match is performed.

[0037] According to a second aspect, a camera target handover system is applied to a monitoring area, wherein the monitoring area includes at least two cameras, and the system includes:

[0038] The camera association module generates a homography matrix between two cameras based on the projective relationship between the field of view images in the two cameras;

[0039] A target mapping module maps the target objects captured by the two cameras to each other's camera using the homography matrix, and matches the target objects in the two cameras with the mapped objects to generate respective one-way matching matrices;

[0040] A target matching module is configured to combine the unidirectional matching matrices of the two cameras to obtain a bidirectional matching matrix, so as to obtain a corresponding relationship between the target objects in the two cameras according to the element values ​​in the bidirectional matching matrix;

[0041] The search and tracking module uses the correspondence between the target objects in the two cameras to find the known tracking target in the original camera in the destination camera to continue tracking.

[0042] According to a third aspect, an electronic device includes:

[0043] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in the first aspect or any optional implementation manner of the first aspect by executing the computer instructions.

[0044] The technical solution of the present invention has the following advantages:

[0045] The embodiment of the present invention provides a camera target handover method and handover system. The method specifically includes: connecting the field of view of two cameras by establishing a homography matrix between the two cameras, and then mapping the target objects captured by the two cameras to the other camera respectively. After the two cameras receive the mapped target objects, they match the mapping points representing the mapped target objects with the target detection points of the target objects captured by themselves, and obtain two one-way matching matrices respectively. In order to further improve the matching accuracy, the corresponding values ​​of the two one-way matching matrices are added, and the target objects that are successfully matched twice are found according to the element values ​​in the matrix, so that the same target objects that appear simultaneously in the two cameras can be accurately found. Therefore, when there is a known target object to be tracked in one of the cameras, it can be accurately found in the other camera, so as to quickly switch the target ID of the target object and realize the handover function. Furthermore, the present invention adopts secondary matching of the target appearance for the case where only one one-way matching matrix is ​​successfully matched, which improves the accuracy of target matching and greatly improves the handover success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A schematic diagram of the steps of a camera target handover method according to an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of a matching process of mapping points and target detection points in a camera target handover method according to an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of a target object bidirectional matching process of a camera target handover method according to an embodiment of the present invention;

[0050] Figure 4 A schematic diagram of the structure of a camera target handover system according to an embodiment of the present invention;

[0051] Figure 5 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0053] The technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] See also Figure 1 The embodiment of the present invention provides a camera target handover method, which is applied to a monitoring area, and the monitoring area includes at least two cameras. The method specifically includes the following steps:

[0055] Step S1: Generate a homography matrix between two cameras according to the projective relationship between the field of view images in the two cameras; Step S2: Use the homography matrix to map the target objects captured by each camera to the other camera, and match the target objects in the two cameras with the mapped objects respectively to generate their own one-way matching matrices; Step S3: Merge the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the corresponding relationship between the target objects in the two cameras according to the element values ​​in the two-way matching matrix; Step S4: Use the corresponding relationship between the target objects in the two cameras to find the known tracking target in the original camera in the destination camera to continue tracking.

[0056] Among them, step S1: generate the homography matrix between the two cameras according to the projective relationship between the field of view images in the two cameras. Specifically, in the process of target handover of the cameras, it is necessary to know how to accurately find the known tracking target in the previous camera in the latter camera. The existing technical means are commonly used in three ways: based on target feature matching, based on field of view dividing line, and based on camera coordinate calibration. Due to the problems of viewing angle and brightness change between different cameras for the same target, the intra-class difference of the target is large, resulting in slightly poor matching robustness of the method based on target feature matching; the method based on field of view dividing line will continuously transmit the recognition error to the rear camera once it is generated during the tracking process. When there are many targets photographed by the camera, this error will be amplified, resulting in the failure of target handover, so that the target tracking cannot be completed, and its reliability needs to be further strengthened; and the method based on camera coordinate calibration is to locate the target object with the help of spatial coordinates, so as to realize camera handover and tracking, but this method needs to establish a complete camera coordinate system in the monitoring area, and locate the target object in real time, which requires a lot of material and financial resources in actual implementation, and is difficult to implement. Therefore, it is of great research value to study a technical solution that can improve the accuracy of target recognition and is easy to implement. The embodiment of the present invention provides a camera target handover method that is executed between two cameras in the monitoring area. It only needs to control the data interaction between the two cameras to accurately match the target object within the camera range. It is very easy to implement from two cameras to multiple cameras. In order to improve the accuracy of the two cameras in identifying the same target object, the embodiment of the present invention adopts a two-way matching of the target object between the two cameras to improve the accuracy. For example: the same target object is identified in camera A and assigned digital ID 10, and is identified in camera B and assigned digital ID 30. Both cameras A and B are controlled to match once. When the matching results of the two cameras are reported as successful, the target is considered to be successfully matched. On this basis, if the target object with digital ID 10 in camera A is the target being tracked, it can be accurately found in camera B to achieve target handover. Since the matching operation needs to be performed in a complete computer control system, before matching, the control center needs to map the target objects captured by the two cameras to the other camera respectively to achieve the target matching operation. Compared with the appearance feature matching method of traditional technology, the handover method provided by the embodiment of the present invention does not directly map the appearance features of the target to the other party, but maps the position coordinate points of the target in the image to the other party through the image coordinate system. On the one hand, it avoids confusion of appearance, and on the other hand, it reduces the amount of calculation and improves the real-time performance of the handover system.Before mapping the detection points of the target object, it is necessary to establish a mapping relationship between the two cameras. The camera handover method provided in the embodiment of the present invention establishes a mapping relationship between the field of view images of the two cameras based on the projection between images of the same position taken by cameras at different positions, thereby obtaining a homography matrix representing the mapping relationship.

[0057] Specifically, in one embodiment, the specific steps of establishing the homography matrix in step S1 are as follows:

[0058] Step S11: Obtain the road region of interest in the two camera field of view images. Specifically, firstly, it is necessary to manually mark the road portion as the region of interest for subsequent processing, so as to discard the image of the area that the target object obviously cannot reach during the target tracking process, while reducing the amount of calculation, removing some interference and improving the accuracy of target recognition.

[0059] Step S12: Match the feature points of the road area of ​​interest in the two images to generate the feature point sets of the two images. Specifically, in the area of ​​interest of the scene image, select the pure background image without interference from movable objects captured by the two cameras, use the SURF algorithm to extract the feature points that can represent the image features, and generate feature point descriptors; then use the Euclidean distance to calculate the similarity between the feature point descriptors in the two images, and perform the initial matching of the feature points between the two images; since there are many mismatches in the feature points after the initial matching, the RANSAC algorithm is used to purify the feature points, eliminate the outer points in the feature point pairs, and obtain the maximum inner point set to facilitate the calculation of the final homography matrix.

[0060] Among them, the steps of using the RANSAC algorithm to purify the feature points are as follows:

[0061] 1. Randomly select 4 pairs of feature points from the initial matching feature point pairs as the initial internal point set, and use them to calculate the initial homography matrix;

[0062] 2. Substitute the remaining feature point pairs into the initial homography matrix and calculate the error of the homography mapping. If the mapping error is less than the preset threshold, it is an inlier and is added to the initial inlier set to obtain an updated inlier set; if it is greater than the threshold, it is an outlier.

[0063] 3. Count the number of inliers in the obtained inlier set. If it is less than the preset threshold, repeat steps 1-3. Otherwise, reselect the initial inlier set and calculate the initial homography matrix.

[0064] 4. Repeat the above steps k times. The calculation method of k is as follows:

[0065]

[0066] Among them, ε is the proportion of internal points in the feature point pair, n is the minimum number of sample points required to calculate the model, and p is the confidence level.

[0067] 5. Find the inlier set with the largest number of inliers as the optimal inlier set, and the largest inlier set is the optimized feature point set.

[0068] Step S13: Generate a homography matrix using the projective relationship between the feature point sets of the two images. Specifically, homography is a concept in projective geometry, also known as projective transformation. It maps points (three-dimensional homogeneous vectors) on a projective plane to another projective plane, and maps straight lines to straight lines, with line-preserving properties. In general, homography is a linear transformation of three-dimensional homogeneous vectors, and the calculation formula is x1=Hx2, where x1 and x2 are the feature point sets of the two images respectively. This is an equation of homogeneous coordinates, and H multiplied by a non-zero scale factor, the above equation still holds, and the homography matrix H can associate the two images. Since both cameras need to map their target detection points to each other's cameras, the positions of x1 and x2 are swapped through the above formulas to calculate the two homography matrices H1 and H2 respectively.

[0069] Among them, step S2: use the homography matrix to map the target objects captured by the two cameras to each other's camera, and match the target objects in the two cameras with the mapped objects to generate their own one-way matching matrices. Specifically, by establishing the homography matrix in step S1, all the target objects captured by the two cameras can be mapped to each other in the actual target tracking process. Before performing two-way matching, it is necessary to control the two cameras to match the target detection points and the mapping points to obtain two one-way matching results, and then merge them. For example: camera A captured 5 target objects, whose target IDs are 1, 2, 3, 4, and 5 respectively, and camera B captured 6 target objects, whose target IDs are 10, 11, 12, 13, 14, and 15. The one-way matching matrix is ​​used to control camera A and camera B to respectively determine which of the above 5 target objects and 6 target objects exist in the shooting pictures of camera A and camera B at the same time, and determine the corresponding relationship of their target IDs.

[0070] Specifically, in one embodiment, in the above step S2, in order to ensure the real-time handover of the target and reduce the system running time, the two cameras are controlled to perform the following steps simultaneously:

[0071] Step S21: Obtain target detection points representing the target object in the field of view image, wherein the target detection points include at least the target ID of the target object. Specifically, before performing target matching, for targets that are not obstructed in the camera, target detection under a single camera can obtain detection points of each single camera representing the target. Its target detection points include at least the target ID, which is convenient for the subsequent determination of the correspondence between targets captured by different cameras. In addition, in an embodiment of the present invention, the target detection point also includes a bounding box for the target, calibrating its position and size in the image coordinate system, so as to facilitate the secondary appearance detection in subsequent steps, and selecting the midpoint of the frame line under each target boundary box as the target detection point representing the target.

[0072] Step S22: Receive the mapping points of the other party's camera, and establish a one-to-one correspondence between the target detection points and the mapping points, where the mapping points are generated by mapping the target detection points of the other party's camera. Figure 2 As shown, after determining the target detection point of this camera, by receiving the mapping point mapped by the target detection point of the other camera through the homography matrix established in step S1, the matching relationship between each mapping point and all target detection points is determined by traversing. In other words, each single mapping point is compared with each target detection point of this camera one by one, so as to find the target detection points representing the same target object in this camera, establish a one-to-one correspondence, and generate a one-way matching matrix. Taking the comparison process of a single mapping point as an example, the specific steps of establishing a one-to-one correspondence between the target detection point and the mapping point are as follows:

[0073] 1. Calculate the Euclidean distance from a single mapping point to each target detection point in the image coordinate system, and obtain the minimum distance;

[0074] 2. When the minimum distance is less than the preset threshold, a single mapping point is matched with the target detection point corresponding to the minimum distance, and the target IDs corresponding to these two points are matched once;

[0075] 3. When the minimum distance is above the preset threshold, the matching of the single mapping point is abandoned. That is, the distance between the received mapping point and all the target detection points in this camera is relatively far. At this time, we believe that the target object represented by this mapping point does not appear in this camera, so the matching is abandoned.

[0076] Step S23: Match the target IDs of the target detection points and the mapping points according to the one-to-one correspondence to generate a one-way matching matrix according to the matching results. Specifically, after the one-to-one correspondence in step S22 is determined, the target IDs representing the same target object in the two cameras can be matched according to the correspondence between the target detection points and the mapping points, and a one-way matching matrix is ​​generated by the data set composed of their matching relationships. For example: the target ID of a pedestrian in camera A is 10, and the target ID in camera B is 30. The two IDs can be associated through the one-to-one correspondence in step S23. If the pedestrian is a known tracking target in camera A, then the target recognition and target handover of the pedestrian can be achieved through the ID30 of camera B corresponding to the ID10 of camera A. By performing target matching in the image coordinate system through the minimum Euclidean distance, the complexity of the target recognition algorithm is greatly reduced, and interference such as confusion of the target appearance is reduced. On the basis of improving the accuracy of target recognition, the difficulty of implementing the technical solution is reduced.

[0077] Among them, the dimension of the one-way matching matrix is ​​m×n, where m and n are the number of target objects in the two cameras respectively; for the convenience of description and judgment, the one-way matching matrix is ​​a binary matrix, and the value of the element represents the matching state of the target ID. In the embodiment of the present invention, 0 and 1 are used to represent the matching state, 1 represents matching, and 0 represents mismatching. The row and column positions of the elements represent the correspondence between the target detection points and the mapping points. For example, if the value of an element in the 3rd row and the 4th column is 1, the information contained in the element is that the 3rd target object in camera A and the 4th target object in camera B are the same target object. Through the established index, the original target IDs of the two target objects are found, thereby determining the correspondence between the target IDs.

[0078] Among them, step S3: merge the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the corresponding relationship between the target objects in the two cameras according to the element values ​​in the two-way matching matrix. Specifically, since the one-way matching matrix controls the matching operation completed by one of the cameras, if the matching result is used as the final result, when a matching error occurs, the same problem as that based on the field of view boundary line will occur, that is, in the process of the camera handover, the error is continuously transmitted backward, resulting in an inaccurate handover result. In order to solve the above problem, in an embodiment of the present invention, if Figure 3 As shown, the corresponding values ​​of the unidirectional matching matrices of the two cameras obtained in step S23 above are added together to obtain a bidirectional matching matrix. By judging the element values ​​in the bidirectional matching matrix, the accuracy of target matching can be greatly improved. The specific matching steps are as follows:

[0079] Step S31: Add corresponding elements of the unidirectional matching matrices of the two cameras to obtain a bidirectional matching matrix.

[0080] Step S32: The maximum value element in the bidirectional matching matrix indicates that the match is accurate, so that the target IDs corresponding to the rows and columns where the maximum value elements are located are matched. Specifically, since the one-way matching matrix is ​​a binary matrix, when the sum of the corresponding elements of the two one-way matrices is the maximum value among all matrix elements, it means that both cameras have determined that the target object corresponding to the element is the same target, and the next step can be performed to match the target IDs of the target object in the two cameras. In the embodiment of the present invention, the binary matrix adopts a 0-1 matrix, so when a certain element value is 2, the element value is the maximum value in the matrix, which means that the matching results of the two one-way matching matrices are both 1, and it can be determined that the target object appears in camera A and camera B at the same time, so as to match the corresponding target ID.

[0081] Step S33: The middle value element in the bidirectional matching matrix indicates that the match needs to be verified, so as to perform a secondary verification on the appearance of the target objects corresponding to the rows and columns where the middle value elements are located. Specifically, when the value of an element is neither the minimum value nor the maximum value, it means that the target object corresponding to the element is judged in only one camera as: the target object appears in both cameras at the same time and the match is successful, but the other camera judges that: the target object does not appear in both cameras at the same time and the match is successful, such as the element with an element value of 1 in the 0-1 matrix example of step S32. For the above-mentioned doubtful situations, further secondary verification is required to improve the accuracy of target matching. The appearance method is used for judgment in the embodiment of the present invention, and its commonly used appearance judgment methods include color, contour, facial features, etc. Since there are problems such as large variations in local features of targets between cameras, low or non-uniform camera resolution, and considering that the target color features have a certain robustness to the above problems, the embodiment of the present invention selects a color histogram as the appearance feature of the target for secondary verification, extracts the HSV color histogram of the target, and obtains a one-dimensional histogram feature vector after quantization. The similarity of the histograms between targets is measured by the Bhattacharyya distance, and targets with a similarity greater than a threshold or higher are matched. The method includes:

[0082] 1. Use the bicubic interpolation method to unify the size of the two target object images, taking the smaller target object image as the benchmark. Specifically, since the distance and posture of the targets captured by the two cameras are different, the sizes of their frame selections are also different. In order to improve the accuracy of similarity analysis, the captured images are first resized;

[0083] 2. Convert the target object image to HSV space. To ensure computational efficiency and accuracy, perform 16-4-4 level non-uniform quantization on the H, S, and V components of the converted image. The formula is as follows:

[0084]

[0085] 3. Synthesize a one-dimensional color vector from the quantized three-channel components, and extract the color histogram of the target object from the color vector;

[0086] 4. Use Bhattacharyya distance to measure the similarity of two color histograms. When the similarity is greater than the preset similarity value, it means that the secondary verification is successful. The target object corresponding to the intermediate value element appears in both cameras at the same time and meets the matching requirements of projection and appearance. The target IDs corresponding to the rows and columns of the intermediate value element are matched. Otherwise, it means that the target object of the element in the secondary verification does not appear in both cameras at the same time and no match is made.

[0087] Step S34: The minimum value element in the bidirectional matching matrix indicates that no matching is done, that is, the target IDs corresponding to the rows and columns where the minimum value element is located are not matched. Specifically, the minimum value element in the bidirectional matching matrix means that the matching results of the two unidirectional matching matrices are both negative results "not the same target object", so the target object corresponding to the element is not matched directly.

[0088] Among them, step S4: use the correspondence between the target objects in the two cameras to find the known tracking target in the original camera in the destination camera to continue tracking. Specifically, after the matching result of the two-way matching matrix is ​​determined, the known tracking target in the original camera is found in the destination camera through the two-way matching matrix, and the handover of the target is realized. For example: in camera A, the target car 1 is being tracked, and its target ID is 10. By finding the corresponding target ID of target ID 10 in the two-way matching matrix is ​​30, and then the target object with target ID 30 is searched in camera B. After locking the target object, the handover is completed and the tracking of the target is continued. This method has excellent performance when applied to the camera handover scene of multi-target tracking because the features it discriminates are simple and not easy to be confused, combined with its high real-time characteristics.

[0089] By executing the above steps, a camera target handover method provided by an embodiment of the present invention connects the field of view of two cameras by establishing a homography matrix between the two cameras, and then maps the target objects captured by the two cameras to the other camera respectively. After the two cameras receive the mapped target objects, they match the mapping points representing the mapped target objects with the target detection points of the target objects captured by themselves, and obtain two one-way matching matrices respectively. In order to further improve the matching accuracy, the corresponding values ​​of the two one-way matching matrices are added, and the target objects that are successfully matched twice are found according to the element values ​​in the matrix, so that the same target objects that appear simultaneously in the two cameras can be accurately found. Therefore, when there is a known target object to be tracked in one of the cameras, it can be accurately found in the other camera, so as to quickly switch the target ID of the target object and realize the handover function. Furthermore, the present invention adopts secondary matching of the target appearance for the case where only one one-way matching matrix is ​​successfully matched, which improves the accuracy of target matching and greatly improves the handover success rate. The technical solution provided by the embodiment of the present invention has low implementation difficulty, is easy to promote on a large scale without adding more hardware, and has excellent performance when applied to the scene of multi-target handover.

[0090] like Figure 4 As shown, this embodiment also provides a camera target handover system, which is applied to electronic equipment, and the system includes:

[0091] The camera association module 101 generates a homography matrix between the two cameras according to the projection relationship between the field of view images of the two cameras. For details, please refer to the relevant description of step S1 in the above method embodiment, which will not be repeated here.

[0092] The target mapping module 102 uses the homography matrix to map the target objects captured by the two cameras to each other's camera, and matches the target objects in the two cameras with the mapped objects to generate their own one-way matching matrices. For details, please refer to the relevant description of step S2 in the above method embodiment, which will not be repeated here.

[0093] The target matching module 103 combines the unidirectional matching matrices of the two cameras to obtain a bidirectional matching matrix, so as to obtain the correspondence between the target objects in the two cameras according to the element values ​​in the bidirectional matching matrix. For details, please refer to the relevant description of step S3 in the above method embodiment, which will not be repeated here.

[0094] The search and tracking module 104 uses the correspondence between the target objects in the two cameras to find the known tracking target in the original camera in the destination camera to continue tracking. For details, please refer to the relevant description of step S4 in the above method embodiment, which will not be repeated here.

[0095] A camera target handover system provided in an embodiment of the present invention is used to execute a camera target handover method provided in the above embodiment. Its implementation method and principle are the same. For details, please refer to the relevant description of the above method embodiment and will not be repeated here.

[0096] Figure 5 An electronic device according to an embodiment of the present invention is shown, the device comprises: a processor 901 and a memory 902, which can be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0097] The processor 901 may be a central processing unit (CPU). The processor 901 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0098] The memory 902 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above method embodiments. The processor 901 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 902, that is, implementing the methods in the above method embodiments.

[0099] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 901, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely arranged relative to the processor 901, and these remote memories may be connected to the processor 901 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0100] One or more modules are stored in the memory 902 , and when executed by the processor 901 , the method in the above method embodiment is executed.

[0101] The specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here.

[0102] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the implemented program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0103] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A camera target handover method, characterized in that: The method is applied to a monitoring area including at least two cameras, and the method comprises: Generate a homography matrix between two cameras according to the projective relationship between the field of view images in the two cameras; The target objects captured by the two cameras are mapped to each other's camera by using the homography matrix, and the target objects in the two cameras are matched with the mapped objects respectively to generate respective one-way matching matrices; Combining the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the corresponding relationship between the target objects in the two cameras according to the element values ​​in the two-way matching matrix; Using the correspondence between the target objects in the two cameras, the known tracking target in the original camera is found in the destination camera to continue tracking; The process of using the homography matrix to map the target objects captured by the two cameras to the other camera, and matching the target objects in the two cameras with the mapped objects to generate their own one-way matching matrices, wherein the two cameras are controlled to perform the following steps simultaneously: obtaining target detection points representing the target objects in the field of view image, wherein the target detection points at least include the target ID of the target object; receiving the mapping points of the other camera, and establishing a one-to-one correspondence between the target detection points and the mapping points, wherein the mapping points are generated by mapping the target detection points of the other camera; matching the target detection points and the target IDs of the mapping points according to the one-to-one correspondence, so as to generate a one-way matching matrix according to the matching results; The dimension of the one-way matching matrix is ​​m×n, where m and n are the number of target objects in the two cameras respectively; the one-way matching matrix is ​​a binary matrix; the row and column positions of the elements represent the corresponding relationship between the target detection point and the mapping point, and the value of the element represents the matching status of the target ID; The merging of the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the correspondence between the target objects in the two cameras according to the element values ​​in the two-way matching matrix, comprises: adding the corresponding elements of the one-way matching matrices of the two cameras to obtain a two-way matching matrix; the maximum value element in the two-way matching matrix indicates that the match is accurate, so as to match the target IDs corresponding to the rows and columns where the maximum value element is located; the middle value element in the two-way matching matrix indicates that the match is to be verified, so as to perform a secondary appearance verification on the target objects corresponding to the rows and columns where the middle value element is located; the minimum value element in the two-way matching matrix indicates that no match is to be made, that is, the target IDs corresponding to the rows and columns where the minimum value element is located are not matched.

2. The method according to claim 1, characterized in that The step of generating a homography matrix between two cameras according to a projective relationship between field of view images in the two cameras comprises: Obtaining the road area of ​​interest in the field of view images of two cameras; Matching feature points of the road area of ​​interest in the two images to generate feature point sets for the two images; The homography matrix is ​​generated by using the projective relationship between the two feature point sets.

3. The method according to claim 2, characterized in that The matching of the feature points of the road area of ​​interest in the two images to generate feature point sets of the two images respectively includes: Extracting feature points of the road area of ​​interest in the two images; Perform initial feature point matching based on the similarity between the feature points of the two images; The mismatched points in the initial matching result of the feature points are eliminated, and the feature point sets of the two images are obtained.

4. The method according to claim 1, characterized in that: In the process of establishing the one-to-one correspondence between the target detection points and the mapping points, for any single mapping point, the single mapping point is matched with each target detection point, wherein for any single mapping point, the step of matching the single mapping point with each target detection point includes: Calculating the Euclidean distance from the single mapping point to each target detection point in the image coordinate system, and obtaining the minimum distance therebetween; When the minimum distance is less than a preset threshold, matching the single mapping point with the target detection point corresponding to the minimum distance; When the minimum distance is above the preset threshold, the matching of the single mapping point is abandoned.

5. The method according to claim 1, characterized in that The intermediate value element in the bidirectional matching matrix indicates that the match needs to be verified, so that the target objects corresponding to the rows and columns where the intermediate value elements are located are subjected to secondary appearance verification, including: Taking the target object image with a smaller size as a reference, unifying the sizes of the two target object images; The target object image is converted to HSV space, and the H, S, and V components of the converted image are non-uniformly quantized at 16-4-4 levels. The formula is as follows: Combining the quantized three-channel components into a one-dimensional color vector, and extracting a color histogram of the target object from the color vector; The similarity between the two color histograms is measured. When the similarity is greater than a preset similarity value, the target IDs corresponding to the rows and columns where the intermediate value elements are located are matched, otherwise no match is performed.

6. A camera target handover system, characterized in that: The system is applied to a monitoring area, wherein the monitoring area includes at least two cameras, and the system includes: The camera association module generates a homography matrix between two cameras based on the projective relationship between the field of view images in the two cameras; A target mapping module, using the homography matrix to map the target objects captured by the two cameras to the other camera, and matching the target objects in the two cameras with the mapping objects to generate their own one-way matching matrices; for the process of using the homography matrix to map the target objects captured by the two cameras to the other camera, and matching the target objects in the two cameras with the mapping objects to generate their own one-way matching matrices, wherein the two cameras are controlled to perform the following steps simultaneously: obtaining target detection points representing the target objects in the field of view image, wherein the target detection points at least include the target ID of the target object; receiving the mapping points of the other camera, and establishing a one-to-one correspondence between the target detection points and the mapping points, wherein the mapping points are generated by mapping the target detection points of the other camera; matching the target detection points and the target IDs of the mapping points according to the one-to-one correspondence, so as to generate a one-way matching matrix according to the matching results; The dimension of the one-way matching matrix is ​​m×n, where m and n are the number of target objects in the two cameras respectively; the one-way matching matrix is ​​a binary matrix; the row and column positions of the elements represent the corresponding relationship between the target detection point and the mapping point, and the value of the element represents the matching status of the target ID; The merging of the one-way matching matrices of the two cameras to obtain a two-way matching matrix, so as to obtain the corresponding relationship between the target objects in the two cameras according to the element values ​​in the two-way matching matrix, comprises: adding the corresponding elements of the one-way matching matrices of the two cameras to obtain a two-way matching matrix; the maximum value element in the two-way matching matrix indicates that the match is accurate, so as to match the target IDs corresponding to the rows and columns where the maximum value element is located; the middle value element in the two-way matching matrix indicates that the match is to be verified, so as to perform a secondary appearance verification on the target objects corresponding to the rows and columns where the middle value element is located; the minimum value element in the two-way matching matrix indicates that no match is to be made, that is, the target IDs corresponding to the rows and columns where the minimum value element is located are not matched; A target matching module is configured to combine the unidirectional matching matrices of the two cameras to obtain a bidirectional matching matrix, so as to obtain a corresponding relationship between the target objects in the two cameras according to the element values ​​in the bidirectional matching matrix; The search and tracking module uses the correspondence between the target objects in the two cameras to find the known tracking target in the original camera in the destination camera to continue tracking.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 5 by executing the computer instructions.

Images