Target detection method and device based on monocular vision

Abstract

The present invention discloses a target detection method and device based on monocular vision, which relates to the field of image recognition technology and can accurately locate the 3D spatial coordinates of a target in a scene in real time, thereby achieving real-time positioning and tracking of the target in 3D space. The method includes: using a SLAM algorithm to model the scene to obtain a global map model, while simultaneously obtaining the external parameters of the SLAM camera; obtaining a set of images captured by a security camera, calibrating the security camera based on the external parameters of the SLAM camera, and obtaining the external parameters of the security camera in the global map model; identifying the target in the image captured by the security camera through target detection technology, and performing monocular distance measurement on the target; converting the coordinates of the target in the camera coordinate system to the global coordinate system determined by the SLAM camera, and displaying them in real time in the global map model. The device applies the method proposed in the above scheme.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a target detection method and device based on monocular vision. Background Art

[0002] In the era of booming artificial intelligence (AI), various AI applications are mushrooming. With the advent of the intelligent era, integrating offline and online retail models, or replicating online retail models offline, has become a research hotspot. Real-time detection, positioning, and tracking of objects in a scene, thereby capturing their motion trajectory and activity areas, is crucial for the digitization and intelligentization of offline operations.

[0003] Current methods for localizing targets include GPS-based methods using the target's own device, visual odometry SLAM methods, and methods that rely on externally mounted cameras for detection and tracking. GPS-based methods often fail in indoor scenarios, while methods based on the target's own device are not suitable for retail scenarios. Methods that rely on externally mounted cameras can be categorized as those that rely on camera overlap and those that do not. Methods that rely on camera overlap, also known as binocular-based methods, require that the associated cameras have a large overlap in their field of view when installed. Image matching is then performed between the two cameras to recover the target's 3D position. However, a large overlap significantly reduces the effective field of view of each camera, increasing the number of cameras required per unit area and the hardware cost of the solution. Methods that do not rely on camera overlap currently use techniques such as ReID combined with camera position information to obtain discontinuous target trajectories, failing to obtain real-time, continuous trajectories. Target tracking is also limited to the camera image space, and the physical meaning of the tracking results is not closely linked to the real world, limiting their usefulness. In addition, cross-camera tracking methods rely on high-performance ReID technology, which has high requirements on the accuracy and robustness of related algorithms and the computing power of computing devices. Summary of the Invention

[0004] The purpose of the present invention is to provide a target detection method and device based on monocular vision, which can accurately locate the 3D spatial coordinates of the target in the scene in real time and realize real-time positioning and tracking of the target in 3D space.

[0005] In order to achieve the above object, a first aspect of the present invention provides a target detection method based on monocular vision, comprising:

[0006] Use SLAM algorithm to model the scene to obtain a global map model, and at the same time obtain the external parameters of the SLAM camera;

[0007] Acquire a set of images captured by a security camera, calibrate the security camera based on the extrinsic parameters of the SLAM camera, and obtain the extrinsic parameters of the security camera in the global map model;

[0008] Identify targets in images captured by security cameras using target detection technology and perform monocular distance measurement on the targets;

[0009] The coordinates of the target in the camera coordinate system are converted to the global coordinate system determined by the SLAM camera and displayed in real time in the global map model.

[0010] Preferably, the method of using the SLAM algorithm to model the scene to obtain a global map model and simultaneously obtaining the external parameters of the SLAM camera includes:

[0011] Use SLAM algorithm to obtain the scene point cloud reconstruction results, SLAM camera pose set and reconstructed image frame set;

[0012] Constructing a global map model based on the point cloud reconstruction results, and scanning and photographing the scene using a SLAM camera to obtain a set of reconstructed image frames consisting of multiple scene images;

[0013] The FAST algorithm is used to sequentially obtain feature points in two adjacent scene images and obtain the feature descriptors of the feature points. Then, based on the feature descriptors of the feature points, similar feature points in the two adjacent scene images are matched in sequence, and a unified camera coordinate system with the camera coordinate system of the first scene image as a reference is constructed, and the global coordinate system is obtained after conversion.

[0014] Preferably, the method of obtaining a set of images captured by a security camera, calibrating the security camera based on the extrinsic parameters of the SLAM camera, and obtaining the extrinsic parameters of the security camera in the global map model includes:

[0015] Performing point matching of the same name on the image set captured by the security camera and the reconstructed image frame set;

[0016] Solve the 3D spatial coordinates of the same-name points in the global coordinate system based on the SLAM camera pose set;

[0017] The posture parameters of the security camera relative to the SLAM camera are calculated based on the 3D spatial coordinates to obtain the external parameters of the security camera in the global map model.

[0018] Preferably, the method of identifying a target in an image captured by a security camera using target detection technology and performing monocular ranging on the target includes:

[0019] Use target detection technology to locate the target in the security camera image and identify the target's external rectangular frame;

[0020] Calculating an average physical size of the target using a clustering algorithm based on a plurality of depth image sets containing the target captured by a depth camera;

[0021] According to the width and height parameters of the circumscribed rectangular frame, the focal length of the security camera and the average physical size, the distance of the target relative to the corresponding security camera is measured using the monocular ranging principle.

[0022] Preferably, after the step of measuring the distance of the target relative to the corresponding security camera using the monocular ranging principle, the method further includes:

[0023] Based on the internal parameters of the security camera and the coordinates of the center point of the circumscribed rectangular frame in the image coordinate system, the target is converted into coordinates relative to the camera coordinate system.

[0024] Preferably, the method of converting the coordinates of the target in the camera coordinate system to the global coordinate system determined by the SLAM camera and displaying them in real time in the global map model includes:

[0025] Based on the external parameters of the security camera in the global map model, the target coordinates relative to the camera coordinate system are converted into 3D spatial coordinates relative to the target in the global coordinate system through a spatial coordinate system conversion matrix;

[0026] The 3D spatial coordinates of the target in the global coordinate system are displayed in real time in the global map model.

[0027] Preferably, it also includes:

[0028] Target tracking technology is used to track the target in real time in a global map model, and the real-time position trajectory and activity area heat map of the target are drawn.

[0029] Compared with the prior art, the target detection method based on monocular vision provided by the present invention has the following beneficial effects:

[0030] In the target detection method based on monocular vision provided by the present invention, the SLAM algorithm is first used to model the monitoring scene to obtain a global map model, and the external parameters of the SLAM camera are obtained. For example, the external parameters are the posture parameters of each SLAM camera. Then, the security cameras installed in the monitoring scene are used to take real-time images to construct an image set. The corresponding security cameras are calibrated in combination with the external parameters of the SLAM cameras at the corresponding positions, and the external parameters of each security camera in the global map model are obtained respectively. Then, the target in the image captured by the security camera is identified by target detection technology, and the target is measured monocularly to obtain the position distance of the target relative to the corresponding security camera. Finally, the coordinates of the target in the camera coordinate system are converted to the global coordinate system and displayed in real time in the global map model.

[0031] In summary, this invention uses a monocular ranging method to locate a target monocularly. Compared to existing GPS-based positioning methods, it can solve the problem of inaccurate indoor positioning. Furthermore, it uses a SLAM algorithm to construct a global map model, and converts the target in the security camera's monitoring image into this global map model for real-time display, enabling continuous tracking of the target in 3D space.

[0032] A second aspect of the present invention provides a monocular vision-based target detection device, which is applied to the monocular vision-based target detection method described in the above technical solution, and the device includes:

[0033] The global modeling unit is used to model the scene using the SLAM algorithm to obtain a global map model and obtain the external parameters of the SLAM camera;

[0034] a calibration unit, configured to obtain a set of images captured by a security camera, calibrate the security camera based on the extrinsic parameters of the SLAM camera, and obtain the extrinsic parameters of the security camera in the global map model;

[0035] The ranging unit is used to identify the target in the image captured by the security camera through target detection technology and perform monocular distance measurement on the target;

[0036] The coordinate conversion unit is used to convert the coordinates of the target in the camera coordinate system into the global coordinate system determined by the SLAM camera and display them in real time in the global map model.

[0037] Preferably, it also includes:

[0038] The target tracking unit uses target tracking technology to track the target in real time in the global map model, and draws the real-time position trajectory and activity area heat map of the target.

[0039] Compared with the prior art, the beneficial effects of the monocular vision-based target detection device provided by the present invention are the same as the beneficial effects of the monocular vision-based target detection method provided by the above technical solution, and will not be repeated here.

[0040] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned monocular vision-based target detection method are executed.

[0041] Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as the beneficial effects of the target detection method based on monocular vision provided by the above technical solution, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0043] Figure 1 Schematic diagram of the process of a target detection method based on monocular vision in an embodiment of the present invention;

[0044] Figure 2 Schematic diagram of the process of obtaining a SLAM camera pose set and reconstructing an image frame set using a SLAM algorithm in an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a process for calibrating a security camera using a SLAM camera pose set and a reconstructed image frame set to obtain the external parameters of the security camera in a global map model in an embodiment of the present invention;

[0046] Figure 4 The figure is a flow chart of obtaining the 3D spatial coordinates of a target using a security camera pose set and a monocular ranging method in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] Example 1

[0049] See also Figure 1 This embodiment provides a target detection method based on monocular vision, including:

[0050] The SLAM algorithm is used to model the scene to obtain a global map model and obtain the external parameters of the SLAM camera; a set of images collected by the security camera is obtained, and the security camera is calibrated based on the external parameters of the SLAM camera to obtain the external parameters of the security camera in the global map model; the target in the image collected by the security camera is identified through target detection technology, and the target is measured monocularly; the coordinates of the target in the camera coordinate system are converted to the global coordinate system and displayed in real time in the global map model.

[0051] In the target detection method based on monocular vision provided in this embodiment, the SLAM algorithm is first used to model the monitoring scene to obtain a global map model, and the external parameters of the SLAM camera are obtained. For example, the external parameters are the posture parameters of each SLAM camera. Then, the security cameras installed in the monitoring scene are used to take real-time images to construct an image set. The corresponding security cameras are calibrated in combination with the external parameters of the SLAM cameras at the corresponding positions to obtain the external parameters of each security camera in the global map model. Then, the target in the image captured by the security camera is identified through target detection technology, and the target is measured monocularly to obtain the position distance of the target relative to the corresponding security camera. Finally, the coordinates of the target in the camera coordinate system are converted to the global coordinate system and displayed in real time in the global map model.

[0052] In summary, this embodiment uses a monocular ranging method to locate a target monocularly. Compared to existing GPS-based positioning methods, it can solve the problem of inaccurate indoor positioning. Furthermore, a SLAM algorithm is used to construct a global map model, and the target in the security camera's monitoring image is converted to this global map model for real-time display, enabling continuous tracking of the target in 3D space.

[0053] See also Figure 2 In the above embodiment, the method of using the SLAM algorithm to model the scene to obtain a global map model and simultaneously obtaining the external parameters of the SLAM camera includes:

[0054] The SLAM algorithm is used to obtain the point cloud reconstruction results of the scene, the SLAM camera pose set and the reconstructed image frame set; a global map model is constructed based on the point cloud reconstruction results, and the scene is scanned and photographed by the SLAM camera to obtain a reconstructed image frame set consisting of multiple scene images; the FAST algorithm is used to sequentially obtain the feature points in two adjacent scene images, and the feature descriptors of the feature points are obtained. Then, based on the feature descriptors of the feature points, similar feature points in the two adjacent scene images are matched in turn, and a unified camera coordinate system with the camera coordinate system of the first scene image as a reference is constructed, and the global coordinate system is obtained after conversion.

[0055] In the specific implementation, the point cloud reconstruction result of the monitoring scene, the SLAM camera pose set and the reconstructed image frame set are obtained by SLAM technology. The SLAM camera pose set includes the pose parameters of each SLAM camera, which is used to represent the installation position of each SLAM camera in the global map model. For example, the SLAM camera pose set is {(R 0 ,T 0 ),(R 1 ,T 1 ),(R 2 ,T 2 ),…,(R n ,T n)}, reconstruct the image frame set {I0,I1,I2,…,I n}, the reconstructed image frame set is the scene image continuously captured by the SLAM camera, where I0 represents the scene image captured by the SLAM camera at the first moment, I1 represents the scene image captured by the SLAM camera at the second moment, and so on. n represents the scene image captured by the SLAM camera at time n+1. This embodiment uses the SLAM algorithm to model the monitoring scene and constructs a global map model based on the point cloud reconstruction results. The SLAM camera is used to continuously scan and capture the monitoring scene. The scene images of the t-1th frame and the tth frame are used as examples for explanation. The FAST algorithm is used to extract feature points from the scene images of the t-1th frame and the tth frame. In the feature point extraction process, for each pixel point in the scene image, the number of pixels (p1, p2, ...) is calculated for a circle with a radius of 3, with each pixel as the center. ,p16), for the sake of convenience in calculation, by setting a threshold, calculate the pixel difference between p1, p9, p5, p13 and the center point p. If at least 3 of their pixel difference absolute values ​​exceed the set threshold, the center point P is selected as a candidate corner point, and then the next step of investigation is carried out. Otherwise, the center point P is considered not to be a candidate corner point. If p is a candidate corner point, calculate the pixel difference between the 16 pixel points p1 to p16 and the center point p respectively. If at least 9 consecutive pixel points have the pixel difference absolute value exceeding the threshold with the center point p, the center point p is finally considered to be a feature point. Otherwise, the center point p is considered not to be a feature point.

[0056] The above method is used to traverse the adjacent frame scene images in turn to obtain the feature points in each frame scene image, and the feature points in each frame scene image are screened by non-maximum suppression. The specific method is as follows: calculate the FAST score value of the feature points in each frame scene image (that is, the score value, also known as the s value, which is used to represent the sum of the absolute values ​​of the difference between the pixels in the neighborhood and the center point pixels), and judge a neighborhood (such as 3x3 or 5x5) with the feature point p as the center point. If there are multiple feature points in the neighborhood, the s value of each feature point is judged. If the center point p has the largest score value among all the feature points in the neighborhood, it is retained. Otherwise, the pixel point is suppressed. If there is only one feature point in the neighborhood, it is retained. The calculation formula of score s is as follows, and t represents the threshold:

[0057]

[0058] After obtaining the feature points, the feature descriptor is calculated for each feature point in the adjacent frame scene images, and then all the feature points in the two adjacent scene images are matched using the feature descriptors. Then, the 3D spatial coordinates of the matching feature points relative to the camera coordinate system at time t-1 are calculated using the depth image of the t-1 frame image, and finally the pose transformation relationship (R t ,T t ), and so on, we can finally obtain a unified camera coordinate system with the camera coordinate system at the first moment of scene shooting (denoted as o) as the reference. Then the transformation relationship of the k-th moment pose transformation relative to the first moment camera pose o is as shown in the formula:

[0059]

[0060] See also Figure 3 In the above embodiment, the method of obtaining a set of images captured by a security camera, calibrating the security camera based on the external parameters of the SLAM camera, and obtaining the external parameters of the security camera in the global map model includes:

[0061] The image set collected by the security camera is matched with the reconstructed image frame set by similarity. Based on the similarity, the similar reconstructed image frame corresponding to each security camera image is obtained to form a security camera image-reconstructed image pair, and then the same-name points are matched for each pair; the 3D spatial coordinates of the same-name points in the global coordinate system are solved according to the SLAM camera pose set; the pose parameters of the security camera relative to the SLAM camera are calculated based on the 3D spatial coordinates to obtain the external parameters of the security camera in the global map model.

[0062] In specific implementation, after the security camera is installed in the monitoring scene, it is used to collect monitoring images in the scene in real time to build an image set. Then, the posture of the security camera is calibrated using the SlAM camera posture parameters as a reference. The following is an example of security camera No. k:

[0063] The visual vocabulary of the bag-of-words model is obtained by using the SLAM reconstructed image frame set and the image collected in real time by the k-th camera. The visual vocabulary corresponding to the security camera to be calibrated is matched with the visual vocabulary of each image in the reconstructed image frame set to obtain the similarity set {l0,l1,l2,…,l n}, based on the threshold, the m SLAM camera pose candidates with the highest matching degree are selected. Then the security camera is calibrated based on each candidate pose, and the pose (R k ,T k ) is used as a calibration reference as an example, and its corresponding reconstructed image frame is I k , respectively treat the image captured by the calibration camera and Ik Obtain feature points and calculate their feature vectors, then perform feature point matching to obtain the same-name points in the two images. The 3D coordinates of the same-name points in the SLAM camera coordinate system are solved through the corresponding SLAM camera external parameters and its depth image. After solving and optimizing the reprojection error through pnp, the pose parameters of the security camera k relative to the SLAM camera are obtained. Finally, based on the m camera poses, the poses of m cameras to be calibrated can be obtained. For these m poses, calculate the reprojection error {e1,e2,e3,…,e m}, the calibrated pose is comprehensively scored according to the reprojection error and image matching similarity. The score of each pose is denoted as s, and the calculation method of s (3) is as follows:

[0064]

[0065] Sort by the score of each posture, and select the posture parameter with the highest score as the calibration result, which is recorded as (r k ,t k ). Taking this as an example, we can obtain the pose set of the security camera {(r 0 ,t 0 ),(r 1 ,t 1 ),(r 2 ,t 2 ),…,(r n ,t n )}, n is the number of security cameras installed, k≤n. Then the external parameters of security camera k relative to the global coordinate system are The conversion relationship formula (4) is as follows:

[0066]

[0067] In the above embodiment, the method of identifying a target in an image captured by a security camera using target detection technology and performing monocular ranging on the target includes:

[0068] The target in the security camera image is located through target detection technology, and the target's circumscribed rectangular frame is identified; based on a set of multiple depth images containing the target taken by the depth camera, the target's size is measured using the depth camera, and the average physical size of the target is calculated using a clustering algorithm for the multiple measured sizes obtained; based on the width and height parameters of the circumscribed rectangular frame, the focal length of the security camera, and the average physical size, the distance of the target relative to the corresponding security camera is measured using the monocular ranging principle.

[0069] In specific implementation, the target markers in the security camera image are located using target detection technology. Taking security camera k as an example, the position of the bounding rectangle of the target marker in the camera k image is detected, and the width or height w' of the target marker image is calculated based on the width and height of the bounding rectangle. By installing a depth camera at the entrance of the scene, the image of the target entering the monitoring scene is collected, and the size set of the target markers is obtained through depth camera measurement. The average physical size w of this type of target marker is solved by the clustering algorithm, combined with the focal length f of security camera k. k , using the monocular ranging principle, the distance d between the target marker and the security camera k can be measured k , the measurement solution formula (5) is as follows:

[0070]

[0071] The coordinates of the target marker relative to the corresponding camera coordinate system can be obtained by using the internal parameters of the security camera and the coordinates of the center point of the circumscribed rectangular frame of the target marker (u, v).

[0072] The calculation formula (6) is as follows:

[0073]

[0074]

[0075]

[0076] where c u 、c v is the optical center coordinate of the security camera in the x and y directions, f u 、f v = is the focal length of the security camera. In real-world applications, objects with standard sizes or small size variations, such as license plates, lane markings, and human heads, can be selected as reference markers for ranging.

[0077] In the above embodiment, after the step of measuring the distance of the target relative to the corresponding security camera using the monocular ranging principle, the method further includes:

[0078] Based on the internal parameters of the security camera and the center point coordinates of the circumscribed rectangular frame in the image coordinate system, the target center point coordinates are converted into coordinates relative to the camera coordinate system as the target position representative.

[0079] See also Figure 4 In the above embodiment, the method of converting the coordinates of the target in the camera coordinate system to the global coordinate system and displaying them in real time in the global map model includes:

[0080] Based on the external parameters of the security camera in the global map model, the target coordinates relative to the camera coordinate system are converted into 3D spatial coordinates relative to the target in the global coordinate system through the spatial coordinate system conversion matrix; the 3D spatial coordinates of the target in the global coordinate system are displayed in real time in the global map model.

[0081] In specific implementation, after obtaining the 3D spatial coordinates of the target relative to the camera coordinate system, the 3D spatial coordinates of the target relative to the camera coordinate system are converted to the unified camera coordinate system determined by the SLAM camera through the spatial coordinate system conversion matrix using the calibrated security camera pose information. That is, the coordinates are converted to the o coordinate system. Similarly, taking the target position in the kth security camera as an example, the reference pose of the kth security camera calibration is (R k ,T k ), the position coordinates of the target in the global coordinate system o are expressed as Finally, the 3D spatial coordinates of the target in the global coordinate system are displayed in real time in the global map model. The conversion formula (7) is as follows:

[0082]

[0083] The above embodiment also includes: using target tracking technology to track the target in real time in a global map model, and drawing the real-time position trajectory and activity area heat map of the target.

[0084] In specific implementation, target tracking technology is used to directly predict, track and connect the target position in real time in the 3D space of the global map model, obtain the real-time position of the target, and obtain the real-time position trajectory and activity area heat map of the target in the entire scene for accurate analysis of target behavior.

[0085] In summary, this embodiment has the following innovations:

[0086] 1. Monocular positioning of the target is performed based on the principle of monocular ranging, which solves the problem of inaccurate indoor positioning based on GPS positioning method;

[0087] 2. Using security cameras to locate targets can obtain continuous trajectory information of targets in real time, and the cost of security cameras is relatively low;

[0088] 3. Based on the principle of monocular ranging, the target is positioned monocularly and the 3D spatial coordinates of the target are obtained by combining calibration information. No large overlapping area is required between security cameras, which increases the effective utilization area of ​​security cameras and reduces the hardware cost per unit area.

[0089] 4. Use the SLAM camera pose obtained in combination with SLAM scene reconstruction to perform a one-time joint calibration of the SLAM cameras in the entire scene to form a unified coordinate system. This method can be reused in different scenes. The scene reconstruction result is used as a global map model to display the location of the tracked target in real time.

[0090] 5. Real-time tracking and positioning of targets in a global map model has clear physical meaning and is more accurate than pure image space positioning solutions. It does not rely on ReID technology, saves computing resources, and reduces hardware costs.

[0091] As can be seen, this embodiment utilizes reusable camera calibration and monocular ranging technology to perform real-time, full-scene 3D positioning and tracking of targets, enabling digital modeling of target movement trajectories in offline monitoring scenarios. Using monocular ranging and SLAM camera calibration technology, targets are located in real time with a positioning error within 10%. This embodiment has diverse application scenarios. For example, it can be used to locate and track vehicles in parking lots, obtain real-time trajectory information after entering the garage, extract the vehicle's final stopping point to determine the vehicle's parking position, and perform real-time intelligent maintenance of parking spaces in the garage, providing real-time information on which spaces are occupied and which are vacant. In combination with signage and other methods, newly entered vehicles can be guided to vacant spaces, improving parking efficiency. In combination with a parking control system, some parking spaces can be reserved in advance, solving parking difficulties and enhancing the parking experience. In large parking lots, a car search system can also be used to help drivers find their vehicles, addressing the issue of forgotten parking locations and requiring assistance.

[0092] For another example, this embodiment can be used to detect and track pedestrians in scenes such as shopping malls, and obtain the continuous real-time activity trajectory of pedestrians and regional heat maps, which plays an important role in customer flow statistics and merchant traffic diversion; when applied to stores, it can obtain customers' areas of interest, understand customers' preferences, know which areas are more popular for goods, or which areas are easier for customers to walk to, etc., to guide stores in the placement of goods and the selection of goods.

[0093] For ease of understanding, the following terms are explained:

[0094] 1. Point cloud: A collection of points after the pixels on the image are converted into three-dimensional space.

[0095] 2. Global coordinate system (world coordinate system): A coordinate point formed by taking a certain point in the real world as the coordinate system.

[0096] 3. Camera coordinate system: A coordinate system that is parallel to the x-axis and y-axis of the imaging plane coordinate system, with the optical axis of the camera as the axis and perpendicular to the image plane.

[0097] 4. Image coordinate system: the rectangular coordinate system uv defined on the image in pixels.

[0098] 5. Camera intrinsic parameters: parameters describing the camera's optical center, focal length, distortion, etc.

[0099] 6. Camera extrinsics: Parameters that describe the rotation and translation of the camera relative to the reference coordinate system.

[0100] Example 2

[0101] This embodiment provides a monocular vision-based target detection device, comprising:

[0102] The global modeling unit is used to model the scene using the SLAM algorithm to obtain a global map model and obtain the external parameters of the SLAM camera;

[0103] a calibration unit, configured to obtain a set of images captured by a security camera, calibrate the security camera based on the extrinsic parameters of the SLAM camera, and obtain the extrinsic parameters of the security camera in the global map model;

[0104] The ranging unit is used to identify the target in the image captured by the security camera through target detection technology and perform monocular distance measurement on the target;

[0105] The coordinate conversion unit is used to convert the coordinates of the target in the camera coordinate system into the global coordinate system and display them in real time in the global map model.

[0106] Compared with the prior art, the beneficial effects of the monocular vision-based target detection device provided by the embodiment of the present invention are the same as the beneficial effects of the monocular vision-based target detection method provided by the above-mentioned embodiment 1, and will not be repeated here.

[0107] Example 3

[0108] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned monocular vision-based target detection method are executed.

[0109] Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment are the same as the beneficial effects of the monocular vision-based target detection method provided by the above technical solution, and will not be elaborated here.

[0110] Those skilled in the art will understand that all or part of the steps in the above-mentioned inventive method can be implemented by instructing related hardware through a program, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-mentioned embodiment method, and the storage medium can be: ROM / RAM, disk, optical disk, memory card, etc.

[0111] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A target detection method based on monocular vision, characterized in that: include: Use SLAM algorithm to model the scene to obtain a global map model and obtain the external parameters of the SLAM camera; Acquire a set of images captured by a security camera, calibrate the security camera based on the extrinsic parameters of the SLAM camera, and obtain the extrinsic parameters of the security camera in the global map model; Identify targets in images captured by security cameras using target detection technology and perform monocular distance measurement on the targets; Convert the coordinates of the target in the camera coordinate system to the global coordinate system and display them in real time in the global map model; The method of using the SLAM algorithm to model the scene to obtain a global map model and obtain the external parameters of the SLAM camera includes: Using a SLAM algorithm to obtain a point cloud reconstruction result of the scene, a SLAM camera pose set, and a reconstructed image frame set; wherein the SLAM camera pose set includes pose parameters of each SLAM camera, which are used to represent the installation position of each SLAM camera in the global map model; Constructing a global map model based on the point cloud reconstruction results, and scanning and photographing the scene using a SLAM camera to obtain a set of reconstructed image frames consisting of multiple scene images; The FAST algorithm is used to sequentially obtain feature points in two adjacent scene images and obtain feature descriptors for the feature points. Based on the feature descriptors of the feature points, similar feature points in the two adjacent scene images are matched in sequence to construct a unified camera coordinate system with the camera coordinate system of the first scene image as a reference. The global coordinate system is obtained through conversion. The method of sequentially acquiring feature points in two adjacent scene images using the FAST algorithm includes: Calculate the FAST score of the feature points in each scene image, and judge a neighborhood with the target feature point as the center point. If there are multiple feature points in the neighborhood, judge the FAST score of each feature point. If the target feature point has the largest score among all the feature points in the neighborhood, retain it; otherwise, suppress the target feature point. If there is only one feature point in the neighborhood, retain it.

2. The method according to claim 1, characterized in that The method of obtaining a set of images captured by a security camera, calibrating the security camera based on the external parameters of the SLAM camera, and obtaining the external parameters of the security camera in a global map model includes: Performing point matching of the same name on the image set captured by the security camera and the reconstructed image frame set; Solve the 3D spatial coordinates of the same-name points in the global coordinate system based on the SLAM camera pose set; The posture parameters of the security camera relative to the SLAM camera are calculated based on the 3D spatial coordinates to obtain the external parameters of the security camera in the global map model.

3. The method according to claim 2, characterized in that Methods for identifying targets in images captured by security cameras using target detection technology and performing monocular ranging on the targets include: Use target detection technology to locate the target in the security camera image and identify the target's external rectangular frame; Calculating an average physical size of the target using a clustering algorithm based on a plurality of depth image sets containing the target captured by a depth camera; According to the width and height parameters of the circumscribed rectangular frame, the focal length of the security camera and the average physical size, the distance of the target relative to the corresponding security camera is measured using the monocular ranging principle.

4. The method according to claim 3, characterized in that After the step of measuring the distance of the target relative to the corresponding security camera using the monocular ranging principle, the method further includes: Based on the internal parameters of the security camera and the coordinates of the center point of the circumscribed rectangular frame in the image coordinate system, the target is converted into coordinates relative to the camera coordinate system.

5. The method according to claim 4, characterized in that The method of converting the coordinates of the target in the camera coordinate system into the global coordinate system and displaying the coordinates in real time in the global map model includes: Based on the external parameters of the security camera in the global map model, the target coordinates relative to the camera coordinate system are converted into 3D spatial coordinates relative to the target in the global coordinate system through a spatial coordinate system conversion matrix; The 3D spatial coordinates of the target in the global coordinate system are displayed in real time in the global map model.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: Target tracking technology is used to track the target in real time in a global map model, and the real-time position trajectory and activity area heat map of the target are drawn.

7. A target detection device based on monocular vision, characterized in that: include: The global modeling unit is used to model the scene using the SLAM algorithm to obtain a global map model and obtain the external parameters of the SLAM camera; a calibration unit, configured to obtain a set of images captured by a security camera, calibrate the security camera based on the extrinsic parameters of the SLAM camera, and obtain the extrinsic parameters of the security camera in the global map model; The ranging unit is used to identify the target in the image captured by the security camera through target detection technology and perform monocular distance measurement on the target; A coordinate conversion unit, used to convert the coordinates of the target in the camera coordinate system into the global coordinate system and display them in real time in the global map model; The method of using the SLAM algorithm to model the scene to obtain a global map model and obtain the external parameters of the SLAM camera includes: Using a SLAM algorithm to obtain a point cloud reconstruction result of the scene, a SLAM camera pose set, and a reconstructed image frame set; wherein the SLAM camera pose set includes pose parameters of each SLAM camera, which are used to represent the installation position of each SLAM camera in the global map model; Constructing a global map model based on the point cloud reconstruction results, and scanning and photographing the scene using a SLAM camera to obtain a set of reconstructed image frames consisting of multiple scene images; The FAST algorithm is used to sequentially obtain feature points in two adjacent scene images and obtain feature descriptors for the feature points. Based on the feature descriptors of the feature points, similar feature points in the two adjacent scene images are matched in sequence to construct a unified camera coordinate system with the camera coordinate system of the first scene image as a reference. The global coordinate system is obtained through conversion. The method of sequentially acquiring feature points in two adjacent scene images using the FAST algorithm includes: Calculate the FAST score of the feature points in each scene image, and judge a neighborhood with the target feature point as the center point. If there are multiple feature points in the neighborhood, judge the FAST score of each feature point. If the target feature point has the largest score among all the feature points in the neighborhood, retain it; otherwise, suppress the target feature point. If there is only one feature point in the neighborhood, retain it.

8. The device according to claim 7, characterized in that Also includes: The target tracking unit uses target tracking technology to track the target in real time in the global map model, and draws the real-time position trajectory and activity area heat map of the target.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images