Method, device and equipment for determining target based on multiple cameras

By conducting surveying, calibration and projection matrix calculation on multiple cameras, the problem of insufficient positioning accuracy and fault tolerance in multi-camera linkage system is solved, and the three-dimensional positioning of high-precision target objects and system fault tolerance are improved in complex environments.

CN118781207BActive Publication Date: 2024-12-24SHENZHEN FANGDU ELECTRONIC CO LTD

Patent Information

Application Number
CN202411269204.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-24
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

In multi-camera-linked monitoring systems, it is difficult for the prior art to maintain high accuracy throughout the monitoring area, especially in view angle occlusion and complex environments, and it is difficult to achieve the improvement of real-time target tracking and positioning accuracy.

Method used

By surveying and calibration of multiple cameras, the projection matrix of each camera is calculated, the two-dimensional coordinates of the target object are identified and back-projected into the three-dimensional space, and the three-dimensional coordinates of the target object are calculated using the projection vectors of different cameras to enhance the fault tolerance of the system and positioning accuracy.

Benefits of technology

It realizes high-precision target object positioning in complex environments, reduces errors, enhances the system's fault tolerance and reliability, and ensures that the target object can be accurately positioned even if a certain camera viewing angle is blocked.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781207B_ABST
    Figure CN118781207B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device and equipment for determining a target based on multiple cameras, wherein the method for determining a target based on multiple cameras includes: surveying and calibrating the multiple cameras respectively, calculating the coordinates of several calibration points of each camera, and calculating the mapping vector set of each calibration point of each camera in the actual three-dimensional space; obtaining the target object in the actual three-dimensional space identified by each camera, and extracting the two-dimensional coordinates of the target object in the image; back-projecting the identified two-dimensional coordinates into the three-dimensional space according to the projection matrix of the camera to obtain a projection vector; the projection vectors of different cameras intersect in the three-dimensional space, and calculating the three-dimensional coordinates of the target object through the intersection of different projection vectors. The technical solution of the present invention aims to improve positioning accuracy, enhance the fault tolerance of the system, solve the problem of visual angle occlusion, and support real-time target tracking in complex environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of camera monitoring technology, and in particular to a method, device and equipment for determining a target based on multiple cameras. Background Art

[0002] In monitoring systems, using camera calibration technology to correlate multiple camera images is an important means to improve monitoring efficiency and accuracy. In the calibration process of multiple camera linkage, geometric principles in computer vision are usually used, such as perspective projection models. These principles help determine the internal parameters (such as focal length, distortion, etc.) and external parameters (camera position and direction) of each camera, as well as the relative position relationship between cameras. The calibration results are usually more accurate near the selected feature points, but there may be large errors in other places, which may affect the accuracy of the linkage system in the entire monitoring area. Summary of the invention

[0003] The main purpose of the present invention is to provide a method, device and equipment for target determination based on multiple cameras, aiming to improve positioning accuracy, enhance the fault tolerance of the system, solve the problem of visual occlusion, and support real-time target tracking in complex environments.

[0004] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions.

[0005] In a first aspect, the present invention proposes a method for determining a target based on multiple cameras, comprising:

[0006] Conducting survey and calibration on the multiple cameras respectively, calculating the coordinates of a number of calibration points of each camera, and calculating a mapping vector set of each calibration point of each camera in the actual three-dimensional space;

[0007] Obtain the target object in the actual three-dimensional space recognized by each camera, and extract the two-dimensional coordinates of the target object in the image;

[0008] According to the projection matrix of the camera, the identified two-dimensional coordinates are back-projected into the three-dimensional space to obtain a projection vector;

[0009] The projection vectors of different cameras intersect in three-dimensional space, and the three-dimensional coordinates of the target object are calculated through the intersection of different projection vectors.

[0010] In combination with the first aspect, in a possible implementation manner, the step of surveying and calibrating the multiple cameras respectively, calculating the coordinates of a number of calibration points of each camera, and calculating a mapping vector set of each calibration point of each camera in the actual three-dimensional space includes:

[0011] Conducting survey and calibration on the multiple cameras respectively to determine their internal and external parameters;

[0012] Select several known calibration points, and calculate the projection matrix of the camera by projecting the calibration points in each camera's two-dimensional image. The projection matrix maps the three-dimensional space points to the camera's two-dimensional image, and obtains a mapping vector set of each calibration point in the three-dimensional space.

[0013] In combination with the first aspect, in a possible implementation, the internal parameters are focal length and principal point coordinates, and the external parameters are the position and / or direction of the camera in space.

[0014] In combination with the first aspect, in a possible implementation manner, the step of obtaining the target object in the actual three-dimensional space recognized by each camera and extracting the two-dimensional coordinates of the target object in the image includes:

[0015] Identify the target object in the image through image processing algorithms;

[0016] Record the 2D coordinates of the target object in each camera image.

[0017] In combination with the first aspect, in a possible implementation manner, the projection vectors of the different cameras intersect in the three-dimensional space, and the step of calculating the three-dimensional coordinates of the target object through the intersection of the different projection vectors includes:

[0018] Calculate the intersection of the projection vector of each camera in three-dimensional space;

[0019] The intersection point of the projection vectors is determined using a geometric algorithm.

[0020] In combination with the first aspect, in a possible implementation, in the step of using a geometric algorithm to determine the intersection of the projection vectors, if the projection vectors do not completely intersect, the closest intersection position is determined and the closest intersection position is used as the three-dimensional coordinates of the target object.

[0021] In a second aspect, a device for determining a target based on multiple cameras is provided, comprising:

[0022] A surveying and calibration unit, used to survey and calibrate the multiple cameras respectively, calculate the coordinates of a number of calibration points of each camera, and calculate a mapping vector set of each calibration point of each camera in the actual three-dimensional space;

[0023] An acquisition unit, used to acquire a target object in the actual three-dimensional space recognized by each camera, and extract the two-dimensional coordinates of the target object in the image;

[0024] A back-projection unit, used to back-project the identified two-dimensional coordinates into three-dimensional space according to the projection matrix of the camera to obtain a projection vector;

[0025] The three-dimensional positioning unit is used to calculate the three-dimensional coordinates of the target object through the intersection of different projection vectors according to the intersection of projection vectors of different cameras in three-dimensional space.

[0026] In a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the target determination method based on multiple cameras as described in the first aspect or any possible implementation method of the first aspect.

[0027] The technical solution of the present invention calculates the projection matrix of each camera through known calibration points, identifies the target object in the image and extracts the two-dimensional coordinates, back-projects the two-dimensional coordinates into the three-dimensional space through the projection matrix to obtain the projection vector, and calculates the three-dimensional coordinates of the target object through the intersection of the projection vectors of different cameras, thereby accurately determining the three-dimensional position of the target object, enhancing the fault tolerance of the system, and solving the problem of view occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0029] Figure 1 The present invention is a flowchart of a method for determining a target using multiple cameras according to an embodiment of the present invention.

[0030] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In response to the problems in the background technology, the embodiments of the present application adopt the following technical solutions.

[0033] First, in conjunction with reference Figure 1 As shown, Figure 1 The present invention provides a method for determining a target based on multiple cameras, which includes the following steps:

[0034] S10: surveying and calibrating the multiple cameras respectively, calculating the coordinates of a number of calibration points of each camera, and calculating a mapping vector set of each calibration point of each camera in the actual three-dimensional space.

[0035] In this step, multiple cameras need to be arranged to be set up around the reference target at different angles and positions. These cameras may cover different perspectives of the reference target or provide overlapping areas of multiple perspectives. The camera starts to collect images to form a set of images. Each camera may capture the image of the reference target in its field of view, while the area in the occluded part will not be captured. For each image captured by the camera, computer vision or image processing technology is needed to determine the position coordinates of the reference target in the image through target detection, feature matching, or other positioning algorithms. These coordinates are usually pixel coordinates or relative to the camera's coordinate system.

[0036] The purpose of calibration is to understand the intrinsic parameters (such as focal length, principal point coordinates) and extrinsic parameters (such as position, rotation matrix) of the camera, so as to calculate the projection matrix of the camera. Place several calibration points in a scene with known three-dimensional coordinates. For example, place calibration point 1, calibration point 2, calibration point 3, etc. on the floor of a room, and we know their precise coordinates in the room, such as (X1, Y1, Z1), etc. Use the three-dimensional coordinates of the calibration points and their two-dimensional coordinates in the camera image to calculate the projection matrix of the camera using an algorithm (such as the DLT algorithm). These matrices can map points in three-dimensional space to the two-dimensional plane of the image. After each camera is calibrated, back-project each calibration point to obtain the projection vector of the camera corresponding to the calibration point, which represents the direction of the ray starting from the camera. For example, suppose that camera A captures a calibration point at a known position (3, 2, 1) in the room, and its two-dimensional coordinates in the image are (100, 150). Through calibration, we can calculate the projection matrix of camera A, and we can map the two-dimensional coordinates of the calibration point back to a three-dimensional projection vector, which represents the ray starting from camera A and pointing in the direction of the calibration point.

[0037] S20: Acquire the target object in the actual three-dimensional space recognized by each camera, and extract the two-dimensional coordinates of the target object in the image.

[0038] In this step, the camera identifies the target object in the scene through image processing algorithms (such as edge detection, shape recognition, color recognition, etc.). For example, a red ball in a room can be distinguished by color. After identifying the target object, the two-dimensional coordinates (u, v) of the target object in the camera image are extracted. For example, assuming that the target object is a red ball, camera A recognizes the ball and determines that its position in the image is (120, 180). Camera B recognizes that it is at (150,170), and camera C recognizes that it is at (110, 200).

[0039] S30: According to the projection matrix of the camera, the two-dimensional coordinates of the identified target object are back-projected into the three-dimensional space to obtain a projection vector.

[0040] In this step, the 2D coordinates of the target object are back-projected into 3D space using the camera's projection matrix. This generates a projection vector (ray) from the camera's position that indicates the direction in which the target object may be. Assuming that the projection matrix of camera A is known, the 2D coordinates of the ball in camera A (120, 180) can be mapped to a projection vector in 3D space through back-projection, which may represent a ray from camera A pointing to a certain direction in the room. Cameras B and C also generate their own projection vectors.

[0041] S40: Intersecting the projection vectors of different cameras in the three-dimensional space, and calculating the three-dimensional coordinates of the target object through the intersection of these projection vectors.

[0042] In this step, ideally, the projection vectors of these different cameras will intersect at a certain point in the three-dimensional space. This intersection point is the position of the target object in the three-dimensional space. If the projection vectors do not completely intersect, we can use geometric algorithms (such as the least squares method) to find the best intersection point of these rays as the three-dimensional coordinates of the target object. Through the projection vectors of cameras A, B, and C, we can calculate the intersection point of these rays in the three-dimensional space. If the projection vector of camera A is (X1, Y1, Z1), that of camera B is (X2, Y2, Z2), and that of camera C is (X3, Y3, Z3), through calculation, we may get the intersection point (X, Y, Z) = (2.5, 3.1, 1.7). This point is the actual position of the red ball in the room.

[0043] Through the above steps, multiple cameras collaborate to complete the three-dimensional positioning of the target object. The multi-camera system can significantly improve the accuracy of the positioning of objects in three-dimensional space. A single camera can only obtain the two-dimensional projection information of the object and cannot accurately determine the depth of the object. Through the perspectives of multiple cameras, the system can accurately calculate the three-dimensional position of the object and reduce errors and uncertainties. Furthermore, the perspective of a single camera is limited, and the target object may not be seen due to occlusion. The multi-camera system can capture the scene from different angles. Even if the perspective of a camera is blocked, other cameras can still provide valid data to ensure the accurate positioning of the target object. The multi-camera system is more robust. Even if one or several cameras fail or data is lost, the system can still use the data of other cameras for three-dimensional reconstruction to maintain a high degree of accuracy. This redundant design increases the reliability of the system.

[0044] In combination with the first aspect, in a possible implementation manner, the step of surveying and calibrating the multiple cameras respectively, calculating the coordinates of a number of calibration points of each camera, and calculating a mapping vector set of each calibration point of each camera in the actual three-dimensional space includes the following steps:

[0045] S101. Survey and calibrate the multiple cameras respectively to determine their internal and external parameters.

[0046] In this step, the intrinsic parameters are the parameters that describe the internal characteristics of the camera, including focal length, principal point (usually the center of the image), and lens distortion coefficients. These parameters reflect the optical properties of the camera and determine how to map points in the three-dimensional world to the two-dimensional image plane. The external parameters describe the position and orientation (attitude) of the camera in three-dimensional space. It includes the rotation matrix (R) and displacement vector (T) of the camera, which represent the rotation and translation of the camera relative to the world coordinate system. Suppose we have a camera A, and its intrinsic parameters are obtained through the calibration process: focal length: f_x = 1000, f_y = 1000 (unit: pixel), principal point: c_x = 640, c_y = 480 (the center point of the image), distortion coefficient: usually a small value, such as k1 = -0.1, k2 = 0.01, and the external parameters may represent the rotation and displacement of the camera relative to a reference point in the room.

[0047] S102. Select several known calibration points, and calculate the projection matrix of the camera by projecting the calibration points in the two-dimensional image of each camera. The projection matrix maps the three-dimensional space points to the two-dimensional image of the camera, and obtains a mapping vector set of each calibration point in the three-dimensional space.

[0048] In this step, select some calibration points with known positions in the three-dimensional space. The three-dimensional coordinates of these points are known, such as (X1, Y1, Z1), (X2, Y2, Z2), etc. Project these three-dimensional calibration points onto the two-dimensional image plane of the camera to obtain their two-dimensional coordinates in the image (u1, v1), (u2, v2), etc. The projection matrix of the camera is calculated using the three-dimensional coordinates of the calibration points and the corresponding two-dimensional projection coordinates using linear algebra methods (such as the DLT algorithm). The projection matrix can map any point in the three-dimensional space to the two-dimensional image of the camera. For example, suppose we select three calibration points in the room, and their 3D coordinates are: Calibration point 1: P1 = (X1, Y1, Z1) = (1, 2, 3), calibration point 2: P2 = (X2, Y2, Z2) = (4, 5, 6), calibration point 3: P3 = (X3, Y3, Z3) = (7, 8, 9), and the projection coordinates of these points in the 2D image of camera A are: P1 -> (u1, v1) = (100, 200), P2 -> (u2, v2) = (150, 250), P3 -> (u3, v3) = (200, 300). With these data, the projection matrix M of camera A can be calculated. This is usually expressed as: M=K[R|T], where K is the intrinsic parameter matrix, R is the rotation matrix, and T is the displacement vector. The projection matrix M will be used to transform a 3D point (X, Y, Z) into a 2D point (u, v) in the image.

[0049] Using the projection matrix, the coordinates of each calibration point in the three-dimensional space are mapped to the two-dimensional image of the camera, and a mapping vector set is generated. Using the calculated projection matrix M, the three-dimensional coordinates (X, Y, Z) of each calibration point are converted to two-dimensional image coordinates (u, v). For each camera, the three-dimensional coordinates of the calibration point can be obtained by the projection matrix to obtain the corresponding two-dimensional coordinates, and then the projection vector of each calibration point in the three-dimensional space can be obtained by the back-projection operation. These vectors represent the direction from the camera through these calibration points. Through the projection matrix M of camera A, we project the three-dimensional coordinates (1, 2, 3) of the calibration point P1 onto the two-dimensional image and obtain (100, 200). Through back-projection, a projection vector starting from camera A and passing through point P1 can be generated. Similar operations are performed on other calibration points to obtain a set of projection vectors, which form a mapping vector set.

[0050] In combination with the first aspect, in a possible implementation, the internal parameters are focal length and principal point coordinates, and the external parameters are the position and / or direction of the camera in space.

[0051] Understandably, intrinsic parameters are parameters that describe the internal characteristics of the camera, including focal length, principal point (usually the center of the image), and lens distortion. These parameters reflect the optical properties of the camera and determine how to map points in the three-dimensional world to the two-dimensional image plane. Extrinsic parameters describe the position and orientation (attitude) of the camera in three-dimensional space.

[0052] In combination with the first aspect, in a possible implementation manner, the step of obtaining the target object in the actual three-dimensional space recognized by each camera and extracting the two-dimensional coordinates of the target object in the image includes the following steps:

[0053] S201, identifying a target object in an image through an image processing algorithm.

[0054] In this step, object recognition refers to the use of image processing algorithms to identify specific target objects from images captured by the camera. Image processing algorithms can detect and identify target objects based on specific features such as color, shape, texture, or edges. In order to improve recognition accuracy, images usually need to be preprocessed. This may include denoising, grayscale, histogram equalization, etc. In the preprocessed image, the algorithm will extract the characteristics of the target object, such as edges, corners, color distribution, etc.

[0055] S202: Record the two-dimensional coordinates of the target object in each camera image.

[0056] In this step, once the target object is identified in the image of each camera, its 2D coordinates (u, v) can be extracted. These coordinates usually represent the position of the target object on the image plane, such as the center point or the center of the bounding box. These coordinates are the pixel positions in the image and describe the position of the target object in the image. For the subsequent 3D reconstruction or positioning process, these 2D coordinates are very important basic data.

[0057] In combination with the first aspect, in a possible implementation manner, the projection vectors of the different cameras intersect in the three-dimensional space, and the step of calculating the three-dimensional coordinates of the target object through the intersection of the different projection vectors includes:

[0058] S401, performing intersection calculation on the projection vector of each camera in three-dimensional space.

[0059] In this step, after each camera recognizes the target object and records its two-dimensional coordinates, it can use the camera's projection matrix to back-project these two-dimensional coordinates into three-dimensional space, forming a ray extending from the camera position, which is the projection vector. In three-dimensional space, assuming that multiple cameras observe the same target object at the same time, each camera will generate a projection vector pointing to the target object. In theory, these projection vectors should intersect at the real position of the target object. Intersection calculation is to find the intersection of these rays in three-dimensional space. For example, suppose there are two cameras A and B, which observe a target object (such as a bird) at different angles. Camera A back-projects the position of the bird in the image (uA, vA) into three-dimensional space and obtains a projection vector VA; similarly, camera B will also generate a projection vector VB. The projection vector VA is from the position of camera A and points to the direction of the target object in three-dimensional space. The projection vector VB is from the position of camera B and points to the direction of the target object in three-dimensional space. In theory, VA and VB should intersect at the real position of the bird.

[0060] S402: Determine the intersection point of the projection vectors using a geometric algorithm.

[0061] In this step, due to the possible noise and measurement errors in reality, these projection vectors may not completely intersect in three-dimensional space. In order to determine the intersection of these vectors, geometric algorithms, such as least squares method, triangulation method, etc., can be used to find the best intersection between these vectors, that is, the point where their distance is the smallest. This intersection is the estimated three-dimensional coordinate of the target object. If the two projection vectors VA and VB do not intersect, we can calculate the minimum distance between them, and then find the midpoint of this distance as the estimated intersection. For multiple cameras, we can regard the intersection point of all projection vectors as an optimization problem and determine the best intersection point by minimizing the error (such as the sum of squared distances).

[0062] In combination with the first aspect, in a possible implementation, in the step of using a geometric algorithm to determine the intersection of the projection vectors, if the projection vectors do not completely intersect, the closest intersection position is determined and this point is used as the three-dimensional coordinate of the target object.

[0063] Understandably, first, find the closest point pair between the two projection vectors VA and VB (i.e., take a point on VA and VB respectively so that the distance between the two points is the smallest). The midpoint of these two closest points is what we consider to be the "closest intersection point". If there are multiple cameras, the "closest intersection point" of multiple projection vectors can be used as the 3D position of the target object. This is usually achieved by the least squares method, minimizing the sum of the distances of all projection vectors to the intersection point.

[0064] In a second aspect, based on the same inventive concept as in the aforementioned embodiment, an embodiment of the present application provides a target determination device based on multiple cameras, comprising:

[0065] A surveying and calibration unit, used to survey and calibrate the multiple cameras respectively, calculate the coordinates of a number of calibration points of each camera, and calculate a mapping vector set of each calibration point of each camera in the actual three-dimensional space;

[0066] An acquisition unit, used to acquire a target object in the actual three-dimensional space recognized by each camera, and extract the two-dimensional coordinates of the target object in the image;

[0067] A back-projection unit, used to back-project the identified two-dimensional coordinates into three-dimensional space according to the projection matrix of the camera to obtain a projection vector;

[0068] The three-dimensional positioning unit is used to calculate the three-dimensional coordinates of the target object through the intersection of different projection vectors according to the intersection of projection vectors of different cameras in three-dimensional space.

[0069] Those skilled in the art should understand that the division of the various modules in the embodiment is only a division of logical functions, and can be fully or partially integrated into one or more actual carriers in actual application, and these modules can be implemented entirely in the form of software called by the processing unit, or entirely in the form of hardware, or in the form of a combination of software and hardware. It should be noted that each module in the target determination device in this embodiment corresponds one-to-one to each step in the target determination method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned target determination method, which will not be repeated here.

[0070] According to a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is coupled to the processor, the memory is used to store computer program code, the computer program code comprising computer instructions, and when the processor reads the computer instructions from the memory, the electronic device executes the target determination method as described in the aforementioned embodiment.

[0071] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0072] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0073] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0074] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0075] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for determining a target based on multiple cameras, characterized in that: include: Conducting survey and calibration on the multiple cameras respectively, calculating the coordinates of a number of calibration points of each camera, and calculating a mapping vector set of each calibration point of each camera in the actual three-dimensional space; Obtain the target object in the actual three-dimensional space recognized by each camera, and extract the two-dimensional coordinates of the target object in the image; According to the projection matrix of the camera, the identified two-dimensional coordinates are back-projected into the three-dimensional space to obtain a projection vector; The projection vectors of different cameras intersect in three-dimensional space, and the three-dimensional coordinates of the target object are calculated through the intersection of different projection vectors; Calculate the intersection of the projection vector of each camera in three-dimensional space; A geometric algorithm is used to determine the intersection of the projection vectors. If the projection vectors do not completely intersect, the closest intersection position is determined, and the closest intersection position is used as the three-dimensional coordinate of the target object.

2. The method for determining a target based on multiple cameras according to claim 1, characterized in that: The steps of surveying and calibrating the multiple cameras respectively, calculating the coordinates of a number of calibration points of each camera, and calculating a mapping vector set of each calibration point of each camera in the actual three-dimensional space include: Conducting survey and calibration on the multiple cameras respectively to determine their internal and external parameters; Select several known calibration points, and calculate the projection matrix of the camera by projecting the calibration points in each camera's two-dimensional image. The projection matrix maps the three-dimensional space points to the camera's two-dimensional image, and obtains a mapping vector set of each calibration point in the three-dimensional space.

3. The method for determining a target based on multiple cameras according to claim 2, characterized in that: The internal parameters are focal length and principal point coordinates, and the external parameters are the position and / or direction of the camera in space.

4. The method for determining a target based on multiple cameras according to any one of claims 1 to 3, characterized in that: The step of obtaining the target object in the actual three-dimensional space recognized by each camera and extracting the two-dimensional coordinates of the target object in the image includes: Identify the target object in the image through image processing algorithms; Record the 2D coordinates of the target object in each camera image.

5. A target determination device based on multiple cameras, characterized in that: include: A surveying and calibration unit, used to survey and calibrate the multiple cameras respectively, calculate the coordinates of a number of calibration points of each camera, and calculate a mapping vector set of each calibration point of each camera in the actual three-dimensional space; An acquisition unit, used to acquire a target object in the actual three-dimensional space recognized by each camera, and extract the two-dimensional coordinates of the target object in the image; A back-projection unit, used to back-project the identified two-dimensional coordinates into three-dimensional space according to the projection matrix of the camera to obtain a projection vector; A three-dimensional positioning unit, used to calculate the three-dimensional coordinates of the target object through the intersection of different projection vectors according to the intersection of different projection vectors in three-dimensional space; Among them, the target determination device based on multiple cameras is used to calculate the intersection of the projection vector of each camera in three-dimensional space; use a geometric algorithm to determine the intersection of the projection vectors, and if the projection vectors do not completely intersect, determine the closest intersection position, and use the closest intersection position as the three-dimensional coordinates of the target object.

6. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is coupled to the processor, and the memory is used to store computer program code, wherein the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the target determination method based on multiple cameras as described in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Motion capture method, device and system

    CN110544273A

  • Camera calibration processing method and device, equipment and storage medium

    CN114445506A

  • Three-dimensional model rendering method and device

    CN115830202A

  • Reverse projection-based pipe fitting three-dimensional model reconstruction method

    CN116229017A

  • Optical element assembly process pose detection method based on orthogonal vision system and assembly method

    CN116758160A

Cited By

  • Visual inspection method for appearance of product printed by 3D printer

    CN120014177A

  • A method for visually inspecting the appearance of products printed by a 3D printer

    CN120014177B