Automatic target reporting method and system based on three-dimensional imaging

Abstract

The present invention relates to the field of image processing technology, and provides an automatic target reporting method and system based on three-dimensional imaging, including: acquiring images collected by multiple cameras, and correcting the images to obtain corrected images; judging whether to add an arrow based on the corrected images, and if so, calculating the disparity map between the corrected images of different cameras, and intercepting the disparity information of the arrow; obtaining the point cloud information of the arrow based on the disparity information of the arrow, and fitting the arrow; fusing and splicing the arrow point cloud and the target surface point cloud, and then obtaining the arrow impact point in the three-dimensional space according to the spatial position relationship between the two, and determining the position of the arrow impact point in the two-dimensional image in combination with the internal parameters of the camera, and performing ring number determination. Not only can the number of rings be accurately identified, but also the problem of arrow impact point occlusion is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an automatic target reporting method and system based on three-dimensional imaging. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] At present, archery competitions generally use manual target reporting. The existing automatic target reporting systems mainly include photoelectric-based automatic target reporting, acoustic-electric-based automatic target reporting, radar detection-based automatic target reporting, and circuit-based automatic target reporting. These automatic target reporting systems mainly use hardware equipment to modify the target surface. Although the number of rings can be accurately identified, the equipment cost is high and the installation is cumbersome and not suitable for promotion.

[0004] In order to solve the high equipment cost, an automatic target reporting system based on image processing has emerged. This method reduces the cost of hardware while ensuring accurate target reporting. This method generally uses a camera to shoot the archery target surface and intercepts key frames for processing and recognition. When there is no occlusion between arrows, this method can accurately identify the number of rings, but if there is occlusion, this method is difficult to accurately identify the number of rings. Summary of the invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides an automatic target reporting method and system based on three-dimensional imaging. By using multiple cameras to simultaneously capture an image of a target surface, the target surface and the arrow are imaged in three dimensions, and the accurate arrow impact point is obtained in three dimensions and subsequent ring number recognition is performed. Not only can the ring number be accurately recognized, but also the problem of arrow impact point occlusion is solved.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides an automatic target registration method based on three-dimensional imaging, which comprises:

[0008] Acquire images captured by multiple cameras, and correct the images to obtain corrected images;

[0009] Based on the corrected image, determine whether to add an arrow. If so, calculate the disparity map between the corrected images of different cameras and extract the disparity information of the arrow.

[0010] Based on the disparity information of the arrow, the point cloud information of the arrow is obtained, and the arrow is fitted;

[0011] The arrow support point cloud and the target surface point cloud are fused and spliced, and then the arrow impact point in the three-dimensional space is obtained according to the spatial position relationship between the two. Combined with the camera's internal parameters, the position of the arrow impact point in the two-dimensional image is determined to determine the number of rings.

[0012] Furthermore, it also includes: after fusing and splicing the arrow point cloud and the target point cloud, using the Poisson surface reconstruction method to convert the point cloud image into a stereoscopic three-dimensional image.

[0013] Furthermore, the specific steps of the correction are:

[0014] Based on the external parameters of each camera, the images collected by each camera are aligned epipolarly;

[0015] The epipolar aligned images are dedistorted according to the intrinsic parameters of each camera.

[0016] Furthermore, the target surface point cloud acquisition process is as follows:

[0017] Acquire target surface images captured by multiple cameras, and correct the target surface images to obtain corrected target surface images;

[0018] Calculate the disparity map between the corrected target surface images of different cameras to obtain the target surface disparity map;

[0019] According to the target surface disparity map, the three-dimensional point cloud of the target surface is obtained.

[0020] Furthermore, the method for determining whether to add a new arrow is as follows:

[0021] The pixel values ​​and gradient amplitudes of several frames of images are used as background models;

[0022] Based on the background model, it is determined whether there is a foreground in the corrected image. If so, it is determined to be a newly added arrow, and the newly added arrow is presented as a foreground image.

[0023] Furthermore, the arrow fitting method is:

[0024] Based on the point cloud information of the arrow, the axis of the cylinder is fitted by minimizing the error equation;

[0025] Determine the width of the edges on both sides of the arrow in the acquired image and fit the diameter of the cylinder in three-dimensional space;

[0026] According to the axis and diameter of the cylinder, cylinder fitting is performed to obtain the arrow.

[0027] Furthermore, the steps of determining the number of rings are:

[0028] Acquire the ring line of the target surface image;

[0029] For each loop, determine its positional relationship with the center point of the arrow point, and determine the number of loops based on the positional relationship. At the same time, if the center point of the arrow point is outside the loop, determine whether verification is required. If necessary, intercept a square region of interest centered on the center point of the arrow point, and perform sub-pixel edge detection based on grayscale distance on the square region of interest to determine whether the arrow point is connected to the loop. If so, the arrow point is on the loop.

[0030] A second aspect of the present invention provides an automatic target marking system based on three-dimensional imaging, comprising:

[0031] A correction module is configured to: obtain images captured by multiple cameras, and correct the images to obtain corrected images;

[0032] A disparity map calculation module is configured to: determine whether to add an arrow based on the corrected image, and if so, calculate the disparity map between the corrected images of different cameras and extract the disparity information of the arrow;

[0033] An arrow fitting module is configured to: obtain point cloud information of the arrow based on the parallax information of the arrow, and fit the arrow;

[0034] The ring number determination module is configured to: fuse and splice the arrow support point cloud and the target surface point cloud, and then obtain the arrow impact point in the three-dimensional space based on the spatial position relationship between the two, and determine the position of the arrow impact point in the two-dimensional image in combination with the camera's internal parameters to perform ring number determination.

[0035] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the automatic target registration method based on three-dimensional imaging as described above.

[0036] The fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the automatic target registration method based on three-dimensional imaging as described above are implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The invention provides an automatic target reporting method based on three-dimensional imaging, which does not require modification of the target, effectively reduces the cost, and improves the applicability.

[0039] The present invention provides an automatic target reporting method based on three-dimensional imaging, which can effectively reduce the occlusion between arrows through shooting with multiple cameras. Even if the arrow impact point is blocked, the arrow with the blocked arrow impact point can be restored through the three-dimensional point cloud information, thereby obtaining accurate ring number information.

[0040] The present invention provides an automatic target reporting method based on three-dimensional imaging, which can present a three-dimensional image for a referee to check from multiple angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0042] Figure 1 is a flow chart of an automatic target reporting method based on three-dimensional imaging according to Embodiment 1 of the present invention;

[0043] Figure 2 Schematic diagram of the placement of the camera when capturing images according to the first embodiment of the present invention;

[0044] Figure 3 is a network structure diagram based on an adaptive correlation cascade recursive network according to the first embodiment of the present invention;

[0045] Figure 4 The first embodiment of the present invention converts the disparity map into a geometric relationship diagram of a point cloud;

[0046] Figure 5 is a schematic diagram of a space cylinder in Embodiment 1 of the present invention;

[0047] Figure 6 This is a schematic diagram of the position of the space arrow impact point of the first embodiment of the present invention;

[0048] Figure 7 It is a schematic diagram of an ideal edge model of a two-dimensional image according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0050] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0052] Embodiment 1

[0053] This embodiment provides an automatic target reporting method based on three-dimensional imaging, such as Figure 1 As shown, the specific steps include:

[0054] Step 1: Multiple cameras (two or more) are kept in a fixed position, firstly capturing the calibration plate image before the archery competition, and then capturing the target surface and arrow images during the archery process.

[0055] Camera according to Figure 2 The two cameras (camera 1 and camera 2) are placed in a way that two cameras (camera 1 and camera 2) are used to collect images. The two cameras are symmetrical about the mid-vertical plane of the target surface. The position of the camera is fixed during the collection process. When collecting the calibration plate image, the angle and position of the calibration plate need to be moved; 20 to 25 calibration plate images of black and white checkerboards are obtained.

[0056] Step 2: Camera calibration: Calibrate the camera according to the calibration plate image collected in step 1 to obtain the camera's intrinsic and extrinsic parameters.

[0057] Through camera calibration, the binocular system composed of camera 1 and camera 2 is converted into an ideal binocular system, that is, the two cameras are in an ideal parallel state; the calibration plate image of the black and white chessboard collected in step (1) is converted into the image according to the conversion relationship between the world coordinate system and the pixel coordinate system using Zhang Zhengyou's calibration method:

[0058]

[0059] Among them, M 1 is the camera intrinsic parameter matrix, M 2 is the camera external parameter matrix, s is the scaling factor, (u,v) is a point in the pixel coordinate system, and (X,Y,Z) is a point in the world coordinate system. According to the above formula, the intrinsic parameters of camera 1 and camera 2 (the principal point (u 0 ,v 0 ), focal length f, distortion coefficient (k 1 ,k 2 ,p 1 ,p 2 ,k 3 )) and camera extrinsics (rotation matrix R and translation matrix T between the two cameras).

[0060] Step 3: Image correction: According to the external parameters (rotation matrix) and internal parameters of each camera, the image is corrected to obtain the corrected image.

[0061] According to the internal parameters obtained in step 2, the captured image is distorted and the left and right images are aligned at the same time.

[0062] According to the camera extrinsics obtained in step 2, Bouguet is used to align the images collected by each camera. First, the rotation matrix is ​​decomposed into R = R 1 R 2 ; The camera coordinate system of camera 1 is rotated along the rotation matrix R 1 , the camera coordinate system of camera 2 is rotated along the matrix Then construct the correction matrix R rect =(r 1 ';r 2 ';r 3 '),in The final rotation matrix of camera 1 is: R c1 =R rect *R 1 , the rotation matrix of camera 2 is R c2 =R rect *R 2 ; After rotating the camera coordinate system according to the rotation matrix, the two cameras are transformed into an ideal binocular system.

[0063] According to the distortion coefficient and camera intrinsic parameters, the image after epipolar alignment is distorted. The specific steps are as follows:

[0064] 1) Convert the image pixel coordinate system to the camera coordinate system:

[0065]

[0066] Among them, (x c ,y c ) is a point in the camera coordinate system;

[0067] 2) Remove distortion in the camera coordinate system:

[0068]

[0069] in, (x c_rec t,y c_rec t) is the coordinate in the camera coordinate system after distortion removal;

[0070] 3) Convert the camera coordinate system back to the pixel coordinate system:

[0071]

[0072] Among them, (x p ,y p ) is the pixel coordinate after the distortion is removed.

[0073] Step 4, obtaining the arrow: Based on the corrected image, determine whether a new arrow is added. If so, calculate the disparity map between the corrected images of different cameras and extract the disparity information of the arrow.

[0074] The target detection algorithm based on pixel adaptive segmentation obtains the position information of the newly added arrow, and then proceeds to step 501, and finally extracts the disparity information of the arrow according to the position of the newly added arrow.

[0075] The target detection algorithm based on pixel adaptive segmentation has the following main steps:

[0076] (1) Use the pixel values ​​and gradient amplitudes of the first N frames of images obtained during the game as the background model B(x i ):

[0077] B(x i )={B 1 (x i ),...,B k (x i ),...,B N (x i )}

[0078] in, Background pixel value, Background gradient value, x i refers to the position of each pixel.

[0079] (2) Foreground detection: Based on the background model, determine whether there is a foreground in the corrected image, that is, for the input new image (corrected image) I(x i ) makes the following judgment:

[0080]

[0081] Among them, F(x i )=1 indicates the foreground, dist(I(x i ),B k (x i )) represents the distance, num_min represents the distance less than R(x i ), R(x i ) is the classification threshold;

[0082] The distance is expressed as:

[0083]

[0084] in, Represents the average gradient of the last observed image

[0085] (3) Determine the classification threshold R(x i ):

[0086]

[0087] Among them, R inc / dec The change in threshold, pre-specified; R scale Control background complexity for threshold adjustment, pre-specified; Current image and background B k (x i )The average of the minimum distances.

[0088] (4) Background update rate: Randomly select the samples that need to be replaced in the background model and replace them with the current image. The update rate is:

[0089]

[0090] Among them, T inc , T dec The increase or decrease of the update rate needs to be specified in advance; when the update rate meets T lower ≤T(x i )≤T upper When the sample B in the background model k (x i ) is updated, where T lower , T upper is a pre-specified threshold.

[0091] According to the above method, it is determined whether there is a new arrow (that is, if there is a foreground in the corrected image, it is determined to be a new arrow); if so, the new arrow will be presented as a foreground image, and the corresponding original image will be input into the CREStereo network in step 501 to obtain a disparity map, and then the new arrow will be cut out from the disparity image according to the foreground image of the new arrow to obtain the disparity information of the arrow; if there is no new arrow, continue to judge the next image.

[0092] Step 5: If the judgment result of step 4 is that no new arrows are added, the collected image is the target surface image. If the target surface point cloud has not been obtained, steps 501 and 502 are executed to obtain the target surface point cloud. The target surface image can also be obtained before the start of the game.

[0093] Step 501, obtaining disparity: obtaining a disparity map of the target surface image that has gone through steps 2 and 3;

[0094] The target surface image processed in step 3 is input into the CREStereo network (based on the adaptive correlation cascade recursive network) to obtain the corresponding target surface disparity map; the pixel value of the disparity map is the disparity value corresponding to each point; the CREStereo network uses 25,000 pairs of images and 25,000 pairs of disparity images for training. The network structure diagram is shown in the figure below: Figure 3As shown in the paper Practical Stereo Matching via Cascaded Recurrent Network with AdaptiveCorrelation published in 2022 at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), RUM is a recursive update module that calculates and updates the disparity; offset is an offset calculation module that determines the search window range of the disparity; its loss function is defined as:

[0095]

[0096] Among them, s is the level of the feature pyramid s∈{1 / 16,1 / 8,1 / 4}, n is the number of prediction iterations, d gt is the true disparity value, f i s is the predicted value of the s-th layer feature at the i-th iteration, μ s f i s The set of ||d gt -μ s (f i s )|| 1 is the L1 distance between the two.

[0097] Step 502, obtaining the three-dimensional point cloud of the target surface: according to the target surface disparity map and the camera intrinsic parameter matrix obtained by calibration, and the principle of camera pinhole imaging, the three-dimensional point cloud of the target surface is obtained.

[0098] According to the geometric relationship between the camera intrinsic parameters and stereo imaging obtained in step 2 (such as Figure 4 ) to obtain the target surface point cloud; the image collected passes through step 3. After a point P in the space is in the same row of image 1 and image 2 (the image corresponding to camera 1 is image 1, and the image corresponding to camera 2 is image 2), and its coordinates are (x 1 ,y)、(x 2 ,y); from the trigonometric relationship:

[0099]

[0100] Where (X, Y, Z) is a point in three-dimensional space, f is the focal length, and Tx is the baseline (i.e., the distance between the optical centers of the two cameras); let d = x 1 -x 2 , then d is the disparity between image 1 and image 2, and the value is obtained in step 501.

[0101] Step 6: Arrow fitting: Get the three-dimensional point cloud information based on the parallax information of the arrow, and then perform cylindrical fitting to get the complete arrow.

[0102] According to the camera intrinsic parameters obtained in step 2 and the triangular geometric relationship in step 502, the disparity map of the newly added arrow intercepted in step 4 is converted into a point cloud; after obtaining the point cloud information of the arrow, the arrow needs to be fitted. The main reasons for fitting are as follows: 1) The arrow's impact point is blocked, and its point cloud is not in contact with the contact point; 2) The arrow is broken when the disparity is obtained, and the arrow is not displayed completely; 3) Since the image of the arrow does not capture all the faces of the arrow, the required point cloud is only a part of the arrow. According to the characteristic that the distance from the point to the axis on the cylindrical surface is equal (such as Figure 5 ) Fit the existing arrow point cloud using the least squares method; the specific steps are as follows:

[0103] 1) From the geometric properties of the cylinder, we know that the distance from any point P (x, y, z) on the cylindrical surface to the axis is equal, that is:

[0104] (xx 0 ) 2 +(yy 0 ) 2 +(zz 0 ) 2 -[a(xx 0 )+b(yy 0 )+c(zz 0 )] 2 =r 2

[0105] Among them, (a, b, c) axis direction vector, P 0 (x 0 ,y 0 ,z 0 ) a point on the axis;

[0106] 2) Establish the error equation

[0107] f=(xx 0 ) 2 +(yy 0 ) 2 +(zz 0 ) 2 -[a(xx 0 )+b(yy 0 )+c(zz 0 )] 2 -r 2

[0108] The parameters of the fitted cylinder, i.e., the axis and the radius, can be obtained by solving the minimum value of f. However, since the radius of the arrow tail is larger as it is closer to the camera, while the radius of the arrowhead is smaller, there is a deviation in the radius obtained by the least squares method. Therefore, for the first arrow, the width of the two edges on both sides of the arrowhead is determined based on the foreground image of the arrow, and is converted into the diameter 2r of the fitted arrow in the three-dimensional space using the formula in step 502, so as to obtain accurate arrow information.

[0109] That is, based on the point cloud information of the arrow, the axis of the cylinder is fitted by minimizing the error equation; the width of the two edges on both sides of the arrowhead in the obtained image is determined (i.e., the width of the two edges on both sides of the arrowhead determined based on the foreground image of the arrow for the first arrow), and the diameter of the cylinder in the three-dimensional space is fitted; according to the axis and diameter of the cylinder, cylinder fitting is performed to obtain the arrow.

[0110] Step 7: Obtain the arrow impact point: The point cloud of the arrow and the point cloud of the target surface are fused and spliced, and then the intersection point of the two in space is obtained according to their spatial position relationship, that is, the arrow impact point in the three-dimensional space.

[0111] The point cloud of the newly obtained arrow and the point cloud of the target surface are fused and spliced to obtain a complete point cloud image (such as Figure 6 ), and the arrow impact point can be obtained according to the obtained point cloud of the target surface and the information of the fitted arrow, that is, the points on the target surface whose distance from the axis is less than the radius belong to the arrow impact point area; according to the axis direction (a, b, c) and a point P 0 (x 0 , y 0 , z 0 ) obtained in step 6, the axis can be expressed as:

[0112]

[0113] For any point P c (x c , y c , z c ) on the target surface, the distance to the axis can be expressed as:

[0114]

[0115] When dis < r, the points are the points included in the arrow impact point area, and the arrow impact point area is as shown in Figure 6 .

[0116] Step 8, ring number detection: First, determine the position of the arrow impact point in the two-dimensional image based on the camera internal parameter matrix and the point cloud information of the arrow impact point. Then determine the number of rings: 1) According to the corrected target surface image, perform ellipse fitting to obtain the ring line information, and perform preliminary detection of the number of rings based on the positional relationship between the arrow impact point and the ring line. When the arrow impact point is far from the ring line (not within the set threshold range), directly output the number of detected rings; 2) When the arrow impact point is close to the ring line (within the threshold setting range), due to the error of the ring line fitting, the result of the preliminary detection needs to be accurately verified, and finally the number of rings is obtained after accurate verification.

[0117] First, the three-dimensional arrow impact point of the newly added arrow in step 7 is used in reverse according to the internal reference obtained in step 2 and the formula in step 502 to obtain the arrow impact point area in the two-dimensional image, and then the ring number is determined.

[0118] Get the ring lines of the target surface image; for each ring line, determine the positional relationship with the center point of the arrow impact point, and determine the number of rings based on the positional relationship. At the same time, if the center point of the arrow impact point is outside the ring line, determine whether verification is required. If necessary, intercept the square region of interest centered on the center point of the arrow impact point, perform sub-pixel edge detection based on grayscale distance on the square region of interest, and determine whether the arrow impact point is connected to the ring line. If so, the arrow impact point is on the ring line. Specifically, the number of rings is determined in the following two steps:

[0119] 1) According to the corrected target surface image, color extraction is performed on each ring area to obtain the corresponding ring area contour. Ellipse fitting is performed on each ring area contour to obtain two ellipses, large and small. The small ellipse is the inner ring line of the ring area (i.e., the ring line with a high number of rings), and the large ellipse is the outer ring line of the ring area (the ring line with a low number of rings). Each ring line can be expressed as:

[0120]

[0121] Among them, (x center ,y center ) is the center point of each loop (the center point of each loop will have some slight differences after being fitted into an ellipse, and the center point corresponding to each loop will be used in the subsequent judgment), θ n is the rotation angle of the nth loop, a n is the major semi-axis of the nth loop, b n is the minor semi-axis of the nth loop; use the center point of the arrow point (x arrow ,y arrow ) and each loop line to determine the position, namely:

[0122]

[0123] Among them, Δ n <1 means (x arrow ,yarrow )In the loop, let n = n + 1 and calculate Δ again n ; Δ n =1 means (x arrow ,y arrow ) on this loop, the number of loops at this time is n; Δ n >1 means (x arrow ,y arrow ) is outside this loop, and we need to judge (x arrow ,y arrow ) is close to this loop line, the basis for judgment is: Where D represents the distance between two adjacent loops. n When it is not within the above judgment range (x arrow ,y arrow ) is not close to the loop line, and the number of loops is determined to be n-1. n When within the above judgment range, the results of the preliminary test need to be accurately verified.

[0124] 2) When Δ n When the threshold condition is met, the intercept is (x arrow ,y arrow ) is the center of a D×D square region of interest S; sub-pixel edge detection based on grayscale distance is performed on the region; if the arrow point is connected to the ring line, the number of rings is n, otherwise it is n-1; the principle of sub-pixel edge detection based on grayscale distance is:

[0125] For the image G(i,j), the k-level grayscale distance of the target area S is Defined as:

[0126]

[0127] Where (i, j) is the coordinate of the pixel point, and n is the total number of pixels in the S region. The ideal edge model of a two-dimensional image is as follows: Figure 7 , is the gray value h 1 and gray value h 2 Its normalized model consists of the grayscale values ​​h on both sides of the edge. 1 and h 2 , edge position ρ and edge direction θ, which are jointly determined by the four parameters and can be expressed as: Assume ω 1 and ω 2 are grayscale values ​​h 1 and h 2 The ratio of pixels to the total number of edge pixels, and ω 1 +ω 2 =1, then the grayscale distances of the first three levels are: This is equivalent to finding the weighted sum of the grayscale values ​​of each pixel in the unit circle: So we can get the edge position ρ and edge direction θ:

[0128] ρ=cosδ

[0129]

[0130]

[0131] Among them, (x 0 ,y 0 ) is the grayscale centroid coordinate of the unit circle. By calculating the four parameters of the normalized model, the two-dimensional edge sub-pixel coordinate (x κ ,y κ )for:

[0132]

[0133] Step 9: Surface reconstruction: After fusing the target point cloud and the arrow point cloud, perform surface reconstruction to obtain three-dimensional object information.

[0134] After the arrow point cloud and the target point cloud are fused, a complete point cloud image is obtained. The Poisson surface reconstruction method is used to convert the point cloud image into a stereoscopic three-dimensional image with a complete surface. The specific method is as follows:

[0135] 1) Define the octree: For the input point cloud data S, construct an adaptive octree so that each point sample falls into a leaf node with a depth of D (the maximum depth of the octree);

[0136] 2) Set the function space: set the node o function for each node of the octree Among them, oc represents the center of the node, ow represents the width of the node, and F is the basis function; the basis function is the nth convolution of the box filter B(t) with itself, that is:

[0137] F(x,y,z)≡(B(x)B(y)B(z)) *n

[0138]

[0139] Among them, x, y and z are three coordinates in three-dimensional space.

[0140] 3) Calculate the vector field: Use trilinear interpolation to linearly interpolate the eight adjacent nodes of the current node. The approximate model of the vector field is:

[0141]

[0142] Among them, s is the K neighboring area of ​​any point in the point cloud data S, α o,sis the weight of linear interpolation, NgbrD(s) is the 8 nodes with depth D in the K neighboring area of ​​node o, is the vertex normal vector of o;

[0143] 4) Construct Poisson's equation: According to the vector field, a Poisson's equation can be given as: is a vector field, which refers to the approximate model of the vector field above Solve the Poisson equation;

[0144] 5) According to the solution of 4), extract the isosurfaces and splice the obtained surfaces to obtain a complete surface model.

[0145] The present invention uses a method of three-dimensional point cloud information to identify the number of rings, which has the following advantages over the prior art:

[0146] 1) No need to modify the target, which effectively reduces the cost and improves the applicability;

[0147] 2) Shooting with multiple cameras can effectively reduce the occlusion between arrows. Even if the arrow impact point is blocked, the arrow with the blocked impact point can be restored through the 3D point cloud information to obtain accurate ring number information;

[0148] 3) It can present three-dimensional images for referees to check from multiple angles.

[0149] Embodiment 2

[0150] This embodiment provides an automatic target marking system based on three-dimensional imaging, which specifically includes the following modules:

[0151] A correction module is configured to: obtain images captured by multiple cameras, and correct the images to obtain corrected images;

[0152] A disparity map calculation module is configured to: determine whether to add an arrow based on the corrected image, and if so, calculate the disparity map between the corrected images of different cameras and extract the disparity information of the arrow;

[0153] An arrow fitting module is configured to: obtain point cloud information of the arrow based on the parallax information of the arrow, and fit the arrow;

[0154] The ring number determination module is configured to: fuse and splice the arrow support point cloud and the target surface point cloud, and then obtain the arrow impact point in the three-dimensional space based on the spatial position relationship between the two, and determine the position of the arrow impact point in the two-dimensional image in combination with the camera's internal parameters to perform ring number determination.

[0155] The three-dimensional reconstruction module is configured to: fuse and splice the arrow point cloud and the target point cloud, and then use the Poisson surface reconstruction method to convert the point cloud image into a stereoscopic three-dimensional image.

[0156] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0157] Embodiment 3

[0158] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the automatic target registration method based on three-dimensional imaging as described in the first embodiment are implemented.

[0159] Embodiment 4

[0160] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the automatic target registration method based on three-dimensional imaging as described in the first embodiment are implemented.

[0161] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0162] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1One or more processes and / or blocks Figure 1 The functions specified in one or more blocks.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 One or more processes and / or blocks Figure 1 The steps of the functions specified in one or more blocks.

[0165] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc.

[0166] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automatic target reporting method based on three-dimensional imaging, It is characterized in that include: Acquire images captured by multiple cameras, and correct the images to obtain corrected images; Based on the corrected image, determine whether to add an arrow. If so, calculate the disparity map between the corrected images of different cameras and extract the disparity information of the arrow. Based on the disparity information of the arrow, the point cloud information of the arrow is obtained, and the arrow is fitted; The target surface images collected by multiple cameras are acquired, and the target surface images are corrected to obtain the corrected target surface images; the disparity maps between the corrected target surface images of different cameras are calculated to obtain the target surface disparity map; according to the target surface disparity map, the three-dimensional point cloud of the target surface is obtained; the arrow support point cloud and the target surface point cloud are fused and spliced, and then the arrow impact point in the three-dimensional space is obtained according to the spatial position relationship between the two, and the position of the arrow impact point in the two-dimensional image is determined in combination with the internal parameters of the camera, and the number of rings is determined.

2. The automatic target reporting method based on three-dimensional imaging as claimed in claim 1, It is characterized in that Also includes: After the arrow point cloud and the target point cloud are fused and spliced, the point cloud image is converted into a stereoscopic three-dimensional image using the Poisson surface reconstruction method.

3. The automatic target reporting method based on three-dimensional imaging as claimed in claim 1, It is characterized in that The specific steps of the correction are: Based on the external parameters of each camera, the images collected by each camera are aligned epipolarly; The epipolar aligned images are dedistorted according to the intrinsic parameters of each camera.

4. The automatic target reporting method based on three-dimensional imaging as claimed in claim 1, It is characterized in that The method for determining whether to add an arrow is as follows: The pixel values ​​and gradient amplitudes of several frames of images are used as background models; Based on the background model, it is determined whether there is a foreground in the corrected image. If so, it is determined to be a newly added arrow, and the newly added arrow is presented as a foreground image.

5. The automatic target reporting method based on three-dimensional imaging as claimed in claim 1, It is characterized in that The arrow fitting method is: Based on the point cloud information of the arrow, the axis of the cylinder is fitted by minimizing the error equation; Determine the width of the edges on both sides of the arrow in the acquired image and fit the diameter of the cylinder in three-dimensional space; According to the axis and diameter of the cylinder, cylinder fitting is performed to obtain the arrow.

6. The automatic target reporting method based on three-dimensional imaging as claimed in claim 1, It is characterized in that The steps of determining the number of rings are: Acquire the ring line of the target surface image; For each loop, determine its positional relationship with the center point of the arrow point, and determine the number of loops based on the positional relationship. At the same time, if the center point of the arrow point is outside the loop, determine whether verification is required. If necessary, intercept a square region of interest centered on the center point of the arrow point, and perform sub-pixel edge detection based on grayscale distance on the square region of interest to determine whether the arrow point is connected to the loop. If so, the arrow point is on the loop.

7. An automatic target reporting system based on three-dimensional imaging, It is characterized in that include: A correction module is configured to: obtain images captured by multiple cameras, and correct the images to obtain corrected images; A disparity map calculation module is configured to: determine whether to add an arrow based on the corrected image, and if so, calculate the disparity map between the corrected images of different cameras and extract the disparity information of the arrow; An arrow fitting module is configured to: obtain point cloud information of the arrow based on the parallax information of the arrow, and fit the arrow; The ring number determination module is configured to: obtain target surface images collected by multiple cameras, and correct the target surface images to obtain corrected target surface images; calculate the disparity map between the corrected target surface images of different cameras to obtain the target surface disparity map; obtain the three-dimensional point cloud of the target surface according to the target surface disparity map; fuse and splice the arrow support point cloud and the target surface point cloud, and then obtain the arrow impact point in the three-dimensional space according to the spatial position relationship between the two, and determine the position of the arrow impact point in the two-dimensional image in combination with the camera's internal parameters to perform ring number determination.

8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the steps in the automatic target reporting method based on three-dimensional imaging as described in any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the automatic target registration method based on three-dimensional imaging as described in any one of claims 1-6 are implemented.

Images