An image recognition method for predicting separation trajectory of a test multi-body

Abstract

The application discloses an image recognition method for predicting a multi-body separation trajectory in a release test, which aims at capturing a single-plane separation trajectory by using a high-speed camera to shoot a multi-body release separation test in a wind tunnel, and then identifying the axial displacement, longitudinal displacement and pitch angle of the multi-body in the single-plane separation trajectory in real time through an image recognition technology, and belongs to the technical field of wind tunnel tests. The method comprises the following steps: an image calibration technology, an image recognition technology based on pixel edge detection, a multi-body recognition technology and a target follow-up technology. The test is carried out by using a release separation method, and the single-plane axial displacement, longitudinal displacement and pitch angle of a projectile body separation into two projectile bodies can be obtained, and the method has the capability of generating a trajectory pose in real time after the release test is completed.

Description

Technical Field

[0001] This invention relates to an image recognition method for predicting the trajectory of multibody separation in a deployment test, belonging to the field of experimental aerodynamics. Background Technology

[0002] Drop tests, also known as scaled-down model flight tests, are divided into atmospheric model drop tests and wind tunnel model drop tests. Wind tunnel model drop tests offer advantages in terms of test risk, cost, and efficiency. Wind tunnel model drop tests involve remotely controlling a missile model to achieve untethered six-degree-of-freedom free flight in a wind tunnel test section, providing a simulated test environment for scaled-down models that mimics the flight motion of a full-size real aircraft. The earliest prototype of this testing technology was the free-drop test of a model conducted at the Langley Research Center in the United States at specific wind speeds in a small, tiltable wind tunnel test section. In the early 1940s, Boeing's development of the B-47 aircraft, requiring internal weapon loading, necessitated separation characteristic drop tests. In the 1970s, Langley Research Center conducted drop and separation tests on the F-16, F-18, and B-1. Between 2006 and 2007, Cary and Bower, among others, conducted high-speed wind tunnel drop tests of the MK-82 bomb model in Boeing's PSWT wind tunnel. In 2001, Boeing conducted internal weapon compatibility drop tests.

[0003] Image recognition technology automatically processes a large amount of information based on the main features of an image to identify targets within it. The main methods of image recognition technology include feature extraction and feature matching. Feature extraction extracts the key features represented by the original image, while feature matching compares the image to be recognized with known target images to achieve the purpose of recognition. In the 1960s, IBM developed a text symbol recognition system based on template matching. In the 1980s, NEC of Japan developed an image recognition system that could detect and remove stains from steel plates. In the 1990s, the U.S. Army Research Projects Agency (DARPA) established the Image Understanding Research Program, conducting a series of computer vision-based studies and holding several image understanding competitions. Since the 21st century, deep learning technology, such as convolutional neural networks and recurrent neural networks, has been widely applied, further promoting the development of image recognition technology.

[0004] Traditional methods for acquiring projectile separation trajectories involve manually extracting each pixel from frame-by-frame images, then calculating the centroid and attitude angles of each projectile's contour in each frame. Furthermore, the initial calibration of the camera and separation plane also requires manual distance measurement, followed by rough calculation of camera parameters and the transformation relationship between the camera coordinate system and the world coordinate system. Therefore, there is an urgent need for a post-processing method that can acquire trajectory data in real time after the deployment test to save significant analysis time while maintaining identification accuracy. Summary of the Invention

[0005] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide an image recognition method for predicting the multi-body separation trajectory in a deployment test. This method meets the requirement of generating multi-body separation trajectories with higher accuracy and timeliness while the deployment separation test is completed, saving the time cost of manually analyzing and obtaining contour pixels point by point and calculating displacement and attitude frame by frame.

[0006] The technical solution of this invention is: an image recognition method for predicting the trajectory of multiple bodies in a deployment test, comprising:

[0007] 1) Before the test begins, locate the separation center surface of the catapult inside the core of the wind tunnel test section, and place the black and white checkerboard target plane at the separation center surface.

[0008] 2) After setting up the black and white checkerboard target surface, set up and fix the high-speed camera, and take multi-angle pictures of the black and white checkerboard inside the core of the wind tunnel test section for calibration.

[0009] 3) Based on the captured black and white checkerboard calibration images, identify the camera's intrinsic parameter matrix and the world coordinate system extrinsic parameter matrix parameters;

[0010] 4) Based on the identified parameters, establish the coordinate system matrix transformation relationship between the high-speed camera imaging plane and the calibrated black and white checkerboard target plane, and calculate the coordinate position of the black and white checkerboard relative to the high-speed camera through this matrix transformation relationship.

[0011] 5) Based on the coordinates of the black and white checkerboard target surface, a wind tunnel launch and separation test is conducted on the target surface. Using the existing ejection mechanism, the projectile is ejected and begins to separate. The separation process is captured by a high-frequency camera.

[0012] 6) Based on the captured images of the projectile's multi-body separation trajectory, read the high-speed camera's color images frame by frame, and convert each frame of the color image into a black and white image;

[0013] 7) Perform continuous scanning of the projectile outline in the black and white image;

[0014] 8) Determine whether the projectile outline pixels are continuous. If they are not continuous, replace the black pixels that are not connected to the projectile outline with white pixels and regenerate the replaced black and white image and return to step 6); if they are continuous, use multi-body recognition technology to identify the outline information of multiple moving projectiles.

[0015] 9) Based on the obtained contour information of the separated projectile, the target object servo technique is used to obtain the center of mass of the target object and the axis of the projectile.

[0016] 10) Based on the obtained center of mass and line segment of the projectile, calculate the axial displacement, longitudinal displacement and pitch angle of multiple projectiles to complete the real-time prediction of the multi-body separation trajectory during wind tunnel deployment test.

[0017] A high-speed camera was erected and fixed 1.5m away from the outside of the wind tunnel window.

[0018] The process of identifying the camera's intrinsic parameter matrix and world coordinate system extrinsic parameter matrix based on captured black and white checkerboard calibration images includes: after taking more than ten calibration photos, finding the cross intersections between black and white squares, and using the least squares method to identify the camera's intrinsic parameter matrix and world coordinate system extrinsic parameter matrix.

[0019]

[0020] Where A is the intrinsic parameter matrix of the camera, (u0, v0) is the principal point of the camera in the image coordinate system, a and b are the scale factors of the horizontal and vertical axes of the image, and r represents the perpendicularity of the image coordinate axes; B is the extrinsic parameter matrix that fixes the world coordinate system on a black and white checkerboard grid, R 3×3 It is a rotation matrix describing the camera orientation, T 3×1 It is a three-dimensional translation vector describing the position of the camera center.

[0021] The pixel edge detection method is used to continuously scan the outline of the projectile in the black and white image, scan all the pixels of each frame of the image, and scan all the square pixels in the nine-square grid form based on the pixels of this image.

[0022] The step of scanning all the nine-square grid-like square pixels based on the pixels of this image includes: assuming a black and white image has N rows and M columns of pixels, scanning each pixel in order from left to right and from top to bottom, and taking each pixel as the top left corner pixel of each nine-square grid, and scanning down 3 rows and right 3 columns from this pixel, scanning a total of 9 pixels for the nine-square grid. Therefore, an image with N rows and M columns of pixels can be scanned to produce (N-2)×(M-2) nine-square grid-like square pixels.

[0023] The target object tracking technology is used to obtain the centroid of the target object and the axis of the projectile. This means that the centroid coordinates of the object are calculated using the formula for calculating the centroid of a uniform density object. The midpoint of the head of the contour is found and connected to the centroid of the object as the axis. The axis is then connected into a line segment and marked with different colored pixels.

[0024] The determination of whether the projectile outline pixels are continuous includes: determining whether a black pixel is the projectile outline based on whether there are two connected black pixels in the horizontal, vertical, and diagonal directions within the nine-grid; replacing black pixels not connected to the projectile outline with white pixels, these black pixels not connected to the projectile are the image background pixels, thereby eliminating the interference of image background pixels; and then using pixel edge detection technology for iteration. If the projectile outline is finally obtained, the projectile outline pixels are considered continuous; otherwise, they are not continuous.

[0025] The method of using multi-body recognition technology to identify the contour information of multiple moving projectiles includes: identifying the projectile contour by using contour center marking during separation; after separation, if the center value of the original contour changes significantly, it is considered that the contour shape has changed, that is, the projectile itself has separated again, and the center value of the contour needs to be remarked, that is, the contour information of multiple moving projectiles is identified.

[0026] The advantages of this invention compared to the prior art are:

[0027] 1) The entire calibration process is simple, saving time and cost compared with other image recognition technologies used in wind tunnel testing during the early stage of wind tunnel test preparation;

[0028] 2) The displacement accuracy within the calibration surface can reach 0.1 mm, and the attitude angle positioning accuracy can reach 0.1°, which is far superior to the identification accuracy of other multi-view or monocular vision systems applied in wind tunnel tests;

[0029] 3) It can accurately identify multiple moving objects and give the physical center of mass position of the projectile. Existing technology can only identify a single moving object and cannot identify the center of mass position in real time.

[0030] 4) The automatic recognition and calculation processing method achieves real-time synchronization compared with existing image recognition methods used in wind tunnel tests, enabling timely and accurate acquisition of separation trajectories and attitude angles, which is beneficial for on-site data analysis. Attached Figure Description

[0031] Figure 1 Flowchart of an image recognition method for predicting the trajectory of multiple bodies in a deployment test.

[0032] Figure 2 A diagram illustrating the calibration of the target surface using a black and white checkerboard pattern.

[0033] Figure 3 Schematic diagram of Zhang Zhengyou's image calibration technology.

[0034] Figure 4 A schematic diagram illustrating the conditions for determining the continuity of pixel edges.

[0035] Figure 5 Schematic diagram of the application of multi-body recognition algorithm and target tracking algorithm.

[0036] Figure 6 A schematic diagram of the multibody separation trajectory, where (a) represents the axial displacement, (b) represents the longitudinal displacement, and (c) represents the pitch angle. Detailed Implementation

[0037] The following is in conjunction with the appendix Figure 1 ~Attached Figure 6 The specific embodiments of the present invention will be described in further detail below.

[0038] like Figure 1 As shown in the embodiment of this application, an image recognition method for predicting the trajectory of multiple bodies in a deployment test is provided, including the following steps:

[0039] 1) Before the test, locate the separation center plane of the catapult inside the core of the wind tunnel test section, and place the black and white checkerboard target plane near the separation center plane. Use a high-speed camera for the test to photograph the black and white checkerboard target from different angles and in different planes, such as... Figure 2 As shown;

[0040] 2) After setting up the black and white checkerboard target surface according to step 1), set up and fix a high-speed camera 1.5m away from the outside of the wind tunnel window, and take multi-angle pictures of the black and white checkerboard inside the wind tunnel test section for calibration.

[0041] 3) Following step 2), after taking more than ten calibration photos, such as... Figure 2 As shown, find the cross intersection between the black and white grids, and use the least squares method to identify the camera intrinsic parameter matrix and the world coordinate system extrinsic parameter matrix, as shown in Formula 1 and Formula 2.

[0042]

[0043] In Formula 1, A is the intrinsic parameter matrix of the camera, (u0, v0) is the principal point of the camera in the image coordinate system, a and b are the scale factors of the horizontal and vertical axes of the image, and r represents the perpendicularity of the image coordinate axes.

[0044]

[0045] In Formula 2, B represents the extrinsic parameter matrix that fixes the world coordinate system on a black and white checkerboard, and R... 3×3 It is a rotation matrix describing the camera orientation, T 3×1 It is a three-dimensional translation vector describing the position of the camera center.

[0046] 4) Based on the parameters identified in step 3), establish a mapping relationship between the high-speed camera's imaging plane and the calibrated checkerboard target plane, and calculate the coordinate position of the checkerboard target plane relative to the high-speed camera using this mapping relationship. Figure 3 The diagram illustrates the mapping relationship between the camera coordinate system, image coordinate system, and world coordinate system. The high-speed camera and the captured image are formed using the principle of pinhole imaging. The image coordinate system and the world coordinate system are related by rigid body transformation. Ultimately, the image coordinate system is obtained by correcting distortion in the pixel coordinate system, and then the world coordinate system is calibrated. The mapping relationship is shown in formulas 3, 4, and 5.

[0047]

[0048]

[0049]

[0050] World coordinate system: The world coordinate system defined in this invention is introduced to describe the position of an object in the real world. The general coordinate description is P. w =(X w Y w Z w ), the unit is m.

[0051] Camera coordinate system: A coordinate system established on the camera, defined to describe the position of objects from the camera's perspective. It serves as an intermediary between the world coordinate system and the image / pixel coordinate system. The coordinates are described as P... c =(X c Y c Z c The unit is meters (m). In Formula 4, f represents the focal length.

[0052] Image coordinate system: Introduced to describe the projection and transmission relationship of an object from the camera coordinate system to the image coordinate system during the imaging process, and to facilitate further obtaining the coordinates in the pixel coordinate system. The general coordinate system is described as P = (X, Y, 1), and the unit is m.

[0053] Pixel coordinate system: Introduced to describe the coordinates of image points on a digital image (photograph) after an object is imaged. It is the coordinate system where the information we actually read from the camera resides. The coordinates are (u, v), and the unit is pixels.

[0054] 5) Based on the coordinates of the black and white checkerboard target surface calculated in step 4), after the wind tunnel is started, a release and separation test is conducted on the target surface. Using a cylinder ejection mechanism, the projectile is ejected and begins its separation motion. During its motion, the projectile separates into a warhead and a tail section at a certain moment. This separation process is captured at high frequency by a high-speed camera. Figure 5 As shown;

[0055] 6) According to step 5), use a high-speed camera to capture the multi-body separation trajectory at high frequency. Import each captured frame into the pre-compiled multi-body separation trajectory image recognition software, convert the color image to a grayscale image, and then convert it to a black and white image through the adjusted threshold.

[0056] 7) Based on the black and white image obtained from the post-processing in step 6), pixel edge detection technology is used to continuously scan the outline of the projectile in the image, scanning all pixels of each frame, and then scanning all the square pixels in a 3x3 grid. For example: a black and white image has N rows and M columns of pixels. Each pixel is scanned from left to right and from top to bottom, and each pixel is used as the top left corner pixel of each 3x3 grid. From this pixel, the image is scanned down 3 rows and then to the right 3 columns, resulting in a 3x3 grid of 9 pixels. Therefore, an image with N rows and M columns of pixels can be scanned to produce (N-2) × (M-2) square pixels in a 3x3 grid. The 3x3 grid pixel pattern is as follows: Figure 4 As shown;

[0057] 8) Determine if the projectile outline pixels are continuous. If not, replace black pixels not connected to the projectile outline with white pixels and regenerate the replaced black and white image, returning to step 6). If continuous, use multi-body recognition technology to identify the outline information of multiple moving projectiles: During separation, use outline center marking to identify the projectile outline. After separation, if the center value of the original outline changes significantly, it is considered that the outline shape has changed, meaning the projectile itself has separated again. The center value of the outline needs to be re-marked, thus identifying the outline information of multiple moving projectiles, such as... Figure 5 As shown.

[0058] The determination of whether the projectile outline pixels are continuous includes: judging whether a black pixel is the projectile outline based on whether there are two connected black pixels in the horizontal, vertical, and diagonal directions within the nine-grid. Figure 4 As shown; based on the above continuity judgment of contour points, black pixels that are not connected to the projectile contour are replaced with white pixels. These black pixels that are not connected to the projectile are the image background pixels, thereby eliminating the interference of image background pixels. Then, pixel edge detection technology is used for iteration. If the projectile contour is finally obtained, the projectile contour pixels are considered to be continuous; otherwise, they are not continuous.

[0059] 9) Based on the contour information of the separated projectile obtained in step 8), the target object homing technique is used to obtain the center of mass of the target object and the axis of the projectile.

[0060] The method of obtaining the center of mass of the target object and the axis of the projectile by using the target object tracking technology refers to calculating the coordinates of the center of mass of the object using the formula for calculating the center of mass of a uniform density object. The formula for the center of mass coordinates is shown in Formula 6.

[0061]

[0062] In the above formula, u is the x-coordinate of the pixel coordinate system, and v is the y-coordinate of the pixel coordinate system. u1 and u2 are the two x-axis coordinate values ​​in this coordinate system, and v1 and v2 are the two y-axis coordinate values ​​in this coordinate system. and Let x and y be the centroid of the profile.

[0063] Then, connect the midpoint of the outline's head to the object's centroid as an axis, and connect these axes to form a line segment, marking it with pixels of different colors, such as... Figure 5 As shown;

[0064] 10) Based on the obtained center of mass and line segments of the projectiles, calculate the axial displacement, longitudinal displacement, and pitch angle of multiple projectiles. Thus, based on the image recognition method described in steps 1) to 9), real-time prediction of the multi-body separation trajectory during wind tunnel deployment tests is completed. Figure 6 As shown.

[0065] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention based on the above-disclosed technical content without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. An image recognition method for predicting the trajectory of multiple bodies in a deployment test, characterized in that, include: 1) Before the test begins, locate the separation center surface of the catapult inside the core of the wind tunnel test section, and place the black and white checkerboard target plane at the separation center surface. 2) After setting up the black and white checkerboard target surface, set up and fix the high-speed camera, and take multi-angle pictures of the black and white checkerboard inside the core of the wind tunnel test section for calibration. 3) Based on the captured black and white checkerboard calibration images, identify the camera's intrinsic parameter matrix and the world coordinate system extrinsic parameter matrix parameters; 4) Based on the identified parameters, establish the coordinate system matrix transformation relationship between the high-speed camera imaging plane and the calibrated black and white checkerboard target plane, and calculate the coordinate position of the black and white checkerboard relative to the high-speed camera through this matrix transformation relationship. 5) Based on the coordinates of the black and white checkerboard target surface, a wind tunnel launch and separation test is conducted on the target surface. Using the existing ejection mechanism, the projectile is ejected and begins to separate. The separation process is captured by a high-frequency camera. 6) Based on the captured images of the projectile's multi-body separation trajectory, read the high-speed camera's color images frame by frame, and convert each frame of the color image into a black and white image; 7) Perform continuous scanning of the projectile outline in the black and white image; 8) Determine whether the projectile outline pixels are continuous. If they are not continuous, replace the black pixels that are not connected to the projectile outline with white pixels and regenerate the replaced black and white image and return to step 6); if they are continuous, use multi-body recognition technology to identify the outline information of multiple moving projectiles. 9) Based on the obtained contour information of the separated projectile, the target object servo technique is used to obtain the center of mass of the target object and the axis of the projectile. 10) Based on the obtained center of mass and line segment of the projectile, calculate the axial displacement, longitudinal displacement and pitch angle of multiple projectiles to complete the real-time prediction of the multi-body separation trajectory during wind tunnel deployment test.

2. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 1, characterized in that, A high-speed camera was erected and fixed 1.5m away from the outside of the wind tunnel window.

3. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 1, characterized in that, The process of identifying the camera's intrinsic parameter matrix and world coordinate system extrinsic parameter matrix based on captured black and white checkerboard calibration images includes: after taking more than ten calibration photos, finding the cross intersections between black and white squares, and using the least squares method to identify the camera's intrinsic parameter matrix and world coordinate system extrinsic parameter matrix. Where A is the intrinsic parameter matrix of the camera, (u0, v0) is the principal point of the camera in the image coordinate system, a and b are the scale factors of the horizontal and vertical axes of the image, and r represents the perpendicularity of the image coordinate axes; B is the extrinsic parameter matrix that fixes the world coordinate system on a black and white checkerboard grid, R 3×3 It is a rotation matrix describing the camera orientation, T 3×1 It is a three-dimensional translation vector describing the position of the camera center.

4. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 1, characterized in that, The pixel edge detection method is used to continuously scan the outline of the projectile in the black and white image, scan all the pixels of each frame of the image, and scan all the square pixels in the nine-square grid form based on the pixels of this image.

5. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 4, characterized in that, The step of scanning all the nine-square grid-like square pixels based on the pixels of this image includes: assuming a black and white image has N rows and M columns of pixels, scanning each pixel in order from left to right and from top to bottom, and taking each pixel as the top left corner pixel of each nine-square grid, and scanning down 3 rows and right 3 columns from this pixel, scanning a total of 9 pixels for the nine-square grid. Therefore, an image with N rows and M columns of pixels can be scanned to produce (N-2)×(M-2) nine-square grid-like square pixels.

6. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 1, characterized in that, The target object tracking technology is used to obtain the centroid of the target object and the axis of the projectile. This means that the centroid coordinates of the object are calculated using the formula for calculating the centroid of a uniform density object. The midpoint of the head of the contour is found and connected to the centroid of the object as the axis. The axis is then connected into a line segment and marked with different colored pixels.

7. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 1, characterized in that, The determination of whether the projectile outline pixels are continuous includes: determining whether a black pixel is the projectile outline based on whether there are two connected black pixels in the horizontal, vertical, and diagonal directions within the nine-grid; replacing black pixels not connected to the projectile outline with white pixels, these black pixels not connected to the projectile are the image background pixels, thereby eliminating the interference of image background pixels; and then using pixel edge detection technology for iteration. If the projectile outline is finally obtained, the projectile outline pixels are considered continuous; otherwise, they are not continuous.

8. The image recognition method for predicting the trajectory of multiple bodies in a deployment test according to claim 1, characterized in that, The method of using multi-body recognition technology to identify the contour information of multiple moving projectiles includes: identifying the projectile contour by using contour center marking during separation; after separation, if the center value of the original contour changes significantly, it is considered that the contour shape has changed, that is, the projectile itself has separated again, and the center value of the contour needs to be remarked, that is, the contour information of multiple moving projectiles is identified.

Images