Armature attitude diagnosis method, system and device, medium and program product
Through the combination of convolutional neural network and contour fitting, the attitude of the armature point cloud model is automatically adjusted, which solves the problem of large error in armature posture diagnosis in electromagnetic emission in the prior art, and realizes high-precision armature posture calculation.
Patent Information
- Application Number
- CN202411108601.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-13
AI Technical Summary
In the prior art, armature attitude diagnosis during electromagnetic emission depends on artificial marking of landmark points, with large errors and time-consuming and labor-intensive objects, and complex structures require hundreds of landmark points, which affects the accuracy of attitude calculation.
Convolutional neural network is used to predict the pose of the armature point cloud model, and combined with the contour fitting method, the pose is adjusted through iterative calculations to reduce artificial marking errors, and the three-dimensional contour point matching is directly performed, so that the calculation accuracy reaches the pixel level.
It realizes automated and accurate armature attitude diagnosis, reduces artificial marking errors, improves calculation accuracy, and saves manpower and time.
Smart Images

Figure CN118982573A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of electromagnetic emission, and relates to an armature attitude diagnosis method, system, equipment, medium and program product. Background Art
[0002] Electromagnetic rail launch is a new launch technology that uses electromagnetic force (energy) to propel projectiles to high or ultra-high speeds. It can break through the speed limit of traditional chemical launch methods, and the muzzle velocity can reach more than 2km / s. It has significant advantages such as high launch kinetic energy, high system efficiency, high launch frequency, fast start-up time, strong continuous launch capability, and good safety and concealment. It has important application prospects in the military field. The electromagnetic launch process is often accompanied by strong interference factors such as arc light and smoke, which is not conducive to image diagnosis based on visible light imaging. X-rays can penetrate arc light, smoke and other interferences to perform imaging diagnosis on the armature at the muzzle or in the bore during electromagnetic launch. At present, researchers have used X-ray flash photography to obtain armature images for observing deformation, cracking and posture changes of the armature. There are few cases of processing armature images obtained by flash photography.
[0003] At present, the imaging and posture calculation of the armature in the electromagnetic emission process by flash photography is mainly carried out through the method of feature point matching. The main principle of this method is: imaging the armature from different perspectives, manually marking landmark points on the images at different angles, taking the three-dimensional structure of the armature as a priori condition, marking the corresponding three-dimensional landmark points on the three-dimensional model of the object, and calculating the rigid transformation matrix of the object through the registration of multiple two-dimensional images and three-dimensional object landmark points, so as to realize the three-dimensional posture calculation of high-speed moving objects. This trans-membrane state registration method based on landmark points requires human experience to mark landmark points, which will produce large errors and cannot be quantitatively estimated; and for objects with complex structures, hundreds of landmark points are required to realize the registration process, and this marking process will be time-consuming and laborious; and due to imaging from different perspectives, landmark points may be covered due to complex structures or landmark points in different imaging directions may overlap, which will affect the accuracy of posture calculation. Summary of the invention
[0004] The purpose of the present invention is to solve the technical problems in the prior art and to provide an armature attitude diagnosis method, system, device, medium and program product.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] An armature attitude diagnosis method comprises the following steps:
[0007] Acquire an actually captured image, input the actually captured image into a pre-trained convolutional neural network, and obtain a first posture of the armature point cloud model; the pre-trained convolutional neural network training method includes:
[0008] Acquire a training data set, where the training data set is a projection of the armature point cloud model in two axis directions of a pre-established imaging system;
[0009] Input the training data set into the convolutional neural network for training to obtain a trained convolutional neural network;
[0010] Based on the first posture of the armature point cloud model, by comparing the similarity between the projection contour of the armature point cloud model and the contour of the actually captured image, the posture of the armature point cloud model is adjusted to obtain the second posture of the armature point cloud model;
[0011] Based on the second posture of the armature point cloud model, by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, the posture of the armature point cloud model is adjusted to obtain the optimal posture of the armature point cloud model.
[0012] Furthermore, the imaging system includes: two point sources, two imaging plates and the photographed object; the lines connecting the two point sources and the centers of the corresponding imaging plates are perpendicular to each other; the centroid of the photographed object, the point source and the center of the imaging plate are arranged in a straight line; the coordinate origin of the three-dimensional coordinate system is the position of any point source.
[0013] Preferably, the actually captured image is obtained, and the actually captured image is input into a pre-trained convolutional neural network to obtain the first posture of the armature point cloud model, specifically:
[0014] The actual captured image is input into the pre-trained convolutional neural network to obtain the predicted armature angle information;
[0015] Based on the predicted armature angle new type, the rotation angle of the armature point cloud model is adjusted to obtain the first posture of the armature point cloud model.
[0016] Preferably, the first posture of the armature point cloud model is adjusted by comparing the similarity between the projection contour of the armature point cloud model and the contour of the image actually shot, so as to obtain the second posture of the armature point cloud model. The specific method is:
[0017] The armature point cloud model is projected in the imaging system, and the projection contour of the armature point cloud model is extracted; based on the actually captured image, the contour of the actually captured image is extracted;
[0018] The armature point cloud model is rotated around the X, Y, and Z axes at the same step size, and the similarity between the projection contour and the actual captured image contour is compared until the most similar projection contour is found;
[0019] According to the armature angle corresponding to the closest projection contour, the rotation angle of the armature point cloud model is adjusted to obtain the second posture of the armature point cloud model.
[0020] Preferably, the second posture based on the armature point cloud model is adjusted by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, and the specific method for obtaining the posture of the optimal armature point cloud model is:
[0021] The armature point cloud model is projected in the imaging system to extract the projection contour of the armature point cloud model; based on the actually captured image, the contour of the actually captured image is extracted;
[0022] Convert the two-dimensional coordinate point set of the re-projected contour and the contour of the image actually captured into a three-dimensional coordinate point set;
[0023] Based on the three-dimensional coordinate point set of the reprojected contour and the actual image contour, the ICP algorithm is used to find the three-dimensional coordinate point set of the reprojected contour closest to the actual image contour, and the optimal rigid transformation matrix is obtained;
[0024] Adjust the posture of the armature point cloud model based on the optimal rigid transformation matrix;
[0025] Calculate the scale difference between the re-projected contour and the contour of the image actually captured, where the scale difference is the area difference S of the two contours;
[0026] Iterate the above steps until S converges to S∈(0.99,1.01); obtain the posture of the optimal armature point cloud model, where the posture of the optimal armature point cloud model is the posture of the armature point cloud model of the last iteration.
[0027] Furthermore, the rigid transformation matrix is:
[0028]
[0029] Among them, R is a 3*3 rotation matrix and T is a 3*1 translation vector.
[0030] A computer program product comprises computer instructions, wherein the computer instructions instruct a computer to execute the armature attitude diagnosis method.
[0031] An armature attitude diagnosis system, comprising:
[0032] The armature posture prediction module is used to obtain an actually captured image, input the actually captured image into a pre-trained convolutional neural network, and obtain a first posture of the armature point cloud model; the pre-trained convolutional neural network training method includes:
[0033] Acquire a training data set, where the training data set is a projection of the armature point cloud model in two axis directions of a pre-established imaging system;
[0034] Input the training data set into the convolutional neural network for training to obtain a trained convolutional neural network;
[0035] The armature posture initial adjustment module adjusts the posture of the armature point cloud model based on the first posture of the armature point cloud model by comparing the similarity between the projection contour of the armature point cloud model and the contour of the actually captured image, thereby obtaining the second posture of the armature point cloud model;
[0036] The armature attitude fine-tuning module adjusts the attitude of the armature point cloud model based on the second attitude of the armature point cloud model by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, so as to obtain the optimal attitude of the armature point cloud model.
[0037] An electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned armature attitude diagnosis method.
[0038] A computer-readable storage medium stores a computer program, which implements the armature attitude diagnosis method when executed.
[0039] Compared with the prior art, the present invention has the following beneficial effects:
[0040] The present invention combines a convolutional neural network with a contour fitting method, predicts the posture information of an armature point cloud model through a convolutional neural network, and uses a contour fitting method to adjust the posture of the armature point cloud model in a small range based on the posture, without the need to manually mark imaging at different angles, thus saving manpower; and the present invention uses contour constraints of two axes to adjust the posture of the armature point cloud model, reaches convergence through iterative calculation, and can reduce the error caused by manually marking landmark points.
[0041] Furthermore, the present invention can directly perform ICP iteration to obtain the rigid transformation matrix of the armature by directly converting the two-dimensional contour points into three-dimensional ones, without the need for complex conversion of the two-dimensional transformation matrix to the three-dimensional transformation matrix, and can achieve pixel-level calculation accuracy to further reduce the armature posture calculation error. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0043] Figure 1 It is a block diagram of the armature attitude diagnosis method of the present invention;
[0044] Figure 2 It is a schematic diagram of the flow chart of the armature attitude diagnosis method of the present invention;
[0045] Figure 3 It is a graph of angle errors of each axis of the armature attitude diagnosis method of the present invention;
[0046] Figure 4 It is the posture reconstruction result of the electromagnetic launch high-speed armature based on the flash photography technology of the present invention;
[0047] Figure 5 This is a block diagram of the armature attitude diagnosis system of the present invention. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention described and marked in the drawings here can be arranged and designed in various different configurations.
[0049] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0051] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
[0052] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.
[0053] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0054] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0055] See also Figure 1 The present invention discloses an armature attitude diagnosis method, comprising the following steps:
[0056] S1, obtaining an actually captured image, inputting the actually captured image into a pre-trained convolutional neural network, and obtaining a first posture of the armature point cloud model; the convolutional neural network training method includes:
[0057] Acquire a training data set, where the training data set is a projection of the armature point cloud model in two axis directions of a pre-established imaging system;
[0058] The training data set is input into the convolutional neural network for training to obtain a trained convolutional neural network.
[0059] S2, based on the first posture of the armature point cloud model, by comparing the similarity between the projection contour of the armature point cloud model and the contour of the actually captured image, adjusting the posture of the armature point cloud model to obtain a second posture of the armature point cloud model.
[0060] S3, based on the second posture of the armature point cloud model, by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, adjusting the posture of the armature point cloud model to obtain the optimal posture of the armature point cloud model.
[0061] In one embodiment, Figure 2 As shown, a method for diagnosing armature attitude is provided, comprising the following steps:
[0062] S1, establishing an armature point cloud model based on a pre-established imaging system and a three-dimensional coordinate system;
[0063] Pre-establish the imaging system and armature point cloud model: the imaging system consists of two point sources, two imaging plates and an armature point cloud model. The initial state of the imaging system is: the point source, the centroid of the object and the center of the imaging plate are in a straight line, and the lines connecting the two groups of point sources and the corresponding imaging plate centers are perpendicular to each other; the entire imaging system is an orthogonal two-axis imaging setting.
[0064] The imaging system needs to determine the distance between the point source and the armature point source model (SOD) and the distance between the point source and the imaging plate (SDD), and the size and resolution of the two imaging plates used must be determined, and the two imaging plates must be the same size. The SOD and SDD distances in the two axis directions must be consistent to ensure the same magnification.
[0065] One of the point source positions is selected as the coordinate origin of the imaging system, a three-dimensional coordinate system is established, and the coordinates of the two point sources and the corresponding imaging plate and the object under the coordinate system are determined; a three-dimensional point cloud model of the object is established in the three-dimensional coordinate system of the imaging system, and the object described in the present invention is the armature.
[0066] S2, obtaining an actually captured image, inputting the actually captured image into a pre-trained convolutional neural network, and obtaining armature angle prediction information; adjusting the posture of the armature point cloud model according to the armature angle prediction information, and obtaining a first posture of the armature point cloud model;
[0067] S201, firstly, training the convolutional neural network, wherein the training method of the convolutional neural network is:
[0068] In the established imaging system, the positions of the point source and the imaging plate in the two-axis direction are kept fixed, the posture of the armature point cloud model is adjusted based on the initial position, and the armature is controlled to rotate around the X, Y, and Z axes at any angle in an external rotation manner to generate projection data. The method of generating projection data is: by writing code to calculate the projection coordinates of the armature point cloud model in a point projection manner, and then using the OpenCV library to write code to convert the projection coordinates into pixel coordinates to generate a binary image, the binary image size is 1600*1600, and the pixel size is 40pix / mm. Then calculate the signal-to-noise ratio of the image obtained by actual experimental shooting, use the OpenCV library to write code to add Poisson noise and Gaussian noise so that the generated image has the same signal-to-noise ratio as the image obtained by actual shooting, and simulate the quality of the X-ray image obtained by actual experimental shooting.
[0069] Set the parameters of the convolutional neural network ResNet34: modify the input layer to meet the reading of training data. The format of the training data is two images per group, the label is the corresponding rotation angle around each axis (XYZ order), and the output is the predicted rotation angle around each axis. The loss function is set to the root mean square error between the predicted value and the label. Use the S2 method to generate about 30,000 sets of data (that is, generate binary images with the same signal-to-noise ratio as the images obtained by the actual experiment), and randomly divide these binary images into a training set and a test set with a ratio of 9:1 to train the convolutional neural network; during training: the initial learning rate is set to 0.01, the learning rate is reduced to 0.001 after 30 epochs, and the learning rate is reduced to 0.0001 after the 50th epoch. A total of 72 epochs are trained, and batch_size is set to 128.
[0070] S202, input the X-ray image actually taken into the trained convolutional neural network, the X-ray image actually taken is the image actually taken, and the image actually taken is the X-ray image of orthogonal imaging; the convolutional neural network outputs 3 values, the output values are the angle values a, b, c corresponding to the rotation of the armature point cloud model around the imaging system coordinate system in the order of XYZ, calculate the rotation matrix R based on the output angle values, use the PCL library to write a program, multiply the armature point cloud model from the initial position by the rotation matrix R, adjust the posture of the armature point cloud model, and obtain the first posture of the armature point cloud model; the rotation matrix is expressed as:
[0071]
[0072] Where a, b, and c represent the rotation angles of the armature point cloud model around the X, Y, and Z axes, respectively.
[0073] S3, based on the first posture of the armature point cloud model, projecting the armature point cloud model in the imaging system, extracting the projection contour of the armature point cloud model; extracting the contour of the image actually shot; comparing the similarity between the projection contour of the armature point cloud model and the contour of the image actually shot, and finding the closest armature angle; based on the closest armature angle, adjusting the posture of the armature point cloud model to obtain the second posture of the armature point cloud model;
[0074] The armature point cloud model in the first posture is projected from two directions, and the projection contour is extracted; the projection contour and the actual imaged X-ray image contour are rotated in a small range around the XYZ three axes with a certain step length, and the contour similarity of the two-axis images is compared through the matchshape function in the OpenCV library to find the armature angle with the closest contour (determined according to the contour matching score), adjust the armature point cloud model posture, and obtain the second posture of the armature point cloud model; the step length can be set according to the actual situation. The smaller the step length, the more accurate the result. At the same time, the calculation speed becomes slower as the data volume increases.
[0075] S4, based on the second posture of the armature point cloud model, projecting the armature point cloud model in the imaging system to obtain a re-projected contour of the armature point cloud model; comparing the scale difference between the re-projected contour of the armature point cloud model and the contour of the actually captured image to find the optimal rigid transformation matrix; based on the optimal rigid transformation matrix, obtaining the posture of the optimal armature point cloud model;
[0076] S401, re-projecting the armature point cloud model in the imaging system, extracting the re-projected contour of the armature point cloud model; extracting the contour of the actually captured image based on the actually captured image; calculating the contour scale difference S between the re-projected contour and the contour of the actually imaged X-ray image (by comparing the lengths or areas of the two contours); and then converting the two-dimensional contour point set into a three-dimensional coordinate point set in the imaging system coordinate system; the method for converting the two-dimensional contour point set into a three-dimensional coordinate point set is: assigning a third value missing from the coordinate point according to the position of the imaging plate, and obtaining the three-dimensional coordinate point set of the two contours.
[0077] S402, then use the three-dimensional coordinate point set of the re-projected contour and the actual imaged X-ray image contour to perform ICP (iterative closest point algorithm) calculation, find the optimal matching parameters R and T of the three-dimensional point set that is closest to the projection contour and the actual imaged X-ray image contour, and obtain the rigid transformation matrix H (the translation vector needs to be adjusted according to the scale difference); adjust the point cloud model posture by multiplying the three-dimensional point set of the armature point cloud model by the rigid transformation matrix; iterate the above process until the scale difference S between the re-projected contour of the armature point cloud model and the actual imaged X-ray image contour converges within the set threshold (S∈(0.99, 1.01));
[0078] Based on the above method, the same operation is performed in two orthogonal directions of the imaging system, and iterative calculations are performed in sequence to obtain the optimal rigid transformation matrix and the posture of the optimal armature point cloud model.
[0079] The rigid transformation matrix is:
[0080]
[0081] Among them, R is a 3*3 rotation matrix and T is a 3*1 translation vector.
[0082] The ICP algorithm error function is:
[0083]
[0084] Among them, n is the number of contour coordinate points, q i is the actual wheel image contour point coordinate, p i are the coordinates of the projection contour points.
[0085] The optimal rigid transformation matrix is the product of the initial rigid transformation matrix and all rigid transformation matrices calculated in the iterative process; the optimal posture of the armature point cloud model is the posture of the armature point cloud model in the last iteration.
[0086] In order to further illustrate the superiority of the armature attitude diagnosis method of the present invention, firstly, the optimal rigid transformation matrix obtained by the present invention is decomposed into Euler angles to obtain the angle parameters of the armature point cloud model rotating around the X, Y, and Z axes, and the translation vector is extracted, and the translation vector is the displacement of the armature point cloud model; then, the point cloud model reconstructed by the armature attitude diagnosis method of the present invention is compared with the standard point cloud model; the angle error and displacement error of the armature point cloud model reconstructed by the present invention are analyzed respectively, and the results are as follows: Figure 3 and Figure 4 As shown; it can be seen that the armature attitude diagnosis method of the present invention has fast calculation speed and high accuracy.
[0087] In one embodiment, Figure 5 As shown, the present invention discloses an armature attitude diagnosis system, comprising:
[0088] The armature posture prediction module is used to obtain an actually captured image, input the actually captured image into a pre-trained convolutional neural network, and obtain a first posture of the armature point cloud model; the pre-trained convolutional neural network training method includes:
[0089] Acquire a training data set, where the training data set is a projection of the armature point cloud model in two axis directions of a pre-established imaging system;
[0090] Input the training data set into the convolutional neural network for training to obtain a trained convolutional neural network;
[0091] The armature posture initial adjustment module adjusts the posture of the armature point cloud model based on the first posture of the armature point cloud model by comparing the similarity between the projection contour of the armature point cloud model and the contour of the actually captured image, thereby obtaining the second posture of the armature point cloud model;
[0092] The armature attitude fine-tuning module adjusts the attitude of the armature point cloud model based on the second attitude of the armature point cloud model by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, so as to obtain the optimal attitude of the armature point cloud model.
[0093] In one embodiment, the present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.
[0094] The computer program product or computer program provided by the present invention is a set of software solutions designed for armature attitude diagnosis. It integrates advanced image processing, machine learning and three-dimensional point cloud processing technology, can automatically analyze the actual captured images, and quickly and accurately determine the attitude information of the armature based on these images; the computer program or computer program product of the present invention is designed to be stored in various computer-readable storage media, including but not limited to hard disks, solid-state hard disks, USB flash drives, optical disks, cloud storage, etc.; when the processor of the computer device reads and executes these computer instructions from the computer-readable storage medium, it will automatically execute the steps in the above-mentioned method embodiments, thereby realizing the automatic and intelligent diagnosis of the armature attitude.
[0095] In one embodiment, the present invention provides a device including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0096] The device integrates high-performance memory and processor to ensure efficient execution of complex image processing, machine learning and three-dimensional point cloud processing tasks; the memory is used to store key data such as pre-trained convolutional neural network models, armature point cloud models, training data sets and actual captured images; the processor has powerful computing power, can quickly execute complex image processing and machine learning algorithms, supports multi-task parallel processing, and can process multiple image data simultaneously.
[0097] In another example, the device also includes an image input interface and a data transmission interface. The image input interface supports importing actual captured images from external devices such as cameras and scanners; the data transmission interface is used to exchange data with computers, other devices or networks.
[0098] In one embodiment, the present invention provides a computer-readable storage medium storing a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0099] The computer-readable storage media of the present invention include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-RLTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card or secure digital (SD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device configured to store the computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.
[0100] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An armature attitude diagnosis method, characterized in that: The following steps are involved: Acquire an actually captured image, and input the actually captured image into a pre-trained convolutional neural network to obtain a first posture of the armature point cloud model; The pre-trained convolutional neural network training method includes: Acquire a training data set, where the training data set is a projection of the armature point cloud model in two axis directions of a pre-established imaging system; Input the training data set into the convolutional neural network for training to obtain a trained convolutional neural network; Based on the first posture of the armature point cloud model, by comparing the similarity between the projection contour of the armature point cloud model and the contour of the actually captured image, the posture of the armature point cloud model is adjusted to obtain the second posture of the armature point cloud model; Based on the second posture of the armature point cloud model, by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, the posture of the armature point cloud model is adjusted to obtain the optimal posture of the armature point cloud model.
2. The armature attitude diagnosis method according to claim 1, characterized in that: The imaging system comprises: two point sources, two imaging plates and an object to be photographed; the lines connecting the two point sources and the centers of the corresponding imaging plates are perpendicular to each other; the centroid of the object to be photographed, the point source and the center of the imaging plate are arranged in a straight line; the coordinate origin of the three-dimensional coordinate system is the position of any point source.
3. The armature attitude diagnosis method according to claim 1, characterized in that: The actual captured image is obtained, and the actual captured image is input into a pre-trained convolutional neural network to obtain a first posture of the armature point cloud model, specifically: The actual captured image is input into the pre-trained convolutional neural network to obtain the predicted armature angle information; Based on the predicted armature angle new type, the rotation angle of the armature point cloud model is adjusted to obtain the first posture of the armature point cloud model.
4. The armature attitude diagnosis method according to claim 1, characterized in that: The first posture of the armature point cloud model is based on the comparison of the similarity between the projection contour of the armature point cloud model and the contour of the image actually shot, and the posture of the armature point cloud model is adjusted to obtain the second posture of the armature point cloud model. The specific method is: The armature point cloud model is projected in the imaging system, and the projection contour of the armature point cloud model is extracted; based on the actually captured image, the contour of the actually captured image is extracted; The armature point cloud model is rotated around the X, Y, and Z axes at the same step size, and the similarity between the projection contour and the actual captured image contour is compared until the most similar projection contour is found; According to the armature angle corresponding to the closest projection contour, the rotation angle of the armature point cloud model is adjusted to obtain the second posture of the armature point cloud model.
5. The armature attitude diagnosis method according to claim 1, characterized in that: The specific method of adjusting the posture of the armature point cloud model by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the image actually shot to obtain the posture of the optimal armature point cloud model is as follows: The armature point cloud model is projected in the imaging system to extract the projection contour of the armature point cloud model; based on the actually captured image, the contour of the actually captured image is extracted; Convert the two-dimensional coordinate point set of the re-projected contour and the contour of the image actually captured into a three-dimensional coordinate point set; Based on the three-dimensional coordinate point set of the reprojected contour and the actual image contour, the ICP algorithm is used to find the three-dimensional coordinate point set of the reprojected contour that is closest to the actual image contour, and the optimal rigid transformation matrix is obtained; Adjust the posture of the armature point cloud model based on the optimal rigid transformation matrix; Calculate the scale difference between the re-projected contour and the contour of the image actually captured, where the scale difference is the area difference S of the two contours; Iterate the above steps until S converges to S∈(0.99,1.01); obtain the posture of the optimal armature point cloud model, where the posture of the optimal armature point cloud model is the posture of the armature point cloud model of the last iteration.
6. The armature attitude diagnosis method according to claim 5, characterized in that: The rigid transformation matrix is: Among them, R is a 3*3 rotation matrix and T is a 3*1 translation vector.
7. A computer program product, comprising computer instructions, characterized in that: The computer instructions instruct the computer to execute the armature attitude diagnosis method described in any one of claims 1-6.
8. An armature attitude diagnosis system, based on the armature attitude diagnosis method according to claim 1, characterized in that: include: An armature posture prediction module is used to obtain an actually shot image, input the actually shot image into a pre-trained convolutional neural network, and obtain a first posture of the armature point cloud model; The pre-trained convolutional neural network training method includes: Acquire a training data set, where the training data set is a projection of the armature point cloud model in two axis directions of a pre-established imaging system; Input the training data set into the convolutional neural network for training to obtain a trained convolutional neural network; The armature posture initial adjustment module adjusts the posture of the armature point cloud model based on the first posture of the armature point cloud model by comparing the similarity between the projection contour of the armature point cloud model and the contour of the actually captured image, thereby obtaining the second posture of the armature point cloud model; The armature attitude fine-tuning module adjusts the attitude of the armature point cloud model based on the second attitude of the armature point cloud model by comparing the scale difference between the projection contour of the armature point cloud model and the contour of the actually captured image, so as to obtain the optimal attitude of the armature point cloud model.
9. An electronic device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the armature attitude diagnosis method as described in any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the armature attitude diagnosis method as described in any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Image-based object model matching posture measurement method
CN102411779A
Attitude measurement method and device
CN105184803A
Three-dimensional human body model attitude and shape reconstruction method through video sequence
CN108629801A
High-precision pose tracking and detecting method for microstructure features
CN112634365A
Method and device for determining pose information of automobile
CN115115704A
