Missile loading position pose deviation detection system and method based on binocular vision

Abstract

The application provides a missile loading position deviation detection system and method based on binocular vision, which comprises a binocular camera, a sling target group, a missile cradle target group and a camera controller. The application can effectively solve the high-precision positioning problem of the loading operation of the vehicle-mounted missile automatic loading system, and has flexible arrangement, convenient operation and high automation degree, and can provide real-time state feedback information for the missile automatic loading control.

Description

Technical Field

[0001] This invention relates to the field of missile loading control technology, and more specifically, to a missile loading posture deviation detection system and method based on binocular vision. Background Technology

[0002] Existing tactical missile loading systems mostly employ manual or semi-automatic loading modes, resulting in low loading efficiency and high operational skill requirements. Therefore, the demand for developing a vehicle-mounted fully automatic missile loading system adapted to field environments is becoming increasingly strong.

[0003] During missile loading, the pin holes of the missile loading module must be precisely aligned with the positioning pins on the missile carrier, placing high demands on the positioning accuracy of the automatic loading system. However, the automatic loading system consists of a complex multi-degree-of-freedom robotic arm and lifting device. Elastic deformation caused by heavy loads and control errors at each joint are transmitted step-by-step and amplified at the end, making it difficult for the system's open-loop accuracy to meet the loading alignment requirements. Furthermore, the random parking positions of the loading vehicle and the launch vehicle under field conditions further increase the uncertainty of loading posture deviation. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of this invention is to provide a missile loading posture deviation detection system and method based on binocular vision.

[0005] A missile loading pose deviation detection method based on binocular vision according to the present invention includes:

[0006] Step S1: Calibrate the lifting device data: Measure the pose relationship between the lifting device target group and the lifting device coordinate system, and record it in the camera controller;

[0007] Step S2, Calibrate missile bracket data: Measure the pose relationship between the missile bracket target group and the missile bracket coordinate system, and record it in the camera controller;

[0008] Step S3, Calibrate the reference pose for loading: When the lifting device holds the missile module and positions it on the missile bracket, use a binocular camera to simultaneously photograph the lifting device target group and the missile bracket target group. The camera controller calculates and records the relative pose of the lifting device coordinate system and the missile bracket coordinate system as the reference pose during missile loading operations.

[0009] Step S4: Real-time pose detection during loading operation: During missile loading operation, a binocular camera is used to simultaneously photograph the lifting target group and the missile bracket target group, and the camera controller calculates the real-time pose of the lifting coordinate system and the missile bracket coordinate system.

[0010] Step S5: Calculate the loading pose deviation: Using the camera controller, calculate the pose deviation between the real-time pose of the spreader coordinate system and the reference pose based on the calibrated loading operation reference pose and the detected real-time loading pose.

[0011] Preferably, in step S1: a rigging coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the rigging coordinate system in the camera coordinate system at this time is denoted as... Let m be the number of targets in the lifting target group. Then, plot the three-dimensional coordinates of each target in the camera coordinate system.

[0012] In step S2: the missile carrier coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the missile carrier coordinate system in the camera coordinate system at this time is denoted as... Let n be the number of targets in the missile carrier target group. Then, plot the three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system.

[0013] Preferably, in step S3, the three-dimensional coordinates of each target in the sling target group in the camera coordinate system are marked. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. Calculate the reference relative pose of the sling coordinate system and the missile carrier coordinate system using the following steps:

[0014] Step S3.1, Coordinate Normalization: Normalize the calibrated and measured values ​​of each target according to the following formula:

[0015]

[0016]

[0017] This represents the normalized coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0018] This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0019] This indicates the three-dimensional coordinates of the k-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0020] This indicates the normalized coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory;

[0021] This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory;

[0022] This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory;

[0023] This represents the normalized coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0024] This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0025] This indicates the three-dimensional coordinates of the k-th target in the missile carrier target group in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0026] This indicates the normalized coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before leaving the factory;

[0027] This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory;

[0028] This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory;

[0029] Step S3.2: Calculate the pose matrix H of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system. s H f Construct the following matrix from normalized coordinates. and Singular value decomposition yields:

[0030] H s =U s Σ s V sT H f =U f Σ f V f T (3)

[0031] U s U f Represents a left singular matrix;

[0032] Σ s Σ f Represents a diagonal matrix;

[0033] V s V f Represents a right singular matrix;

[0034] Then, calculate the relative pose matrices of the current moment between the sling coordinate system, the missile bracket coordinate system, and the calibration time in steps S1 and S2.

[0035]

[0036] in:

[0037]

[0038]

[0039] This allows us to obtain the pose matrix of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system at the current moment.

[0040]

[0041] Step S3.3: Calculate the loading reference pose: Calculate the pose relationship between the spreader coordinate system and the missile bracket coordinate system during loading alignment using the following formula, and record it as the loading reference pose.

[0042]

[0043] Preferably, in step S4:

[0044] The three-dimensional coordinates of each target in the camera coordinate system of the lifting target group are then marked. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. The pose matrices of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system during the loading operation were calculated respectively. This leads to the positional relationship between the lifting device coordinate system and the missile support coordinate system during the loading operation.

[0045]

[0046] In step S5: Calculate the loading position deviation

[0047]

[0048] A missile loading posture deviation detection system based on binocular vision according to the present invention includes:

[0049] Module M1, Calibration of Lifting Gear Data: Measure the pose relationship between the lifting gear target group and the lifting gear coordinate system, and record it in the camera controller;

[0050] Module M2, calibrating missile bracket data: measuring the pose relationship between the missile bracket target group and the missile bracket coordinate system, and recording it in the camera controller;

[0051] Module M3, Calibration and Loading Reference Posture: When the lifting device holds the missile module and positions it on the missile bracket, a binocular camera is used to simultaneously photograph the lifting device target group and the missile bracket target group. The camera controller calculates and records the relative posture of the lifting device coordinate system and the missile bracket coordinate system as a reference posture during missile loading operations.

[0052] Module M4, Real-time pose detection during loading operation: During missile loading operation, a binocular camera is used to simultaneously photograph the lifting target group and the missile bracket target group, and the camera controller calculates the real-time pose of the lifting coordinate system and the missile bracket coordinate system.

[0053] Module M5, Calculate the loading posture deviation: Using the camera controller, the posture deviation between the real-time posture of the spreader coordinate system and the reference posture is calculated based on the calibrated loading operation reference posture and the detected real-time loading posture.

[0054] Preferably, in module M1: a rigging coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the rigging coordinate system in the camera coordinate system at this time is denoted as... Let m be the number of targets in the lifting target group. Then, plot the three-dimensional coordinates of each target in the camera coordinate system.

[0055] In module M2: the missile carrier coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the missile carrier coordinate system in the camera coordinate system at this time is denoted as... Let n be the number of targets in the missile carrier target group. Then, plot the three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system.

[0056] Preferably, in module M3, the three-dimensional coordinates of each target in the target group of the lifting device in the camera coordinate system are marked. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. Calculate the reference relative pose of the sling coordinate system and the missile carrier coordinate system using the following steps:

[0057] Module M3.1, Coordinate Normalization: The calibrated and measured values ​​of each target are normalized according to the following formula:

[0058]

[0059]

[0060] This represents the normalized coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0061] This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0062] This indicates the three-dimensional coordinates of the k-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0063] This indicates the normalized coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory;

[0064] This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory;

[0065] This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory;

[0066] This represents the normalized coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0067] This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0068] This indicates the three-dimensional coordinates of the k-th target in the missile carrier target group in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0069] This indicates the normalized coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before leaving the factory;

[0070] This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory;

[0071] This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory;

[0072] Module M3.2 calculates the pose matrix H of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system. s H f Construct the following matrix from normalized coordinates. and Singular value decomposition yields:

[0073] H s =U s Σ s V s T H f =U f Σ f V f T (3)

[0074] U s U f Represents a left singular matrix;

[0075] Σ s Σ f Represents a diagonal matrix;

[0076] V s V f Represents a right singular matrix;

[0077] Then, calculate the relative pose matrices of the current moment between the sling coordinate system, the missile bracket coordinate system, and the calibration time of module M1 and module M2.

[0078]

[0079] in:

[0080]

[0081]

[0082] This allows us to obtain the pose matrix of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system at the current moment.

[0083]

[0084] Module M3.3, Calculate the loading reference pose: Calculate the pose relationship between the spreader coordinate system and the missile bracket coordinate system during loading alignment using the following formula, and denote it as the loading reference pose.

[0085]

[0086] Preferably, in module M4:

[0087] The three-dimensional coordinates of each target in the camera coordinate system of the lifting target group are then marked. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. The pose matrices of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system during the loading operation were calculated respectively. This leads to the positional relationship between the lifting device coordinate system and the missile support coordinate system during the loading operation.

[0088]

[0089] In module M5: calculate the loading position deviation.

[0090]

[0091] According to the present invention, a missile loading posture deviation detection device based on binocular vision includes: a binocular camera, a lifting target group, a missile bracket target group, and a camera controller;

[0092] It also includes the aforementioned missile loading posture deviation detection system based on binocular vision; or, the aforementioned missile loading posture deviation detection method based on binocular vision.

[0093] Preferably, the binocular camera is installed in a position that allows its field of view to simultaneously cover the lifting target group and the missile bracket target group during the loading process; the lifting target group consists of a set of targets with a fixed positional relationship to the lifting device; the missile bracket target group consists of a set of targets with a fixed positional relationship to the missile bracket.

[0094] Compared with the prior art, the present invention has the following beneficial effects:

[0095] 1. The installation layout of the binocular camera and each target in this invention is highly flexible and can adapt to various missile loading system structures and various loading conditions.

[0096] 2. This invention uses binocular vision detection technology, which can obtain high-precision three-dimensional coordinates of the target, and thus obtain high-precision pose information of the object being tested, which can effectively improve the detection accuracy of loading pose deviation.

[0097] 3. This invention can realize automatic shooting of binocular cameras and automatic calculation of loading posture deviation through camera controller. It has a high degree of automation and is easy to integrate into the missile automatic loading control system, which significantly improves the accuracy and automation of missile loading operations. Attached Figure Description

[0098] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0099] Figure 1 This is a flowchart illustrating the missile loading posture deviation detection method based on binocular vision according to the present invention.

[0100] Figure 2 This is a schematic diagram of the missile loading posture deviation detection system based on binocular vision according to this application.

[0101] Explanation of reference numerals in the attached figures:

[0102] Detailed Implementation

[0103] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0104] First, the basic embodiments of the present invention will be described. A missile loading posture deviation detection system based on binocular vision provided by the present invention includes: a binocular camera, a hoist target group, a missile bracket target group, and a camera controller.

[0105] The binocular camera is positioned such that its field of view simultaneously covers both the lifting device target group and the missile carrier target group during the loading process. The lifting device target group includes a set of targets with a fixed pose relationship to the lifting device, used to calculate the real-time pose of the lifting device coordinate system. The missile carrier target group includes a set of targets with a fixed pose relationship to the missile carrier, used to calculate the real-time pose of the missile carrier coordinate system. The camera controller controls the binocular camera to capture images, calculates the coordinates of each target, solves for the missile loading pose deviation, and performs missile loading pose deviation detection.

[0106] like Figure 2 As shown, the present invention provides a missile loading posture deviation detection system based on binocular vision, which can be implemented according to the following... Figure 2 The arrangement shown is applied in an automatic missile loading system. The binocular camera is mounted on the outer segment of the loading robotic arm; the target assembly is mounted on the side of the loader; the missile bracket target assembly is mounted on the side of the missile bracket; and the camera controller is installed inside the loader and connected to the binocular camera via a cable.

[0107] According to the present invention, a missile loading posture deviation detection method based on binocular vision is provided, such as... Figure 1 As shown, the missile loading operation using the aforementioned binocular vision-based missile loading posture deviation detection system includes the following steps:

[0108] Step S1: Calibrate the lifting device data before leaving the factory: Before the system leaves the factory, measure the pose relationship between the lifting device target group and the lifting device coordinate system, and record it in the camera controller;

[0109] Step S2, Pre-shipment calibration of missile bracket data: Before the system leaves the factory, measure the pose relationship between the missile bracket target group and the missile bracket coordinate system, and record it in the camera controller;

[0110] Step S3, Pre-shipment calibration of the loading reference pose: Before the system leaves the factory, when the lifting device holds the missile module and accurately positions it on the missile bracket, a binocular camera is used to simultaneously photograph the lifting device target group and the missile bracket target group. The camera controller calculates and records the relative pose of the lifting device coordinate system and the missile bracket coordinate system as the reference pose during the missile loading operation.

[0111] Step S4: Real-time pose detection during loading operation: During missile loading operation, a binocular camera is used to simultaneously photograph the lifting target group and the missile bracket target group, and the camera controller calculates the real-time pose of the lifting coordinate system and the missile bracket coordinate system.

[0112] Step S5: Calculate the loading posture deviation: Using the camera controller, calculate the posture deviation between the real-time posture of the spreader coordinate system and the reference posture based on the loading operation reference posture calibrated before leaving the factory and the real-time loading posture obtained by detection.

[0113] The present invention will now be described in more detail with reference to preferred embodiments of the basic implementation. The present invention provides a missile loading pose deviation detection method based on binocular vision, comprising the following steps:

[0114] Step S1: Pre-shipment calibration of the lifting device data: Before the system leaves the factory, the pose relationship between the lifting device target group and the lifting device coordinate system is measured and recorded in the camera controller. Specifically, the lifting device coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the lifting device coordinate system in the camera coordinate system at this time is denoted as... Let m be the number of targets in the lifting target group. Then, plot the three-dimensional coordinates of each target in the camera coordinate system.

[0115] Step S2: Pre-shipment calibration of missile carrier data: Before the system leaves the factory, the pose relationship between the missile carrier target group and the missile carrier coordinate system is measured and recorded in the camera controller. Specifically, the missile carrier coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the missile carrier coordinate system in the camera coordinate system at this time is denoted as... Let n be the number of targets in the missile carrier target group. Then, plot the three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system.

[0116] Step S3: Pre-shipment Calibration of Loading Reference Posture: Before the system leaves the factory, when the lifting device accurately positions the missile module on the missile carrier, a binocular camera simultaneously captures images of both the lifting device target group and the missile carrier target group. The camera controller calculates and records the relative postures of the lifting device coordinate system and the missile carrier coordinate system, which serve as the reference postures for missile loading operations. Specifically, the three-dimensional coordinates of each target in the lifting device target group in the camera coordinate system at this time are marked as... The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. Calculate the reference relative pose of the sling coordinate system and the missile carrier coordinate system using the following steps:

[0117] Step S3.1, Coordinate Normalization: Normalize the calibrated and measured values ​​of each target according to the following formula:

[0118]

[0119]

[0120] This represents the normalized coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0121] This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0122] This indicates the three-dimensional coordinates of the k-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory.

[0123] This indicates the normalized coordinates of the i-th target in the camera coordinate system when the missile module is accurately positioned on the missile bracket by the lifting device before leaving the factory.

[0124] This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system when the missile module is accurately positioned on the missile bracket by the lifting device before leaving the factory;

[0125] This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the missile module is accurately positioned on the missile bracket by the lifting device before leaving the factory;

[0126] This represents the normalized coordinates of the j-th target in the missile carrier target group in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0127] This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0128] This indicates the three-dimensional coordinates of the k-th target in the missile carrier target group in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory.

[0129] This indicates the normalized coordinates of the j-th target in the missile carrier target group in the camera coordinate system when the lifting device accurately positions the missile module on the missile carrier before leaving the factory.

[0130] This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system when the lifting device accurately positions the missile module on the missile bracket before it leaves the factory;

[0131] This indicates the three-dimensional coordinates of the kth target in the missile bracket target group in the camera coordinate system when the lifting device accurately positions the missile module on the missile bracket before leaving the factory;

[0132] Step S3.2: Calculate the pose matrix H of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system. s H f Construct the following matrix from normalized coordinates. and Singular value decomposition yields:

[0133] H s =U s Σ s V s T H f =U f Σ f V f T (3)

[0134] U s U f Represents a left singular matrix;

[0135] Σ s Σ f Represents a diagonal matrix;

[0136] V s V f Represents a right singular matrix;

[0137] Then, calculate the relative pose matrices of the current moment between the sling coordinate system, the missile bracket coordinate system, and the calibration time in steps S1 and S2.

[0138]

[0139] in:

[0140]

[0141]

[0142] This allows us to obtain the pose matrix of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system at the current moment.

[0143]

[0144] Step S3.3: Calculate the loading reference pose: The pose relationship between the spreader coordinate system and the missile bracket coordinate system during loading alignment can be calculated according to the following formula, and is denoted as the loading reference pose.

[0145]

[0146] Step S4: Real-time pose detection during missile loading: During missile loading, a binocular camera simultaneously captures images of the hoist target group and the missile support target group. The camera controller calculates the real-time pose in both the hoist coordinate system and the missile support coordinate system. Specifically, the three-dimensional coordinates of each target in the hoist target group in the camera coordinate system are plotted at this time. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. Using the same algorithm in step S3, the pose matrices of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system during the loading operation can be calculated respectively. This leads to the positional relationship between the lifting device coordinate system and the missile support coordinate system during the loading operation.

[0147]

[0148] Step S5: Calculate the loading posture deviation: Using the camera controller, calculate the posture deviation between the real-time posture of the spreader coordinate system and the reference posture based on the pre-calibrated loading operation reference posture and the detected real-time loading posture. Specifically, the loading posture deviation can be calculated according to the following formula.

[0149]

[0150] This invention also provides a missile loading posture deviation detection system based on binocular vision. The system can be implemented by executing the steps of the binocular vision-based missile loading posture deviation detection method. That is, those skilled in the art can understand the binocular vision-based missile loading posture deviation detection method as a preferred embodiment of the binocular vision-based missile loading posture deviation detection system. Specifically, the binocular vision-based missile loading posture deviation detection system provided by this invention includes:

[0151] Module M1, Calibration of Lifting Gear Data: Measure the pose relationship between the lifting gear target group and the lifting gear coordinate system, and record it in the camera controller;

[0152] Module M2, calibrating missile bracket data: measuring the pose relationship between the missile bracket target group and the missile bracket coordinate system, and recording it in the camera controller;

[0153] Module M3, Calibration and Loading Reference Posture: When the lifting device holds the missile module and positions it on the missile bracket, a binocular camera is used to simultaneously photograph the lifting device target group and the missile bracket target group. The camera controller calculates and records the relative posture of the lifting device coordinate system and the missile bracket coordinate system as a reference posture during missile loading operations.

[0154] Module M4, Real-time pose detection during loading operation: During missile loading operation, a binocular camera is used to simultaneously photograph the lifting target group and the missile bracket target group, and the camera controller calculates the real-time pose of the lifting coordinate system and the missile bracket coordinate system.

[0155] Module M5, Calculate the loading posture deviation: Using the camera controller, the posture deviation between the real-time posture of the spreader coordinate system and the reference posture is calculated based on the calibrated loading operation reference posture and the detected real-time loading posture.

[0156] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0157] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for detecting missile loading posture deviation based on binocular vision, characterized in that, include: Step S1: Calibrate the lifting device data: Measure the pose relationship between the lifting device target group and the lifting device coordinate system, and record it in the camera controller; Step S2, Calibrate missile bracket data: Measure the pose relationship between the missile bracket target group and the missile bracket coordinate system, and record it in the camera controller; Step S3, Calibrate the reference pose for loading: When the lifting device holds the missile module and positions it on the missile bracket, use a binocular camera to simultaneously photograph the lifting device target group and the missile bracket target group. The camera controller calculates and records the relative pose of the lifting device coordinate system and the missile bracket coordinate system as the reference pose during missile loading operations. Step S4: Real-time pose detection during loading operation: During missile loading operation, a binocular camera is used to simultaneously photograph the lifting target group and the missile bracket target group, and the camera controller calculates the real-time pose of the lifting coordinate system and the missile bracket coordinate system. Step S5: Calculate the loading pose deviation: Using the camera controller, calculate the pose deviation between the real-time pose of the spreader coordinate system and the reference pose based on the calibrated loading operation reference pose and the detected real-time loading pose.

2. The missile loading posture deviation detection method based on binocular vision according to claim 1, characterized in that, In step S1: a rigging coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the rigging coordinate system in the camera coordinate system at this time is denoted as... Let m be the number of targets in the lifting target group. Then, plot the three-dimensional coordinates of each target in the camera coordinate system. In step S2: the missile carrier coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the missile carrier coordinate system in the camera coordinate system at this time is denoted as... Let n be the number of targets in the missile carrier target group. Then, plot the three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system.

3. The missile loading posture deviation detection method based on binocular vision according to claim 2, characterized in that, In step S3, the three-dimensional coordinates of each target in the sling target group in the camera coordinate system are plotted. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. Calculate the reference relative pose of the sling coordinate system and the missile carrier coordinate system using the following steps: Step S3.1, Coordinate Normalization: Normalize the calibrated and measured values ​​of each target according to the following formula: This represents the normalized coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory. This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory. This indicates the three-dimensional coordinates of the k-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory. This indicates the normalized coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory; This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory; This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory; This represents the normalized coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory. This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory. This indicates the three-dimensional coordinates of the k-th target in the missile carrier target group in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory. This indicates the normalized coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before leaving the factory; This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory; This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory; Step S3.2: Calculate the pose matrix H of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system. s H f Construct the following matrix from normalized coordinates. and Singular value decomposition yields: H s =U s S s V s T ,H f =U f S f V f T (3) U s U f Represents a left singular matrix; Σ s Σ f Represents a diagonal matrix; V s V f Represents a right singular matrix; Then, calculate the relative pose matrices of the current moment between the sling coordinate system, the missile bracket coordinate system, and the calibration time in steps S1 and S2. in: This allows us to obtain the pose matrix of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system at the current moment. Step S3.3: Calculate the loading reference pose: Calculate the pose relationship between the spreader coordinate system and the missile bracket coordinate system during loading alignment using the following formula, and record it as the loading reference pose.

4. The missile loading posture deviation detection method based on binocular vision according to claim 3, characterized in that, In step S4: The three-dimensional coordinates of each target in the camera coordinate system of the lifting target group are then marked. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. The pose matrices of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system during the loading operation were calculated respectively. This leads to the positional relationship between the lifting device coordinate system and the missile support coordinate system during the loading operation. In step S5: Calculate the loading position deviation 5. A missile loading posture deviation detection system based on binocular vision, characterized in that, include: Module M1, Calibration of Lifting Gear Data: Measure the pose relationship between the lifting gear target group and the lifting gear coordinate system, and record it in the camera controller; Module M2, calibrating missile bracket data: measuring the pose relationship between the missile bracket target group and the missile bracket coordinate system, and recording it in the camera controller; Module M3, Calibration and Loading Reference Posture: When the lifting device holds the missile module and positions it on the missile bracket, a binocular camera is used to simultaneously photograph the lifting device target group and the missile bracket target group. The camera controller calculates and records the relative posture of the lifting device coordinate system and the missile bracket coordinate system as a reference posture during missile loading operations. Module M4, Real-time pose detection during loading operation: During missile loading operation, a binocular camera is used to simultaneously photograph the lifting target group and the missile bracket target group, and the camera controller calculates the real-time pose of the lifting coordinate system and the missile bracket coordinate system. Module M5, Calculate the loading posture deviation: Using the camera controller, the posture deviation between the real-time posture of the spreader coordinate system and the reference posture is calculated based on the calibrated loading operation reference posture and the detected real-time loading posture.

6. The missile loading posture deviation detection system based on binocular vision according to claim 5, characterized in that, In module M1: a rigging coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the rigging coordinate system in the camera coordinate system at this time is denoted as... Let m be the number of targets in the lifting target group. Then, plot the three-dimensional coordinates of each target in the camera coordinate system. In module M2: the missile carrier coordinate system is measured and constructed in the camera coordinate system, and the pose matrix of the missile carrier coordinate system in the camera coordinate system at this time is denoted as... Let n be the number of targets in the missile carrier target group. Then, plot the three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system.

7. The missile loading posture deviation detection system based on binocular vision according to claim 6, characterized in that, In module M3, the three-dimensional coordinates of each target in the sling target group in the camera coordinate system are plotted. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. Calculate the reference relative pose of the sling coordinate system and the missile carrier coordinate system using the following steps: Module M3.1, Coordinate Normalization: The calibrated and measured values ​​of each target are normalized according to the following formula: This represents the normalized coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory. This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory. This indicates the three-dimensional coordinates of the k-th target in the camera coordinate system, given the pose relationship between the target group and the target group before leaving the factory. This indicates the normalized coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory; This indicates the three-dimensional coordinates of the i-th target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory; This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the missile module is positioned on the missile bracket by the lifting device before leaving the factory; This represents the normalized coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory. This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory. This indicates the three-dimensional coordinates of the k-th target in the missile carrier target group in the camera coordinate system, given the pose relationship between the missile carrier target group and the missile carrier coordinate system before leaving the factory. This indicates the normalized coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before leaving the factory; This indicates the three-dimensional coordinates of the j-th target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory; This indicates the three-dimensional coordinates of the kth target in the camera coordinate system when the lifting device positions the missile module on the missile bracket before it leaves the factory; Module M3.2 calculates the pose matrix H of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system. s H f Construct the following matrix from normalized coordinates. and Singular value decomposition yields: H s =U s S s V s T ,H f =U f S f V f T (3) U s U f Represents a left singular matrix; Σ s Σ f Represents a diagonal matrix; V s V f Represents a right singular matrix; Then, calculate the relative pose matrices of the current moment between the sling coordinate system, the missile bracket coordinate system, and the calibration time of module M1 and module M2. in: This allows us to obtain the pose matrix of the rigging coordinate system and the missile bracket coordinate system in the camera coordinate system at the current moment. Module M3.3, Calculate the loading reference pose: Calculate the pose relationship between the spreader coordinate system and the missile bracket coordinate system during loading alignment using the following formula, and denote it as the loading reference pose.

8. The missile loading posture deviation detection system based on binocular vision according to claim 7, characterized in that, In module M4: The three-dimensional coordinates of each target in the camera coordinate system of the lifting target group are then marked. The three-dimensional coordinates of each target in the missile carrier target group in the camera coordinate system are plotted. The pose matrices of the lifting device coordinate system and the missile bracket coordinate system in the camera coordinate system during the loading operation were calculated respectively. This leads to the positional relationship between the lifting device coordinate system and the missile support coordinate system during the loading operation. In module M5: calculate the loading position deviation.

9. A missile loading posture deviation detection equipment based on binocular vision, characterized in that, include: Binocular camera, target assembly with sling, target assembly with missile support, camera controller; It also includes the missile loading posture deviation detection system based on binocular vision as described in any one of claims 5 to 8; or, the missile loading posture deviation detection method based on binocular vision as described in any one of claims 1 to 4.

10. The missile loading posture deviation detection equipment based on binocular vision according to claim 9, characterized in that, The binocular camera is positioned such that its field of view simultaneously covers both the sling target group and the missile bracket target group during the loading process. The sling target group consists of a set of targets with a fixed positional relationship to the sling. The missile bracket target group consists of a set of targets with a fixed positional relationship to the missile bracket.

Images