A visual positioning feedback method for cable-net chain nodes
Through the visual positioning feedback method of cable network reflective surface chain nodes, the high-precision three-dimensional recognition and closed-loop control of cable network are realized through the system of industrial cameras and visual industrial control machines, and the slow parameter adjustment speed and artificial error of cable network in the existing technology are solved. The big problem improves the sensitivity and stability of the system.
Patent Information
- Application Number
- CN202111043485.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-07
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-09-07
AI Technical Summary
The existing cable-line mechanism has insufficient research on modular grouping, matching and adjustment in integrated radio telescopes and radar equipment, especially the lack of detailed research on local fine adjustment, resulting in conflicts in high precision, high sensitivity and low Manufacturing costs and the need for rapid construction.
Using a visual positioning feedback method of cable-net reflective surface chain nodes, through the systems of n industrial cameras, visual industrial control machines, motion servo actuators and main control machines, the visual industrial control machines are used to calibrate the camera's internal references to create a chain node matching template. The chain nodes are identified and positioned based on the monocular plane deformation matching algorithm to realize the closed-loop active displacement function.
It realizes high-precision three-dimensional identification and positioning of chain nodes, improves positioning speed and computing efficiency, reduces artificial errors, enhances the robustness and stability of the system, and meets the high-precision needs of large radio telescopes and radars.
Smart Images

Figure CN113920157B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the category of intelligent automatic identification and positioning methods, and specifically relates to a visual positioning feedback method for chain nodes of a cable net reflective surface. Background Art
[0002] In view of the great demand for radio telescopes and radar technology in national defense construction, it has become a national strategy to carry out integrated theoretical research on their systems, functions, structures and signal processing, as well as to tackle key technologies. Whether it is a large radio telescope or a shipborne / airborne radar, high-precision, high-sensitivity, modular, lightweight and integrated equipment are the core content of the research.
[0003] The modular design of the Square Kilometer Array radio telescope effectively solves the performance index problems of high sensitivity, high pointing accuracy, high pointing consistency, etc., but it has a strong conflict with the requirements of low manufacturing cost, rapid construction and easy maintenance; my country's "Sky Eye" system has successfully solved a large number of technical problems related to high precision and high sensitivity, and achieved good results. However, most of the existing mechanisms can achieve the overall morphological changes of the system, and there are few studies on more detailed local fine-tuning; there is still room for research and expansion in the modular grouping, matching and adjustment of existing cable-net mechanisms in integrated radio telescopes and radar equipment at home and abroad. Summary of the invention
[0004] The present invention aims to solve the deficiencies of the above-mentioned prior art and proposes a visual positioning feedback method for the chain nodes of a cable net reflective surface with high accuracy and simple structure, in order to form a closed-loop cable net with active displacement function, thereby realizing the tracking and detection of multiple targets.
[0005] The present invention adopts the following technical solutions to solve the technical problems:
[0006] The visual positioning feedback method of a cable net chain node of the present invention is characterized in that it is applied to a system composed of n industrial cameras and lenses, a visual industrial computer, a motion servo actuator and a master control computer, and is performed according to the following steps;
[0007] Step 1: n industrial cameras and lenses are arranged around the top and bottom of the cable net respectively; the camera internal parameters are calibrated using a visual industrial computer, the camera coordinate system is converted to the cable net reflection surface platform coordinate system, a matching template for each chain node is created, and the ROI area in the image captured by each camera is set, that is, the effective detection area of each chain node;
[0008] Step 2, when the motion servo actuator moves into place, the master control computer sends a trigger signal to the visual industrial control computer, and the visual industrial control computer is connected to n industrial cameras through the network port, and obtains the image data collected by the corresponding lens, so as to identify and locate the chain nodes in the image data by using the single-purpose plane deformation matching algorithm, and calculate the three-dimensional coordinates of each chain node in the camera coordinate system; and according to the calibration relationship between the camera and the platform coordinate system, the coordinates of each chain node in the camera coordinate system are converted to the platform coordinate system, and the coordinate space position of each node is obtained and sent to the master control computer;
[0009] Step 3, the master control machine compares the coordinate space position of each node with the theoretical value position. If the error amount is greater than the set amount, the master control machine sends the error compensation amount to the motion servo actuator for adjustment, and then returns to step 2 to execute sequentially until the coordinate space position of each node reaches the specified theoretical value position, thereby forming a target cable net surface; otherwise, it indicates that the current cable net surface is the target cable net.
[0010] The visual positioning feedback method of a cable net chain node described in the present invention is also characterized in that the calibration process of the camera intrinsic parameters in step 1 is as follows:
[0011] After installing the industrial cameras at each workstation, manually adjust the lens light source and focus ring to make the chain nodes clearly imaged within the field of view of each industrial camera. Then place the calibration plate at different positions and postures within the field of view of each industrial camera and collect multiple sets of images to calibrate each industrial camera. The calibration results of each industrial camera are six internal parameters. ,in, represents the calibrated focal length, represents the distortion coefficient, Indicates the calibrated camera pixel size in the x direction, Indicates the calibrated camera pixel size in the y direction, represents the x-coordinate of the calibrated image center point, Indicates the y coordinate of the calibrated image center point.
[0012] The conversion process from the camera coordinate system to the device coordinate system in step 1 is as follows:
[0013] First, a calibration plate with a white background and a rectangular frame pattern is made. After the industrial camera is fixedly installed, the coordinate system of the cable net reflective surface platform is used as the reference, where the origin of the cable net reflective surface platform coordinate system is located at the center of the bottom surface of the platform. According to the right-hand rule, the z-axis is vertically upward, and the o-xy plane is located on the bottom surface of the platform; the made calibration plate is placed within the field of view of the industrial camera and parallel to the o-xy plane of the cable net reflective surface platform coordinate system; the distance D from the bottom surface of the cable net reflective surface platform to the plane where the calibration plate is located is measured with a measuring tool, and then the industrial camera is controlled to collect the image of the calibration plate;
[0014] Get the image coordinates of the m vertices of the rectangular frame pattern from the calibration plate image , and establish the calibration plate coordinate system with the vertex in the center as the center coordinate, and obtain the coordinates of m points in the calibration plate coordinate system; using the correspondence between the m points, obtain the transfer relationship matrix from the calibration plate coordinate system to the n camera coordinate systems , and is used for coordinate system transformation.
[0015] The creation process of each chain node matching template in step 1 is as follows:
[0016] The chain node is set to be composed of a disk with a number of through holes, with the center point on the chain node as the origin, and two center lines passing through the origin and perpendicular to each other as the horizontal axis and the vertical axis, so as to construct a node coordinate system of the plane where the chain node is located;
[0017] According to the coordinates of several equally divided points on the circumference of the center point and the chain node in the node coordinate system and their coordinates in the image coordinate system collected by the industrial camera, the transfer relationship matrix between the node coordinate system of the plane where the chain node is located and the camera coordinate system is obtained. ;
[0018] According to the chain node image collected by the industrial camera, the rectangular area image where the chain node is located is selected and S-layer pyramid images are constructed. The contour information is extracted in each layer of the pyramid image. The contour information includes: contour point coordinates and normal vector information. Then, the method of ignoring partial polarity is used to create a matching template based on plane deformation, and the template is named according to the camera and node number.
[0019] The effective detection area of each chain node in step 1 is set according to the following process:
[0020] Determine the lowest and highest surfaces that the chain nodes of the cable net reflection surface can reach. When the cable net is located at the middle surface, control n cameras to collect chain node images to obtain n images, and set the ROI area of each chain node in each image, so that the chain nodes are within the set ROI area when the cable net reaches the highest and lowest surfaces.
[0021] The single-objective plane deformation matching algorithm in step 2 performs a hierarchical search based on shape-based template matching on S-layer pyramid images. The similarity between the matching template and each pixel in the search area is first calculated on the top-layer pyramid image, and then passed downward in sequence until the lowest-layer pyramid image is searched, thereby finding the pixel with the highest similarity in the search area as the best matching pixel.
[0022] The process of shape-based template matching is as follows:
[0023] ① Define the chain node matching template as the corresponding point set in the image coordinate system The direction vector associated with each point ,in, Represents the point set consisting of the edge contours of the chain nodes in the template image The i-th point in the template image, N represents the number of points on the edge contour of the chain node. Represents the i-th point The horizontal axis of Represents the i-th point The vertical coordinate of Represents contour points The normal vector at ;
[0024] ②Use Rotate the matching template to get the i-th point after the transformation , and get the transformed direction vector in, Represents a 2×2 rotation transformation matrix;
[0025] ③ Calculate each point in the ROI area of the chain node image collected in real time The direction vector at in, Represents the horizontal coordinate of the pixel point in the ROI area. Represents the vertical coordinate of the pixel point in the ROI area. express The normal vector at the point, Represents the normal vector in The direction of the component, Represents the normal vector in Directional component;
[0026] ④ Use formula (1) to calculate the midpoint between the matching template and the ROI area Similarity measure :
[0027] (1)
[0028] In formula (1), Indicates the midpoint of the ROI area The normal vector at ; Represents the midpoint of the template image The normal vector at ;
[0029] ⑤ Calculate all points in the ROI area according to step ④, and calculate the similarity s corresponding to the maximum value The point at is the best matching position.
[0030] The calculation process of the three-dimensional coordinates of each chain node in the camera coordinate system in step 2 is as follows:
[0031] Step a: Locate the image that matches the template image based on shape-based template matching Point coordinates The ROI area corresponding to the registration The image coordinates of the points , and then use equations (2) and (3) to calculate the affine transformation matrix :
[0032] (2)
[0033] (3)
[0034] In formula (2) and formula (3), represents the transformed point set, is the affine transformation matrix There are 9 unknown parameters in represents a fixed proportional constant;
[0035] Step b: Transform equation (3) to obtain equation (4):
[0036] (4)
[0037] Step c: Eliminate the scale of equation (4) to obtain equation (5):
[0038] (5)
[0039] Step d: Transform equation (5) to obtain equation (6):
[0040] (6)
[0041] Step e: Write equation (6) in the form of matrix and vector multiplication to obtain equation (7):
[0042] (7)
[0043] In formula (7): is the projection transformation matrix, and N points and Substituting into formula (7), we get: is the coefficient matrix of equation (6) corresponding to the i-th point, and has:
[0044] (8)
[0045] (9)
[0046] Step f: Multiply the point coordinates in the image by the projection transformation matrix Then multiply the transfer relationship matrix of the chain node in the template image in the camera coordinate system by , and finally multiply the transfer matrix from the calibrated camera coordinate system to the cable net reflection surface platform coordinate system on the left to obtain the three-dimensional coordinates of each chain node in the cable net reflection surface platform coordinate system.
[0047] In step 2, the obtained three-dimensional coordinates of the chain nodes in each camera coordinate system are respectively multiplied by the calibrated transfer relationship matrix , thus obtaining the three-dimensional coordinates of the center point of each chain node in the coordinate system of the cable net reflection surface platform.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] 1. The present invention obtains the spatial three-dimensional coordinates of the chain nodes through the deformation matching technology based on monocular vision, and uses the target itself as a template for matching calculation, achieving real-time calculation speed, overcoming the problem of slow calculation speed of the existing parabola parameter adjustment process in which the target point is attached and the three-dimensional coordinates are obtained by scanning with a manual handheld device, thereby improving the three-dimensional recognition and positioning speed of the chain nodes. At the same time, multiple 25-megapixel cameras are used for identification and positioning in different areas, with an accuracy of 0.25mm (1300mm / 5120), meeting the positioning requirements.
[0050] 2. The present invention uses multiple cameras to identify and locate chain nodes by area, uses multi-threading of a visual industrial computer to quickly capture the image of the entire parabola, and uses a deformable template matching algorithm for three-dimensional identification and positioning. Compared with the prior art method of scanning and reconstructing three-dimensional coordinates of chain nodes using manual handheld devices and obtaining them offline, the present invention greatly improves the computing efficiency, and automatically transmits data through a serial port, saving the workload of manual data entry and reanalysis, thereby improving work efficiency.
[0051] 3. The present invention treats the chain nodes as targets for crop template matching, and does not require manual labeling, thus effectively avoiding the errors caused by manual labeling. The chain nodes are identified and located using a relative similarity deformation template matching algorithm based on ignoring polarity, so that the identification and positioning algorithm is not affected by occlusion, confusion, and nonlinear illumination changes, thereby effectively improving the robustness and stability of the reflective surface chain node visual positioning feedback system. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 is a schematic diagram of the camera layout of the present invention;
[0053] FIG2a is a schematic diagram of the distribution of camera monitoring nodes of the present invention;
[0054] Figure 2b It is a schematic diagram of the field of view of the camera monitoring node of the present invention;
[0055] Figure 3 It is a visual positioning feedback system for the cable net reflective surface chain nodes of the present invention;
[0056] Figure 4 It is a schematic diagram of the identification and positioning algorithm flow of the present invention;
[0057] Figure 5 This is the recognition and positioning process based on deformable template matching of the present invention;
[0058] Figure 6 This is the camera intrinsic parameter calibration process of the present invention;
[0059] Figure 7 is a schematic diagram of the calibration plate image of the present invention;
[0060] Figure 8 The present invention is used to create a 7-point distribution on the node of the plane deformation matching template;
[0061] Fig. 9 It is the template image of the intermediate node of camera No. 5 of the present invention;
[0062] Fig.10 It is the outline and matching feature points in the node template created in camera No. 5 of the present invention;
[0063] Fig.11a It is the node template image created in cameras 1-4 of the present invention;
[0064] Fig.11b It is a schematic diagram of the outline and matching feature points in the node template created in cameras 1-4 of the present invention;
[0065] The numbers in the figure are: 1 is industrial camera No. 1 and lens; 2 is industrial camera No. 2 and lens; 3 is industrial camera No. 3 and lens; 4 is industrial camera No. 4 and lens; 5 is industrial camera No. 5 and lens; 6 is chain node; 7 is visual industrial computer; 8 is master control computer. DETAILED DESCRIPTION
[0066] In this embodiment, a visual positioning feedback method for a cable-net chain node is studied for the reflective surface support system of a large-aperture cable-net radio telescope and radar. The basic units of the cable-net radio telescope and radar are modularly designed to construct an active control support system for the reflective surface of a cable-net radar / radio telescope. The decision-making system is used to group and match the local and global module functions of the support system, and coordinated control is used to achieve tracking and detection of multiple targets by multiple modules.
[0067] The cable net is the supporting structure of the reflective surface unit. In order to meet the monitoring requirements of different distances and multiple targets, it can be changed from a spherical reference state to different parabolic states. That is, the cable net needs to have an active displacement function, which is the biggest difference from other cable net structures.
[0068] like Figure 3 As shown in the figure, the multi-information fusion positioning feedback method composed of visual recognition and node space coordinates is applied to a system consisting of n=5 industrial cameras and lenses, 1 visual industrial computer, 1 set of motion servo actuators and 1 master control computer. Figure 3 The overall system structure is given in FIG. 8 , where 8 represents the master control computer, 7 represents the visual industrial control computer, 1-5 represents the industrial camera and lens, and 6 represents the chain nodes on the cable net surface for identification and positioning. The motion servo actuator is composed of a motion controller, a servo driver, a servo motor, and an electric cylinder. In this embodiment, the Yaskawa servo control MP2000 control system architecture is adopted. Through the modular configuration of the MP2000 controller, the ladder diagram design and motion control program design of the MPE720 software are used to realize multi-servo multi-motor control, and the electric cylinder is used to move each node of the cable net to the required spatial position.
[0069] The master control machine uses the JHCTECH industrial computer produced by Shenzhen JHC Technology Development Co., Ltd. as the host computer controller. The industrial computer is equipped with an i5-6500 processor, 8GB memory, 256GB solid state drive, 4 RS232 serial ports, 2 network ports, 4 USB ports, and an aluminum alloy shell.
[0070] In terms of software design, the host computer software consists of three parts: human-computer interaction interface, serial port bottom-level driver and control calculation. The real-time data area displays the coordinate information of the node; the feedback data area is used to display the feedback data from the visual module to facilitate the adjustment of the node position; the jog operation area is used to operate a single node, such as rising or falling; the graphic display area is used to perform specific network operations. The visual feedback system is used to feedback the actual coordinate values of each node in real time; the motion controller module is used to convert the node coordinate data transmitted from the host computer into the actual control value of each electric cylinder; the electric cylinder group is mainly used to pull the nodes on the cable net to rise and fall vertically, and finally adjust the spatial position of the reflective surface node.
[0071] Specifically, according to the design requirements, the cable net reflective surface required for detection and target tracking is input into the master control machine, and converted into the spatial position coordinates of each node that constitutes the surface. The instructions are transmitted to the servo driver and servo motor through the motion controller, and then the electric cylinder executes the position of each spatial node to the required spatial coordinate position. However, during the execution process, it is affected by internal or external factors of the system, such as design, processing and installation errors and chain node coupling. There is a deviation between the actual spatial position and the theoretical position of each cable net reflective surface node. Therefore, a visual feedback system method is needed to locate the chain nodes to provide support for the formation of high-precision paraboloids. Specifically, if Figure 4 As shown, the method is carried out in the following steps;
[0072] Step 1: n industrial cameras and lenses are arranged above and below the cable net with a diameter of 5 meters; Figure 1 The distribution of the five cameras is given in the figure. The numbers 1-5 correspond to cameras 1-5 respectively. 6 represents the monitored cable net parabola node. 7 is the visual industrial computer, which is connected to the five cameras through the network port. Figure 1 According to the distribution pattern of the 29 chain nodes shown in the figure, 5 25-megapixel industrial cameras with 12mm focal length lenses are selected. Among them, the visual recognition camera uses MV-CH250-90GM. The industrial camera uses Gpixel's GMAX0505 CMOS chip with low noise, high resolution and excellent image. Uncompressed images are transmitted in real time through the Gigabit Ethernet interface, with a maximum frame rate of 4.5 fps. Based on the camera field of view of 1300mm×1300mm and the installation height of 1300mm, the camera target surface size is 12.8mm×12.8mm=2.5μm×5120 × 2.5μm×5120. The focal length calculation formula is:
[0073]
[0074] Therefore, a lens with a focal length of 12mm can be selected to meet the spatial layout and detection requirements of the device. Hikvision MVL-KF1224M-25MP lens is selected for comprehensive cost performance. According to the distribution law of the chain nodes of the cable net reflective surface, the distribution of the nodes identified and located by the five cameras is as follows: Figure 1 As shown in the figure, camera 5 is installed directly above the cable net, and cameras 1-4 are installed below the cable net. The distribution of camera monitoring nodes and the field of view are shown in the figure. Figure 2a and Figure 2b shown.
[0075] The chain node visual recognition and positioning process is divided into two parts: offline and online. The offline part uses the visual industrial computer to complete the calibration of the camera's internal parameters, convert the camera coordinate system to the device coordinate system, create matching templates for each chain node, and set the ROI area in the image captured by each camera, that is, the effective detection area of each chain node; the online part completes image acquisition, ROI area extraction, deformation template matching, and three-dimensional coordinate conversion.
[0076] In this embodiment, the visual industrial computer uses Win10 X64 system, develops the software system through Visual Studio 2019 programming tools, adopts C++ programming, and uses Qt for interface design. The system developed for chain node visual positioning has the functions of automatic operation at startup, image acquisition, and device configuration. It supports the increase of the number of cameras or nodes, and has the function of parameter setting and local saving. When the visual industrial computer is powered on, the visual software is automatically started, the software loads the configuration file, and switches to the automatic operation mode.
[0077] In specific implementation, the camera configuration can be used to align the nodes detected by each camera. After selecting the node, the ROI area of the node, the node angle rotation range, the scaling ratio relative to the template node, and the minimum score can be aligned. Different values can be configured for each node. Among them, the ROI area refers to the area where the node is located when the node moves up and down with the parabola; the parameters of node rotation angle, scaling ratio and score are mainly used for shape-based template matching parameter constraints. The larger the rotation angle range, the larger the scaling ratio, and the lower the score, the slower the matching speed and the higher the accuracy. Conversely, the faster the matching speed, the lower the accuracy. Specifically, if Figure 6 As shown in the figure, the calibration process of the camera intrinsic parameters is as follows:
[0078] After installing the industrial cameras at each workstation, manually adjust the lens light source and focus ring to make the chain nodes clearly imaged within the field of view of each industrial camera. Then place the calibration plate at different positions and postures within the field of view of each industrial camera and collect 13-15 sets of images for calibration of each industrial camera. The calibration results of each industrial camera are six internal parameters. ,in, represents the calibrated focal length, represents the distortion coefficient, Indicates the calibrated camera pixel size in the x direction, Indicates the calibrated camera pixel size in the y direction, represents the x-coordinate of the calibrated image center point, Indicates the y coordinate of the center point of the calibrated image. The calibration process is as follows Figure 6 shown.
[0079] In specific implementation, the conversion process from the camera coordinate system to the device coordinate system is as follows:
[0080] First, create a white base with Figure 7 The calibration plate of the rectangular frame pattern shown, wherein the calibration plate has a size of 500 mm × 300 mm, and the length and width of the rectangular frame are 400 mm × 250 mm respectively;
[0081] After the industrial camera is fixedly installed, take the center point of the bottom surface of the cable net reflective surface platform as the origin, and according to the right-hand rule, the z axis is vertically upward, and the o-xy plane is located on the bottom surface of the platform. Place the manufactured calibration plate within the field of view of the industrial camera and parallel to the o-xy plane of the cable net reflective surface platform coordinate system; use the measuring tool to measure the distance D from the bottom surface of the cable net reflective surface platform to the plane where the calibration plate is located, and then control the industrial camera to collect the image of the calibration plate;
[0082] Get the image coordinates of the 9 vertices of the rectangular frame pattern P1-P7 from the calibration plate image The calibration plate coordinate system is established with the vertex of center P5 as the center coordinate, and the coordinates of the 9 points in the calibration plate coordinate system are obtained; using the correspondence between the 9 points, the transfer relationship matrix from the calibration plate coordinate system to the camera coordinate system is obtained. , and is used for coordinate system transformation.
[0083] In the specific implementation, the creation process of each chain node matching template is as follows:
[0084] In this system, the three-dimensional coordinates of the center point of the chain node need to be calculated; the chain node is set to be composed of a disk with several through holes, the center point on the chain node is used as the origin, and the two center lines passing through the origin and perpendicular to each other are used as the horizontal axis and the vertical axis, so as to construct the node coordinate system of the plane where the chain node is located; Figure 8 As shown, in this embodiment, the chain node is composed of a disc with a diameter of 40 mm, and there are 7 through holes on the disc with high processing accuracy.
[0085] According to the coordinates of the center point and several equally divided points on the circumference of the chain node in the node coordinate system and their coordinates in the image coordinate system collected by the industrial camera, the relationship between the node coordinate system of the plane where the chain node is located and the camera coordinate system is obtained; the deformation matching algorithm can realize the three-dimensional spatial recognition and positioning of the known target by the monocular camera. In creating the plane deformation template, it is necessary to give the coordinates of at least 4 coplanar points in the chain node coordinate system and the coordinates of these points in the image. In this embodiment, the relationship between the plane coordinate system where the node is located and the camera coordinate system is constructed using 7 points on the node. The coordinates of the 7 points in the node coordinate system are as follows: Figure 8 And as shown in Table 1.
[0086] Table 1 7-point coordinates
[0087]
[0088] Taking the No. 4 chain node of camera No. 5 as an example, the process of building the template is explained. Fig. 9 and Fig.10 The image of the parabola in the middle position captured by camera No. 5 is used. According to the chain node image captured by the industrial camera, the rectangular area image where the chain node is located is manually selected and S=5 layers of pyramid images are constructed. The contour information is extracted from each layer of the pyramid image. The contour information includes: contour point coordinates and normal vector information, and a plane calibration deformation template is created. Since the chain node is affected by the ambient light, there will be changes in the black and white grayscale values in some areas. Therefore, when creating the template, a matching template based on plane deformation is created by ignoring some polarity, and the created template is named according to the camera and node number, and saved locally in dfm format.
[0089] Camera 5 is located directly above the device. The background image of the node in the field of view is black, so the features extracted for matching are mainly located on the circular surface of the node and the metal connector. Cameras 1-4 are located below the equipment. Affected by the ambient light, the black connecting lines are more obvious than the metal contour features. Therefore, the extracted contour features are as follows: Fig.11a and Fig.11b shown.
[0090] In specific implementation, the effective detection area of each chain node is set according to the following process:
[0091] Since the device moves within a certain spatial range, the cable net reflection surface platform controls the lowest and highest surfaces that the chain nodes can reach. When the cable net is located at the middle surface, n=5 cameras are controlled to collect n=5 chain node images, and the ROI area of each chain node image is manually set so that the chain nodes are within the set ROI area when the cable net reaches the highest and lowest surfaces, thereby improving the accuracy and calculation speed of the matching algorithm.
[0092] Step 2. When the motion servo actuator moves into place, the master control computer sends a trigger signal to the visual industrial computer. The visual industrial computer is connected to n=5 industrial cameras through the network port. When the visual industrial computer receives the request signal "aa 01 0d" sent by the master control through the serial port, it obtains the image data collected by the 5 corresponding lenses through multi-threading, and then uses the single-purpose plane deformation matching algorithm to identify and locate the chain nodes in the image data, and calculate the three-dimensional coordinates of the 29 chain nodes in the camera coordinate system; and according to the calibration relationship between the camera and the platform coordinate system, the coordinates of each chain node in the camera coordinate system are converted to the platform coordinate system, and the coordinate space positions of the 29 nodes are obtained and sent to the master control computer through the serial port and displayed on the interface;
[0093] In specific implementation, the single-purpose plane deformation matching algorithm first conducts a comprehensive search based on shape-based template matching on S=5 layers of pyramid images to determine candidate matching objects; then it tracks through the resolution hierarchy structure, compensates for deformation through template deformation during tracking, matches within the region, calculates similarity, performs threshold segmentation, and extracts local extreme values. As the pyramid level decreases, the template deformation will continue to increase, and larger distortions will be compensated in higher layers of the pyramid, while smaller distortions will be compensated in the bottom layer of the pyramid. In order to more accurately calculate the deformation in a certain layer of the pyramid, the template is split into several parts for local search.
[0094] In this embodiment, a 5-layer image pyramid is used for hierarchical search. The similarity between the matching template and each pixel in the search area is first calculated in the top-level pyramid image, and then passed downward in sequence until the lowest-level pyramid image is searched, thereby finding the pixel with the highest similarity in the search area as the best matching pixel.
[0095] Among them, Figure 5 As shown, the process of shape-based template matching is as follows:
[0096] ① Define the chain node matching template as the corresponding point set in the image coordinate system The direction vector associated with each point ,in, Represents the point set consisting of the edge contours of the chain nodes in the template image The i-th point in the template image, N represents the number of points on the edge contour of the chain node. Represents the i-th point The horizontal axis of Represents the i-th point The vertical coordinate of Represents contour points The normal vector at ;
[0097] ②Use Rotate the matching template to get the i-th point after the transformation , and get the transformed direction vector in, Represents a 2×2 rotation transformation matrix;
[0098] ③ Calculate each point in the ROI area of the chain node image collected in real time The direction vector at in, Represents the horizontal coordinate of the pixel point in the ROI area. Represents the vertical coordinate of the pixel point in the ROI area. express The normal vector at the point, Represents the normal vector in The direction of the component, Represents the normal vector in Directional component;
[0099] ④ Use formula (1) to calculate the midpoint between the matching template and the ROI area Similarity measure :
[0100] (1)
[0101] (2)
[0102] In formula (1), Indicates the midpoint of the ROI area The normal vector at ; Represents the midpoint of the template image The normal vector at; the similarity selected in the present invention is the relative quantity of formula (2), which has the advantages of not being affected by occlusion, confusion and nonlinear illumination changes.
[0103] The present invention performs template recognition and positioning by shape template matching based on hierarchical search of an image pyramid. From the perspective of the hierarchical search and the process based on shape template matching. Only the position most similar to the target object can be found in the image, and the three-dimensional coordinates of the target cannot be calculated. The chain nodes change with the change of the parabola, and there is a change in the three-dimensional posture of the space. To calculate the three-dimensional posture of the node in the camera coordinate system, it is necessary to obtain the image coordinates of the target object point in the image and match the corresponding points on the node. When the match is successful, the three-dimensional posture of the planar target object in the camera coordinate system can be calculated by adopting the following plane space three-dimensional projection transformation principle.
[0104] The calculation process of the three-dimensional coordinates of each chain node in the camera coordinate system is as follows:
[0105] Step a: Based on the detected = Image coordinates of 7 points , and the template image during calibration =7 point coordinates , use equations (3) and (4) to calculate the affine transformation matrix :
[0106] (3)
[0107] (4)
[0108] Step b: Use equation (3) to transform the equation:
[0109] (4)
[0110] Step c: Use formula (4) to eliminate the scale:
[0111] (5)
[0112] Step d: Transform equation (5) to obtain:
[0113] (6)
[0114] Step e: Write equation (6) in the form of matrix and vector multiplication to obtain equation (7):
[0115] (7)
[0116] In formula (7): is the projection transformation matrix, is the coefficient matrix of equation (6) corresponding to the i-th point, and has:
[0117] (8)
[0118] (9)
[0119] N points and Substituting into equation (7), solving the equation can calculate the projection transformation matrix .
[0120] Step f: Multiply the point coordinates in the image by the projection transformation matrix Then multiply the transfer relationship matrix of the chain node in the template image in the camera coordinate system by , and finally multiply the transfer matrix from the calibrated camera coordinate system to the cable net reflection surface platform coordinate system on the left to obtain the three-dimensional coordinates of each chain node in the cable net reflection surface platform coordinate system.
[0121] Then, the obtained three-dimensional coordinates of the chain nodes in each camera coordinate system are multiplied by the calibrated transfer relationship matrix , thus obtaining the three-dimensional coordinates of the center point of each chain node in the coordinate system of the cable net reflection surface platform.
[0122] Step 3: The master control computer compares the coordinate space position of each node with the theoretical execution position. If the error is greater than the set amount, the master control computer sends the error compensation amount to the motion servo actuator for adjustment, and then returns to step 2 to execute sequentially until the coordinate space position of each node reaches the specified theoretical position, thereby forming the target cable net surface; otherwise, it means that the current cable net surface is the target cable net.
[0123] In summary, the present invention is a non-contact real-time identification and positioning method, which provides a more accurate measurement means and closed-loop control method for the cable net active reflection surface platform and active displacement function, and can realize high-precision automatic adjustment of any cable net parabola. The system collects the actual spatial position of the cable net reflection surface chain node through an industrial camera, and outputs the actual node coordinates after identification and positioning calculation by a visual industrial control computer, and then compares the theoretical node coordinates with the master control computer, and then adjusts the nodes that exceed the error allowable value through a motion servo actuator to realize closed-loop control, and finally ensures that the cable net reflection surface node is in the theoretical position to form the required reflection surface, and realizes the active displacement function of the cable net reflection surface, thereby solving the cable net active reflection surface system, which can be changed from a spherical reference state to different parabolic states, as well as a local parabolic state; solves the cable net reflection surface closed-loop control system, and the cable net antenna can reach a diameter of tens of meters to hundreds of meters, and the accuracy can currently be controlled at the millimeter level, which is an ideal form to meet the high-precision, light-weight, and large-diameter requirements of large-scale deployable antennas; this method has the advantages of simple structure, easy implementation, low cost, and convenient management.
Claims
1. A visual positioning feedback method for a cable net chain node, characterized in that It is applied to a system consisting of n industrial cameras and lenses, a visual industrial computer, a motion servo actuator and a master control computer, and is carried out in the following steps; Step 1: n industrial cameras and lenses are arranged around the top and bottom of the cable net respectively; the camera internal parameters are calibrated using a visual industrial computer, the camera coordinate system is converted to the cable net reflection surface platform coordinate system, a matching template for each chain node is created, and the ROI area in the image captured by each camera is set, that is, the effective detection area of each chain node; The creation process of each chain node matching template in step 1 is as follows: The chain node is set to be composed of a disk with a number of through holes, with the center point on the chain node as the origin, and two center lines passing through the origin and perpendicular to each other as the horizontal axis and the vertical axis, so as to construct a node coordinate system of the plane where the chain node is located; According to the coordinates of several equally divided points on the circumference of the center point and the chain node in the node coordinate system and their coordinates in the image coordinate system collected by the industrial camera, the transfer relationship matrix between the node coordinate system of the plane where the chain node is located and the camera coordinate system is obtained. ; According to the chain node image collected by the industrial camera, the rectangular area image where the chain node is located is selected and S-layer pyramid images are constructed, and contour information is extracted from each layer of the pyramid image. The contour information includes: contour point coordinates and normal vector information, and then a matching template based on plane deformation is created by ignoring partial polarity, and the template is named according to the camera and node number; Step 2, when the motion servo actuator moves into place, the master control computer sends a trigger signal to the visual industrial control computer, and the visual industrial control computer is connected to n industrial cameras through the network port, and obtains the image data collected by the corresponding lens, so as to identify and locate the chain nodes in the image data by using the single-purpose plane deformation matching algorithm, and calculate the three-dimensional coordinates of each chain node in the camera coordinate system; and according to the calibration relationship between the camera and the platform coordinate system, the coordinates of each chain node in the camera coordinate system are converted to the platform coordinate system, and the coordinate space position of each node is obtained and sent to the master control computer; The single-purpose plane deformation matching algorithm in step 2 performs a hierarchical search based on shape-based template matching on S-layer pyramid images, firstly calculating the similarity between the matching template and each pixel in the search area in the top-layer pyramid image, and then sequentially passing it downward until the lowest-layer pyramid image is searched, thereby finding the pixel with the highest similarity in the search area as the best matching pixel; The process of shape-based template matching is as follows: ① Define the chain node matching template as the corresponding point set in the image coordinate system The direction vector associated with each point ,in, Represents the point set consisting of the edge contours of the chain nodes in the template image The i-th point in the template image, N represents the number of points on the edge contour of the chain node. Represents the i-th point The horizontal axis of Represents the i-th point The vertical coordinate of Represents contour points The normal vector at ; ②Use Rotate the matching template to get the i-th point after the transformation , and get the transformed direction vector ;in, Represents a 2×2 rotation transformation matrix; ③ Calculate each point in the ROI area of the chain node image collected in real time The direction vector at ;in, Represents the horizontal coordinate of the pixel point in the ROI area. Represents the vertical coordinate of the pixel point in the ROI area. express The normal vector at the point, Represents the normal vector in The direction of the component, Represents the normal vector in Directional component; ④ Use formula (1) to calculate the midpoint between the matching template and the ROI area Similarity measure : (1) In formula (1), Indicates the midpoint of the ROI area The normal vector at ; Represents the midpoint of the template image The normal vector at ; ⑤ Calculate all points in the ROI area according to step ④, and calculate the similarity s corresponding to the maximum value The point at is the best matching position; Step 3, the master control machine compares the coordinate space position of each node with the theoretical value position. If the error amount is greater than the set amount, the master control machine sends the error compensation amount to the motion servo actuator for adjustment, and then returns to step 2 to execute sequentially until the coordinate space position of each node reaches the specified theoretical value position, thereby forming a target cable net surface; otherwise, it indicates that the current cable net surface is the target cable net.
2. The visual positioning feedback method of a cable net chain node according to claim 1, characterized in that: The calibration process of the camera intrinsic parameters in step 1 is as follows: After installing the industrial cameras at each workstation, manually adjust the lens light source and focus ring to make the chain nodes clearly imaged within the field of view of each industrial camera. Then place the calibration plate at different positions and postures within the field of view of each industrial camera and collect multiple sets of images to calibrate each industrial camera. The calibration results of each industrial camera are six internal parameters. ,in, represents the calibrated focal length, represents the distortion coefficient, Indicates the calibrated camera pixel size in the x direction, Indicates the calibrated camera pixel size in the y direction, represents the x-coordinate of the calibrated image center point, Indicates the y coordinate of the calibrated image center point.
3. The visual positioning feedback method of a cable net chain node according to claim 1, characterized in that: The conversion process from the camera coordinate system to the device coordinate system in step 1 is as follows: First, a calibration plate with a white background and a rectangular frame pattern is made. After the industrial camera is fixedly installed, the coordinate system of the cable net reflective surface platform is used as the reference, where the origin of the cable net reflective surface platform coordinate system is located at the center of the bottom surface of the platform. According to the right-hand rule, the z-axis is vertically upward, and the o-xy plane is located on the bottom surface of the platform; the made calibration plate is placed within the field of view of the industrial camera and parallel to the o-xy plane of the cable net reflective surface platform coordinate system; the distance D from the bottom surface of the cable net reflective surface platform to the plane where the calibration plate is located is measured with a measuring tool, and then the industrial camera is controlled to collect the image of the calibration plate; Get the image coordinates of the m vertices of the rectangular frame pattern from the calibration plate image , and establish the calibration plate coordinate system with the vertex in the center as the center coordinate, and obtain the coordinates of m points in the calibration plate coordinate system; using the correspondence between the m points, obtain the transfer relationship matrix from the calibration plate coordinate system to the n camera coordinate systems , and is used for coordinate system transformation.
4. The visual positioning feedback method of a cable net chain node according to claim 1, characterized in that: The effective detection area of each chain node in step 1 is set according to the following process: Determine the lowest and highest surfaces that the chain nodes of the cable net reflection surface can reach. When the cable net is located at the middle surface, control n cameras to collect chain node images to obtain n images, and set the ROI area of each chain node in each image, so that the chain nodes are within the set ROI area when the cable net reaches the highest and lowest surfaces.
5. The visual positioning feedback method of a cable net chain node according to claim 1, characterized in that: The calculation process of the three-dimensional coordinates of each chain node in the camera coordinate system in step 2 is as follows: Step a: Locate the image that matches the template image based on shape-based template matching Point coordinates The ROI area corresponding to the registration The image coordinates of the points , and then use equations (2) and (3) to calculate the affine transformation matrix : (2) (3) In formula (2) and formula (3), represents the transformed point set, is the affine transformation matrix There are 9 unknown parameters in represents a fixed proportional constant; Step b: Transform equation (3) to obtain equation (4): (4) Step c: Eliminate the scale of equation (4) to obtain equation (5): (5) Step d: Transform equation (5) to obtain equation (6): (6) Step e: Write equation (6) in the form of matrix and vector multiplication to obtain equation (7): (7) In formula (7): is the projection transformation matrix, and N points and Substituting into formula (7), we get: is the coefficient matrix of equation (6) corresponding to the i-th point, and has: (8) (9) Step f: Multiply the point coordinates in the image by the projection transformation matrix Then multiply the transfer relationship matrix of the chain node in the template image in the camera coordinate system by , and finally multiply the transfer matrix from the calibrated camera coordinate system to the cable net reflection surface platform coordinate system on the left to obtain the three-dimensional coordinates of each chain node in the cable net reflection surface platform coordinate system.
6. The visual positioning feedback method of a cable net chain node according to claim 5, characterized in that: In step 2, the obtained three-dimensional coordinates of the chain nodes in each camera coordinate system are respectively multiplied by the calibrated transfer relationship matrix , thus obtaining the three-dimensional coordinates of the center point of each chain node in the coordinate system of the cable net reflection surface platform.
