An autonomous aiming method and system based on machine vision

Abstract

The application discloses an autonomous aiming method and system based on machine vision and belongs to the technical field of machine vision. The method comprises the following steps: collecting a target image with a double-layer structure of outer black and inner white by an embedded visual perception module, obtaining an effective target combination through multi-stage feature screening and spatial relationship constraint verification; extracting a black outer frame pixel centroid coordinate and a white inner frame vertex average center coordinate, calculating an adaptive weight coefficient according to the variance of the two, and performing weighted fusion to obtain a target center coordinate; a master control unit drives a two-dimensional holder aiming according to the pixel deviation of the target center coordinate and the image center coordinate through PID control to solve the holder control amount; and the master control unit executes a patrol control thread in parallel to control a mobile platform to autonomously drive along a preset track. The application improves the rectangular target recognition accuracy of a low-cost embedded platform, the center positioning precision under the condition of perspective distortion, and the aiming tracking robustness in the dynamic driving process.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and more specifically to an autonomous aiming method and system based on machine vision. Background Technology

[0002] Machine vision-based autonomous aiming technology has a clear application need in embedded robot competition platforms such as intelligent car racing. These competitions typically include a mobile aiming event: participating car models must autonomously drive along an indoor track, performing real-time identification, aiming, and laser pointing operations on rectangular color targets placed beside the track to complete a combined score based on time and accuracy.

[0003] In close-range visual servoing operations on embedded mobile platforms in competition scenarios, the following problems are encountered: 1) Robust recognition of rectangular targets in complex backgrounds. Competition venues contain complex lighting conditions, track background interference, and geometrically similar objects, such as track boundary cones and other rectangular markers. Traditional feature matching or template matching methods are sensitive to changes in lighting and are prone to misdetecting non-target interference objects as targets. In recent years, although deep learning target detection algorithms represented by the YOLO series have advantages in accuracy, their models have high computational load and high memory consumption, making it difficult to meet the real-time processing requirements on the low-power microcontrollers commonly used in competitions. 2) Centering deviation of rectangular targets under large viewing angle conditions. In competition scenarios, there is often a close-range, large-view relative pose between the car model and the target. Perspective distortion causes regular rectangles in space to degenerate into irregular quadrilaterals in the image plane. Existing visual servoing methods often directly use the centroid of connected components or the center of the bounding box as the target center. This estimation method, which uses pixel area as a weight, introduces systematic bias under distortion. The pixel area on the side closer to the lens is larger, exerting a stronger pull on the centroid, causing the calculated center to deviate from the physical center of the target, directly affecting aiming accuracy. For rectangular targets, although distortion can theoretically be corrected by extracting the geometric center from the four vertices, vertex extraction is sensitive to image noise and lacks stability, affecting aiming accuracy. Furthermore, the path tracking and gimbal aiming control of the mobile platform are usually designed as two independent control loops, lacking an effective coordination mechanism. This causes additional motion of the target in the image. If the timing of the two control loops is not properly matched, oscillations or tracking failures can easily occur, making it difficult to achieve continuous operation of "tracking, recognition, and aiming simultaneously." Summary of the Invention

[0004] To address the aforementioned issues, this invention provides an autonomous aiming method and system based on machine vision. It improves the recall and accuracy of rectangular target identification in complex backgrounds through multi-level feature filtering and spatial relationship constraint verification. It suppresses center positioning deviation due to perspective distortion through adaptive weighted fusion of the centroid of the black outer frame and the vertex center of the white inner frame. Furthermore, it enhances the robustness of aiming and tracking under dynamic driving conditions through parallel collaboration between the visual servo control thread and the tracking control thread. This enables collaborative operation of precise target aiming and autonomous movement on a low-cost embedded platform.

[0005] In a first aspect, the technical solution of the present invention provides an autonomous aiming method based on machine vision, comprising the following steps: SS1 is an image containing a target acquired by an embedded visual perception module. The target has a double-layer structure with a black outer edge and a white inner edge. SS2, within the embedded visual perception module, extracts the black and white connected regions in the image respectively. After multi-level feature filtering and spatial relationship constraint verification, it obtains a matching black outer frame and white inner frame as an effective target combination. SS3, within the embedded visual perception module, extracts the pixel centroid coordinates of the black outer frame and the average vertex center coordinates of the white inner frame in the effective target combination. It calculates an adaptive weight coefficient based on the variance of the pixel centroids of the black outer frame and the variance of the average vertex center of the white inner frame. Based on the adaptive weight coefficient, it performs a weighted fusion of the pixel centroid coordinates and the average vertex center coordinates to obtain the target center coordinates. SS4 transmits the target center coordinates to the main control unit, which calculates the pixel deviation between the target center coordinates and the image center coordinates, inputs the pixel deviation into the PID controller, and calculates the control quantities of the gimbal azimuth axis and pitch axis. SS5 drives the two-dimensional gimbal to rotate according to the control quantity, so that the laser spot converges towards the center of the target. When the pixel deviation is less than the preset threshold, the laser pointer is triggered to complete the aiming. In SS6, while executing the visual servo control threads of steps SS1 to SS5, the main control unit executes the mobile platform tracking control thread in parallel. The tracking control thread collects detection signals from a multi-channel grayscale sensor array located at the bottom of the vehicle body, adjusts the differential speed of the left and right drive wheels in real time, and controls the mobile platform to drive autonomously along a preset trajectory.

[0006] Secondly, the technical solution of the present invention provides an autonomous aiming system based on machine vision, comprising: An embedded visual perception module is used to acquire images containing targets. The targets have a double-layer structure with a black outer border and a white inner border. The embedded visual perception module runs a target recognition algorithm, which extracts black and white connected regions in the image. After multi-level feature filtering and spatial relationship constraint verification, a matching black outer frame and a white inner frame are obtained as a valid target combination. The pixel centroid coordinates of the black outer frame and the average vertex center coordinates of the white inner frame are extracted from the valid target combination. The main control unit is communicatively connected to the embedded visual perception module. The main control unit includes a target center calculation module, a visual servo control module, and a tracking control module, wherein: The target center calculation module is used to receive the centroid coordinates of the black outer frame pixels and the average center coordinates of the white inner frame vertices output by the embedded visual perception module. It calculates the adaptive weight coefficient based on the variance of the centroid of the black outer frame pixels and the variance of the average center of the white inner frame vertices. Based on the adaptive weight coefficient, it performs weighted fusion of the pixel centroid coordinates and the average center coordinates of the vertices to obtain the target center coordinates. The visual servo control module is used to calculate the pixel deviation between the target center coordinates and the image center coordinates. The pixel deviation is input into the PID controller to calculate the azimuth axis control quantity and pitch axis control quantity of the two-dimensional gimbal. The tracking control module is used to collect detection signals from a multi-channel grayscale sensor array located at the bottom of the mobile platform vehicle body, adjust the differential speed of the left and right drive wheels in real time, and generate tracking control quantities. Among them, the visual servo control module and the tracking control module run in parallel threads in the main control unit, enabling the system to autonomously move along a preset trajectory while performing aiming. The two-dimensional gimbal is electrically connected to the main control unit and includes an azimuth axis stepper motor and a pitch axis stepper motor. It is used to receive azimuth axis control signals and pitch axis control signals and adjust the gimbal attitude accordingly. A laser pointer is fixedly mounted on the top of a two-dimensional pan-tilt unit. Its optical axis is parallel to the optical axis of the embedded vision perception module and the distance between them is fixed. It is used to emit lasers. The mobile platform is a differential wheel chassis used to carry the main control unit, embedded vision perception module, two-dimensional gimbal and laser pointer. The mobile platform includes left and right drive wheels and drive modules that drive the left and right drive wheels respectively. The drive modules adjust the differential speed of the left and right drive wheels in response to the tracking control quantity, so that the mobile platform can move autonomously along a preset trajectory.

[0007] As can be seen from the above technical solutions, this application has the following advantages: After acquiring an image containing targets, the embedded visual perception module directly extracts the black and white connected regions within the image. Through multi-level filtering including geometric feature filtering, pixel density filtering, relative proportion verification, and spatial relationship constraint verification, a matching black outer frame and white inner frame are obtained as the effective target combination. This approach, through multi-level constraints based on geometric features, pixel density, and spatial positional relationships, improves the recall and accuracy of rectangular target recognition under complex lighting and background interference without requiring a deep learning model. It also reduces the false detection and false negative rates caused by sudden changes in lighting and interference from geometrically similar objects, thereby enhancing the system's operational stability and reliability under varying competition conditions and preventing recognition jitter or target loss. To address the complementary anti-distortion capabilities of two types of features—the centroid of the black outer frame and the vertex center of the white inner frame—under perspective distortion, a variance-based dynamic weight allocation is designed. This involves calculating adaptive weight coefficients based on the variance of the centroid of the black outer frame pixels and the variance of the average center of the white inner frame vertices, and then weighted and fused to obtain the target center coordinates. When the viewing angle is skewed, leading to increased asymmetric perspective distortion, the variance of the white vertex coordinates increases, and the system automatically reduces its trust in white features, relying instead on the more stable black centroid. When the viewing angle is centered and the distortion is mild, the system automatically increases the weight of white features, utilizing their geometric accuracy to improve center estimation accuracy. This achieves an adaptive balance between geometric accuracy and positional stability in center estimation, effectively suppressing center jumps caused by the sensitivity of the single-vertex method to image noise, improving the center positioning accuracy and smoothness under perspective distortion conditions, making the aiming control output more continuous and smooth, and avoiding gimbal oscillations caused by center estimation jitter. While the visual servo control thread performs target recognition, center estimation, and PID aiming control, the main control unit executes the mobile platform tracking control thread in parallel. The visual servo control thread and the tracking control thread run independently in parallel, without blocking each other in timing, but physically coupled through the same mobile platform. When the tracking control thread adjusts the platform's driving posture, causing a change in camera pose, this disturbance is automatically compensated for by the visual servo control thread through closed-loop feedback, without manual intervention or mode switching. This dual-thread parallel structure, combined with visual closed-loop feedback, effectively improves the robustness and anti-interference capability of the mobile platform's aiming and tracking under dynamic driving conditions. Even when the mobile platform undergoes drastic changes in attitude such as turning, acceleration, or deceleration, it can maintain continuous target lock, ensuring that aiming accuracy is not affected by platform motion coupling. Attached Figure Description

[0008] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic block diagram of an autonomous aiming system based on machine vision, provided for an embodiment of the present invention.

[0010] Figure 2 This is a schematic diagram of the target structure.

[0011] Figure 3 This is a flowchart illustrating an autonomous aiming method based on machine vision, provided as an embodiment of the present invention. Detailed Implementation

[0012] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The various terms used in this specification are merely for describing specific embodiments and do not constitute a limitation thereof.

[0014] Figure 1 This is a schematic block diagram illustrating the structure of a machine vision-based autonomous aiming system provided in an embodiment of the present invention. In this embodiment, the machine vision-based autonomous aiming system can be divided into multiple functional modules according to the functions it performs, such as... Figure 1 As shown. The functional modules include: an embedded vision perception module, a main control unit, a 2D gimbal, a laser pointer, and a mobile platform.

[0015] An embedded visual perception module is used to acquire images containing a target. The target has a double-layered structure with a black outer border and a white inner area. In one optional implementation, the target is an A4-sized rectangular piece of paper, with its outer border outlined by 1.8cm wide black tape, and the inner surface of the paper is white. A red dot is marked on the target's center. Figure 2 As shown.

[0016] The embedded visual perception module internally runs a target recognition algorithm. This algorithm extracts black and white connected regions from the image, and after multi-level feature filtering and spatial relationship constraint verification, obtains matching black outer frames and white inner frames as valid target combinations. It then extracts the pixel centroid coordinates of the black outer frame and the average vertex center coordinates of the white inner frame from the valid target combinations. In an optional implementation, the embedded visual perception module is an OpenMVH7 Plus embedded machine vision module, which integrates an STM32H7 processor with a main frequency of up to 400MHz. It supports an image acquisition frame rate of no less than 60fps at QVGA (320×240) resolution. MicroPython scripts can be run directly on the module to implement image processing algorithms such as color block recognition and edge detection. It communicates with the main control unit via a UART serial interface, and the communication data frames include the target center coordinates, confidence level, and status flags.

[0017] The main control unit is communicatively connected to the embedded visual perception module. The main control unit includes a target center calculation module, a visual servo control module, and a trajectory control module.

[0018] The target center calculation module receives the centroid coordinates of the black outer frame pixels and the average center coordinates of the white inner frame vertices from the embedded visual perception module. It calculates adaptive weight coefficients based on the variance of the centroids of the black outer frame pixels and the variance of the average center coordinates of the white inner frame vertices. Based on the adaptive weight coefficients, it performs weighted fusion of the pixel centroid coordinates and the average center coordinates of the vertices to obtain the target center coordinates.

[0019] The visual servo control module is used to calculate the pixel deviation between the target center coordinates and the image center coordinates. The pixel deviation is input into the PID controller to calculate the azimuth axis control quantity and pitch axis control quantity of the two-dimensional gimbal.

[0020] The tracking control module is used to collect detection signals from a multi-channel grayscale sensor array located at the bottom of the mobile platform vehicle body, adjust the differential speed of the left and right drive wheels in real time, and generate tracking control quantities.

[0021] The visual servo control module and the tracking control module run in parallel threads in the main control unit, enabling the system to autonomously move along a preset trajectory while performing aiming.

[0022] In one optional implementation, the main control unit is a TI MSPM0G3507 microcontroller, which is based on the Arm Cortex-M0+ core with a main frequency of up to 80MHz. It integrates an on-chip math accelerator (including a CORDIC coprocessor and a hardware divider), which can efficiently complete floating-point operations in visual processing and iterative calculations in PID control. It also integrates multiple UART, PWM and GPIO interfaces, which can simultaneously carry out serial communication with the vision module, pulse drive of the gimbal stepper motor, parallel acquisition of grayscale sensor and on / off control of laser pointer.

[0023] In one optional implementation, the multiple grayscale sensors are numbered S1 to S8 from left to right. S4 and S5 are located on the vehicle's central axis and are used to detect the tracking centering state; S3 and S6 are located on both sides of the central axis and are used for yaw detection and attitude correction; S1 is used for right-angle turn triggering; and S2 and S7 are used for redundant detection. An additional grayscale sensor is installed at the left wheel of the vehicle to assist in right-angle turn determination.

[0024] The 2D gimbal is electrically connected to the main control unit and includes an azimuth axis stepper motor and a pitch axis stepper motor. These motors receive azimuth and pitch axis control signals and adjust the gimbal's attitude accordingly. The stepper motors control the rotation angle through pulse frequency and quantity, resulting in high positioning resolution and low cumulative error, making them suitable for continuous fine-tuning and rapid response in visual servoing. In an optional implementation, the 2D gimbal uses two two-phase stepper motors to drive the azimuth and pitch axes respectively, achieving precise two-dimensional adjustment of the laser emission direction.

[0025] A laser pointer is fixedly mounted on the top of a 2D gimbal, with its optical axis parallel to and at a fixed distance from the optical axis of the embedded visual perception module, used to emit laser light. In one optional embodiment, the laser pointer is a 635nm laser pointer with a spot diameter no greater than 0.5cm, providing sufficient energy concentration within a 50cm projection distance to meet aiming and pointing requirements. With the coaxial mounting configuration of "eye in hand," the visual optical axis and the laser optical axis remain parallel and at a constant distance, thus establishing a rigid mapping from image pixel coordinates to the laser pointing angle, avoiding the complex hand-eye calibration process.

[0026] The mobile platform is a differential wheel chassis used to carry the main control unit, embedded vision perception module, 2D gimbal, and laser designator. The mobile platform includes left and right drive wheels and drive modules that drive the left and right drive wheels respectively. The drive modules adjust the differential speed of the left and right drive wheels in response to the tracking control quantity, enabling the mobile platform to move autonomously along a preset trajectory.

[0027] In one optional implementation, the mobile platform employs a three-wheel differential drive structure: two stepper motors drive the left and right drive wheels respectively via an L298N dual H-bridge drive module, with omnidirectional wheels providing support at the front. Straight-line and steering control are achieved through the differential speed of the left and right wheels. The power management module converts the 12V input voltage into three DC outputs: 6V (stepper motor drive), 5V (sensors and OpenMV), and 3.3V (main control chip), providing independent power to each module.

[0028] Figure 3 This is a flowchart illustrating a machine vision-based autonomous aiming method according to an embodiment of the present invention. The machine vision-based autonomous aiming method provided in this embodiment is executed by a machine vision-based autonomous aiming system, which runs on an embedded hardware platform. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted or combined.

[0029] like Figure 1 As shown, the method includes the following steps.

[0030] SS1 is an image containing a target acquired by an embedded visual perception module. The target has a double-layer structure with a black outer edge and a white inner edge.

[0031] Specifically, the target mentioned in this step is an A4-sized rectangular piece of paper, with its outer edge outlined by 1.8cm wide black tape. The inside is white paper, and the target center is marked with a red dot. Figure 2 As shown.

[0032] SS2, within the embedded visual perception module, extracts the black and white connected regions from the image, and through multi-level feature filtering and spatial relationship constraint verification, obtains a matching black outer frame and a white inner frame as an effective target combination.

[0033] SS21, image preprocessing, includes setting the acquired image to QVGA resolution, adjusting exposure parameters to improve image contrast and reduce brightness, and clarifying the boundary between the black border and the white paper surface.

[0034] After preprocessing, the grayscale distribution of the black borders in the image tends to be concentrated, providing a stable input for subsequent color block segmentation.

[0035] SS22 performs initial screening of color patches, including using image segmentation functions to extract black and white connected regions that meet the grayscale threshold in parallel in the whole image, and outputting the bounding box, area, perimeter and pixel density features of each connected region, which constitute the black candidate color patch set and the white candidate color patch set respectively.

[0036] In one optional implementation, the image segmentation function is the OpenMV built-in `img.find_blobs()` function. This function is based on the scan-line seed filling algorithm, which scans the image pixels line by line, marking adjacent connected regions of the same color, and can simultaneously output the bounding boxes (top-left corner coordinates, width, and height), area (total number of pixels), perimeter (number of boundary pixels), and pixel density of each connected region. The initial screening of color patches uses the above function to obtain multi-dimensional low-level data of all candidate color patches at once, providing raw data for geometric feature filtering, pixel density filtering, and relative proportion verification in subsequent steps SS23. This eliminates the need for repeated image traversal, improving processing efficiency on the embedded platform.

[0037] SS23 performs multi-level feature filtering on the black candidate color block set and the white candidate color block set respectively. The multi-level feature filtering includes geometric feature filtering, pixel density filtering and relative proportion verification, and outputs black outer frame candidate regions and white inner frame candidate regions that meet the filtering conditions.

[0038] Specifically, step SS23 further includes the following sub-steps.

[0039] SS231 performs geometric feature filtering, including setting a width-to-height ratio threshold for color blocks, removing candidate color blocks whose width-to-height ratio exceeds the width-to-height ratio threshold, setting a lower limit for the area threshold of color blocks, and removing candidate color blocks whose area is smaller than the lower limit for the area threshold.

[0040] Considering the target is an A4 sheet of paper (standard size 210mm × 297mm, aspect ratio approximately 1:1.5), and that the rectangular projection will undergo non-uniform scaling due to perspective distortion, this embodiment sets the aspect ratio threshold range to 0.7~1.6 to accommodate distortion caused by large viewing angles at close range. Simultaneously, to avoid false detection of distant minor noise or nearby interference, the lower limit of the area threshold is set to 1 / 30 of the total image area, ensuring that the candidate color patches match the actual target in terms of imaging scale.

[0041] SS232 performs pixel density filtering, including calculating the proportion of black pixels in a black candidate color block to the total number of pixels in that color block. When the proportion is less than a first preset threshold, the black candidate color block is retained, and solid black interference objects with a proportion greater than or equal to the first preset threshold are removed.

[0042] The black outer frame is a hollow tape structure, not a solid black block. If the candidate area is a solid black interference object, such as a black rectangular block, its internal pixel density will be significantly higher than that of the hollow frame. Therefore, this embodiment sets the first preset threshold to 0.3, that is, the proportion of black pixels in the black candidate area must be less than 0.3, in order to effectively distinguish between two types of targets that are geometrically similar but have different internal structures: "frame" and "solid block".

[0043] SS233 performs relative proportion verification, which includes matching black candidate color blocks that meet geometric feature filtering and pixel density filtering with white candidate color blocks, calculating the pixel area ratio between the black outer frame and the white inner frame after matching, and retaining the matching combination of the current black candidate color block and white candidate color block when the pixel area ratio is less than a second preset threshold; otherwise, it is discarded.

[0044] Since the target is outlined on a standard A4 sheet of paper using black tape of a fixed width, the ratio of the black border area to the white inner border area is theoretically stable within a certain range. In this embodiment, a second preset threshold is set to 1.4, meaning the ratio of the black and white pixel areas is less than 1.4, to eliminate incorrect combinations where the outer and inner border sizes do not match.

[0045] SS24 verifies the spatial relationship between the candidate regions of the black outer frame and the candidate regions of the white inner frame, and matches the black outer frame and the white inner frame that satisfy the nesting relationship as a valid target combination. The nesting relationship is that the white inner frame is completely located inside the black outer frame.

[0046] The target's "black outer, white inner" structure has a strict nesting relationship: the white inner frame must be completely contained within the black outer frame. Let the bounding box parameter of the black outer frame be... The bounding box parameters of the white inner frame are The condition for determining that the white inner frame is inside the black outer frame is:

[0047] The above constraints, through a 5% margin contraction, ensure that the white inner frame is strictly inside the black outer frame, while also allowing for margin for border width and slight distortion. Only when the white inner frame simultaneously meets all four of the above conditions is the black outer frame and the white inner frame considered to constitute a valid target combination.

[0048] SS25 performs iterative verification, including the outer layer traversing all black candidate color blocks that have passed geometric feature filtering and pixel density filtering, and the inner layer searching for white candidate color blocks that satisfy relative proportion verification and spatial relationship constraint verification within the region of interest formed by shrinking the bounding box of each black candidate color block inward by a preset ratio. The output is the bounding box parameters and four vertex coordinates of the valid target combination that has passed the iterative verification.

[0049] This step's coarse-to-fine iterative strategy reduces computational complexity from global traversal to local search, achieving a high processing frame rate on embedded platforms.

[0050] SS3, within the embedded visual perception module, extracts the pixel centroid coordinates of the black outer frame and the average vertex center coordinates of the white inner frame in the effective target combination. It calculates an adaptive weight coefficient based on the variance of the pixel centroids of the black outer frame and the variance of the average vertex center of the white inner frame. Based on the adaptive weight coefficient, it performs a weighted fusion of the pixel centroid coordinates and the average vertex center coordinates to obtain the target center coordinates.

[0051] It is important to note that accurate estimation of the target center coordinates is a crucial prerequisite for subsequent visual servo control. In practical applications of embedded mobile platforms, there is often a close-range, wide-angle relative pose between the camera and the target. In this case, perspective distortion causes a regular rectangle to degenerate into an irregular quadrilateral in the imaging plane, resulting in systematic positioning errors using traditional centroid methods or simple vertex averaging methods. The essence of perspective distortion lies in the transformation of the center projection from three-dimensional space to the two-dimensional imaging plane. When there is a large angle between the camera's optical axis and the target plane normal, and the shooting distance is close, the "nearer objects appear larger, farther objects appear smaller" effect is particularly significant: the side of the rectangular frame closer to the lens occupies a larger pixel area in the image, while the side farther away is compressed, causing the regular rectangle to project as a trapezoid or an arbitrary quadrilateral.

[0052] Under these conditions, directly using the pixel centroid of the black outer frame as the target center estimate will introduce a significant systematic bias. This is because centroid calculation is weighted by pixel area; the side with the larger area (closer to the lens) exerts a stronger "traction" on the centroid position, causing it to shift towards the closer to the lens, deviating from the target's true geometric center. The magnitude of this shift is positively correlated with the shooting distance and the degree of angle of view; in extreme angles, it can even reach several centimeters, severely impacting subsequent aiming accuracy.

[0053] On the other hand, the white inner frame is enclosed by four vertices, and its geometric center can be obtained by the arithmetic mean of the coordinates of the four vertices. Since the area of ​​the white inner frame is much smaller than that of the black outer frame, it is less affected by the overall scaling due to perspective distortion, and the vertex positions directly reflect the geometric boundaries of the inner frame, with its average center being closer to the true projection position of the target's physical center in the image. However, vertex extraction is quite sensitive to image noise and edge detection accuracy. In cases of uneven lighting or blurred borders, even a small shift in a single vertex can directly lead to a jump in the center estimate, resulting in insufficient stability.

[0054] In summary, the black outer frame centroid has good positional stability (insensitive to local noise) but poor geometric accuracy; the white inner frame vertices have high average geometric accuracy but insufficient stability. The two features exhibit significant complementary characteristics in their distortion resistance, and this embodiment designs a weighted fusion strategy based on this.

[0055] Step SS3 specifically includes the following sub-steps.

[0056] SS31 reads the black outer bounding box parameters and the white inner bounding box parameters from the valid target combination.

[0057] After multi-level feature filtering, spatial relationship constraint verification, and iterative verification in step SS2, the effective target combination includes matched black outer boxes and white inner boxes, each represented by its bounding box parameters. This step reads the bounding box parameters of the black outer boxes. and the bounding box parameters of the white inner frame. This serves as the input data for calculating the centroid coordinates of the black outer frame pixels in subsequent step SS32 and the average center coordinates of the vertices of the white inner frame in step SS33. The bounding box parameters are defined with the top-left corner of the image as the origin, the horizontal direction to the right as the positive x-axis, and the vertical direction downwards as the positive y-axis. and These represent the x and y coordinates of the top-left vertex of the bounding box, respectively. and These represent the width of the bounding box in pixels. and These represent the height in pixels of the bounding box.

[0058] SS32 extracts the pixel centroid coordinates of the black outer frame, including the coordinates of all black pixels within the black outer frame bounding box, and calculates the pixel centroid coordinates of the black outer frame using a weighted average of pixel coordinates. and the variance of the pixel centroid coordinates in the horizontal and vertical directions. .

[0059] Centroid calculation uses pixel area as the weight. Since the pixels of the black outer frame are evenly distributed, its centroid position is not sensitive to local vertex perturbations, and the variance is... It has consistently remained at a low level.

[0060] SS33 extracts the average center coordinates of the vertices of the white inner frame, including extracting the coordinates of the four vertices of the white inner frame bounding box and calculating their arithmetic mean as the average center coordinates of the vertices of the white inner frame. And the variance of the average center coordinates of the vertices in the horizontal and vertical directions. .

[0061] The white inner frame is formed by four vertices, and its geometric center can be obtained by the arithmetic mean of the coordinates of the four vertices. It is less affected by the overall scaling due to perspective distortion and is closer to the true projection position of the target's physical center in the image.

[0062] SS34, based on the variance of the centroid coordinates of the black outer frame pixels. Variance of the average center coordinates of the vertices of the white inner frame The adaptive weighting coefficients are calculated according to the following formula. :

[0063] As shown in the above equation, when the skewed viewpoint leads to increased asymmetric distortion, the variance of the white vertex coordinates increases. As the angle increases, α automatically decreases, and the system reduces its trust in white features, instead relying on the more stable black centroid; when the viewing angle is centered and distortion is relatively mild, As α decreases, the system automatically increases its weight on white features, leveraging their geometric accuracy to improve center estimation precision. Because... The change in α is relatively stable, mainly reflecting the dynamic adjustment of the variance of the white features, achieving an adaptive balance between "geometric accuracy" and "positional stability" in the center estimation.

[0064] SS35, based on adaptive weighting coefficients Centroid coordinates of the black outer frame pixels Average center coordinates of the vertex of the white inner frame Perform weighted fusion and calculate the target center coordinates according to the following formula. :

[0065] Through weighted fusion, adaptive correction of the target center coordinates under perspective distortion conditions was achieved.

[0066] SS4 transmits the target center coordinates to the main control unit, which calculates the pixel deviation between the target center coordinates and the image center coordinates. The pixel deviation is then input into the PID controller to obtain the control quantities of the gimbal's azimuth and pitch axes.

[0067] The OpenMV camera captures images at QVGA resolution. The image coordinate system has its origin at the top left corner, with the positive x-axis pointing horizontally to the right and the positive y-axis pointing vertically downwards. The coordinates of the center pixel of the field of view are... The target center coordinates output from step SS3. The pixel deviation is obtained by subtracting from the center of the field of view:

[0068] In the formula, , These represent the number of pixels that deviate from the center of the field of view in the horizontal and vertical directions, respectively. In the coaxial configuration of "eye in hand," since the camera optical axis and the laser optical axis are parallel and the spacing is fixed, the pixel deviation within a small angle range is approximately linearly related to the angle that the gimbal needs to compensate for.

[0069] The system employs a position-based digital PID control algorithm to perform independent closed-loop control of the gimbal's azimuth (x-axis) and pitch (y-axis) axes. Let k be the sampling sequence number and T be the sampling period. Then, the control quantity at time k is a linear superposition of proportional, integral, and derivative terms:

[0070] In the formula, , , These are the proportional, integral, and differential coefficients of the azimuth axis, respectively. , , These are the proportional, integral, and derivative coefficients for the pitch axis, respectively. Because the mechanical load inertia, transmission clearance, and range of motion differ between the azimuth and pitch axes, the PID parameters for the two axes need to be tuned separately. , These are the pixel deviations at sampling times k and (k-1), respectively.

[0071] SS5 drives the two-dimensional gimbal to rotate according to the control quantity, causing the laser spot to converge toward the center of the target. When the pixel deviation is less than the preset threshold, the laser pointer is triggered to complete aiming.

[0072] The main control unit converts the control output from the PID controller into the pulse direction and number of the stepper motor. After being amplified by the driver, the pulses drive the pan-tilt unit to move in the direction that eliminates the deviation. When the deviation is less than a preset threshold, the main control unit triggers the laser designator via a GPIO signal to complete aiming.

[0073] In SS6, while executing the visual servo control threads of steps SS1 to SS5, the main control unit executes the mobile platform tracking control thread in parallel. The tracking control thread collects detection signals from a multi-channel grayscale sensor array located at the bottom of the vehicle body, adjusts the differential speed of the left and right drive wheels in real time, and controls the mobile platform to drive autonomously along a preset trajectory.

[0074] Step SS6 specifically includes the following sub-steps.

[0075] The SS61 collects detection signals from multiple grayscale sensors located at the bottom of the vehicle body at a preset sampling frequency.

[0076] An array of eight grayscale sensors (numbered S1 to S8) is installed at the front of the vehicle near the ground to detect the position of the black guide line. Among them, S4 and S5 are located on the centerline of the vehicle and are used to detect the tracking centering state; S3 and S6 are used for yaw detection and attitude correction; S1 is used for right-angle turn triggering; S2 and S7 are used for redundant detection; an additional grayscale sensor is installed at the left wheel of the vehicle to assist in right-angle turn determination.

[0077] SS62 determines the lateral offset direction and amount of the vehicle body relative to the preset black guide line based on the detection signals from multiple grayscale sensors.

[0078] Specifically, the absorption of light by the black guide line causes the grayscale sensor above the guide line to output a low-level signal, while the grayscale sensor outside the guide line outputs a high-level signal. The lateral offset direction is determined by comparing the level states of sensors S4 and S5 located near the vehicle's centerline with those of sensors S3 and S6 located on either side of the centerline. When S3 detects a low-level signal but S4 does not, it is determined to be a right yaw; when S6 detects a low-level signal but S5 does not, it is determined to be a left yaw. The lateral offset amount is determined by the number of sensors falling above the black guide line. The sensor arrays S1 to S8 are arranged at fixed intervals. The more sensors in the array that detect low-level signals, the greater the lateral offset of the vehicle relative to the black guide line. When only sensors S4 and S5 detect low-level signals, it indicates that the vehicle is in a centered position. The lateral offset is zero. When sensors S3, S4, and S5 simultaneously detect a low-level signal, it indicates that the vehicle body has deviated from the guide line to the right by one sensor distance. When sensors S2, S3, S4, and S5 simultaneously detect a low-level signal, it indicates that the vehicle body has deviated from the guide line to the right by two sensor distances. When sensors S1, S2, S3, S4, and S5 simultaneously detect a low-level signal, it indicates that the vehicle body has deviated from the guide line to the right by three sensor distances. When sensors S4, S5, and S6 simultaneously detect a low-level signal, it indicates that the vehicle body has deviated from the guide line to the left by one sensor distance. When sensors S4, S5, S6, and S7 simultaneously detect a low-level signal, it indicates that the vehicle body has deviated from the guide line to the left by two sensor distances. When sensors S4, S5, S6, S7, and S8 simultaneously detect a low-level signal, it indicates that the vehicle body has deviated from the guide line to the left by three sensor distances.

[0079] SS63 calculates the target speed difference between the left and right drive wheels based on the lateral offset direction and offset amount.

[0080] Based on the lateral offset direction and offset amount, the target speed difference between the left and right drive wheels is calculated using a PID control algorithm or a differential control algorithm.

[0081] SS64 generates PWM control signals for the left and right drive wheels based on the target speed difference, driving the left and right wheels to rotate at the corresponding speeds, so that the vehicle body travels along the preset black guide line.

[0082] During steps SS61 to SS64, the visual servo control thread and the tracking control thread operate independently with different control cycles. Changes in camera pose caused by the tracking control thread's adjustment of the mobile platform's driving posture are automatically compensated for by the visual servo control thread through the closed-loop feedback in steps SS1 to SS5. These two control threads are independent in timing but physically coupled through the same mobile platform, thus achieving a collaborative operation mode of "moving, recognizing, and aiming simultaneously."

[0083] As a further extension of the above embodiments, in an optional implementation, the system also performs a laser circular trajectory planning step to further realize the function of laser trajectory drawing on the target surface based on the basic aiming function. This step shares the same hardware platform and control framework as steps SS1 to SS6, and specifically includes the following sub-steps.

[0084] S71, Obtain tracking progress: Obtain the tracking progress percentage ρ of the mobile platform along the preset trajectory, the tracking progress percentage It represents the positional progress of the mobile platform on the preset closed trajectory.

[0085] S72, Calculate the target central angle: Based on the tracking progress percentage ρ, calculate the corresponding target central angle θ according to the following formula:

[0086] S73, Establish the ellipse parametric equation: using the target center coordinates obtained in step SS3. Using the ellipse as the center and the length and width pixel dimensions of the rectangular target frame of the effective target combination in the image as a reference, establish the parametric equation of the ellipse and calculate the coordinates of the laser target trajectory points. :

[0087] In the formula, This represents the width (in pixels) of the rectangular target bounding box in the image. Let [value] be the height (in pixels) of the rectangular target bounding box in the image. This is the scaling factor along the major axis of the ellipse. This is the scaling factor along the minor axis of the ellipse.

[0088] It should be noted that due to perspective projection between the camera and the target surface, the circular trajectory of the target surface appears as an ellipse in the image. This embodiment uses the elliptic parametric equation to describe the laser target trajectory, and utilizes the length and width pixel dimensions of the rectangular target frame to reflect the imaging ratio of the target surface in the image in real time, so as to approximately compensate for the influence of perspective distortion on the trajectory shape. , The value is determined by calibrating the perspective distortion parameters of the camera or by experimental trial and error, so that the diameter of the trajectory projected by the laser on the target surface is close to the preset value.

[0089] S74, Calculate displacement deviation: Obtain the coordinates of the current laser spot in the image. Calculate the displacement deviation between the current laser spot coordinates and the target trajectory point coordinates:

[0090] S75, Perform trajectory tracking: Track displacement deviation The target pulse count is converted into a stepper motor, driving the 2D gimbal to track the elliptical trajectory point by point according to the sampling period, so that the laser spot draws a circular trajectory on the target surface. This trajectory tracking process shares the same PID control framework as step SS5, only replacing the desired position with a target trajectory point that dynamically changes over time instead of a fixed target center.

[0091] The circle-drawing task requires the vehicle to simultaneously draw a circular trajectory of a preset radius on the target surface while following a line for one revolution. As the vehicle travels along the preset closed trajectory, the tracking progress percentage ρ increases from 0 to 1, and the central angle θ increases from 0° to 360°, with the laser spot drawing a complete circle on the target surface. Experimental results show that the synchronization error in drawing the circle is less than 1 / 4 revolution.

[0092] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An autonomous aiming method based on machine vision, characterized in that, Includes the following steps: SS1 is an image containing a target acquired by an embedded visual perception module. The target has a double-layer structure with a black outer edge and a white inner edge. SS2, within the embedded visual perception module, extracts the black and white connected regions in the image respectively. After multi-level feature filtering and spatial relationship constraint verification, it obtains a matching black outer frame and white inner frame as an effective target combination. SS3, within the embedded visual perception module, extracts the pixel centroid coordinates of the black outer frame and the average vertex center coordinates of the white inner frame in the effective target combination. It calculates an adaptive weight coefficient based on the variance of the pixel centroids of the black outer frame and the variance of the average vertex center of the white inner frame. Based on the adaptive weight coefficient, it performs a weighted fusion of the pixel centroid coordinates and the average vertex center coordinates to obtain the target center coordinates. SS4 transmits the target center coordinates to the main control unit, which calculates the pixel deviation between the target center coordinates and the image center coordinates, inputs the pixel deviation into the PID controller, and calculates the control quantities of the gimbal azimuth axis and pitch axis. SS5 drives the two-dimensional gimbal to rotate according to the control quantity, so that the laser spot converges towards the center of the target. When the pixel deviation is less than the preset threshold, the laser pointer is triggered to complete the aiming. In SS6, while executing the visual servo control threads of steps SS1 to SS5, the main control unit executes the mobile platform tracking control thread in parallel. The tracking control thread collects detection signals from a multi-channel grayscale sensor array located at the bottom of the vehicle body, adjusts the differential speed of the left and right drive wheels in real time, and controls the mobile platform to drive autonomously along a preset trajectory.

2. The machine vision-based autonomous aiming method according to claim 1, characterized in that, Step SS2 specifically includes: SS21, Image Preprocessing, includes setting the acquired image to QVGA resolution, adjusting exposure parameters to improve image contrast and reduce brightness, and making the boundary between the black border and the white paper surface clearer. SS22 performs initial screening of color blocks, including using image segmentation functions to extract black and white connected regions that meet the grayscale threshold in parallel in the whole image, and outputting the bounding box, area, perimeter and pixel density features of each connected region, which constitute the black candidate color block set and the white candidate color block set respectively. SS23 performs multi-level feature filtering on the black candidate color block set and the white candidate color block set respectively. The multi-level feature filtering includes geometric feature filtering, pixel density filtering and relative proportion verification, and outputs black outer frame candidate regions and white inner frame candidate regions that meet the filtering conditions. SS24 verifies the spatial relationship between the candidate regions of the black outer frame and the candidate regions of the white inner frame, and matches the black outer frame and the white inner frame that satisfy the nesting relationship as a valid target combination. The nesting relationship is that the white inner frame is completely inside the black outer frame. SS25 performs iterative verification, including the outer layer traversing all black candidate color blocks that have passed geometric feature filtering and pixel density filtering, and the inner layer searching for white candidate color blocks that satisfy relative proportion verification and spatial relationship constraint verification within the region of interest formed by shrinking the bounding box of each black candidate color block inward by a preset ratio. The output is the bounding box parameters and four vertex coordinates of the valid target combination that has passed the iterative verification.

3. The machine vision-based autonomous aiming method according to claim 2, characterized in that, Step SS23 specifically includes: SS231 performs geometric feature filtering, including setting a width-to-height ratio threshold for color blocks, removing candidate color blocks whose width-to-height ratio exceeds the width-to-height ratio threshold, setting a lower limit for the area threshold of color blocks, and removing candidate color blocks whose area is smaller than the lower limit for the area threshold. SS232 performs pixel density filtering, including calculating the proportion of black pixels in a black candidate color block to the total number of pixels in the color block. When the proportion is less than a first preset threshold, the black candidate color block is retained, and solid black interference objects with a proportion greater than or equal to the first preset threshold are removed. SS233 performs relative proportion verification, which includes matching black candidate color blocks that meet geometric feature filtering and pixel density filtering with white candidate color blocks, calculating the pixel area ratio between the black outer frame and the white inner frame after matching, and retaining the matching combination of the current black candidate color block and white candidate color block when the pixel area ratio is less than a second preset threshold; otherwise, it is discarded.

4. The machine vision-based autonomous aiming method according to claim 3, characterized in that, In step SS24, the bounding box parameters of the black outer frame are set to... The bounding box parameters of the white inner frame are The condition for determining that the white inner frame is inside the black outer frame is: Only when the white inner frame meets all four of the above conditions is the black outer frame and the white inner frame confirmed to form an effective target combination.

5. The machine vision-based autonomous aiming method according to claim 1, characterized in that, Step SS3 specifically includes: SS31, reads the parameters of the black outer bounding box and the white inner bounding box in the effective target combination; SS32 extracts the pixel centroid coordinates of the black outer frame, including the coordinates of all black pixels within the black outer frame bounding box, and calculates the pixel centroid coordinates of the black outer frame using a weighted average of pixel coordinates. and the variance of pixel centroid coordinates in the horizontal and vertical directions. ; SS33 extracts the average center coordinates of the vertices of the white inner frame, including extracting the coordinates of the four vertices of the white inner frame bounding box and calculating their arithmetic mean as the average center coordinates of the vertices of the white inner frame. And the variance of the average center coordinates of the vertices in the horizontal and vertical directions. ; SS34, based on the variance of the centroid coordinates of the black outer frame pixels. Variance of the average center coordinates of the vertices of the white inner frame The adaptive weighting coefficients are calculated according to the following formula. : SS35, based on adaptive weighting coefficients Centroid coordinates of the black outer frame pixels Average center coordinates of the vertex of the white inner frame Perform weighted fusion and calculate the target center coordinates according to the following formula. : 。 6. The machine vision-based autonomous aiming method according to claim 1, characterized in that, Step SS6, according to the diagram, includes: SS61 collects detection signals from multiple grayscale sensors located at the bottom of the vehicle body at a preset sampling frequency; SS62 determines the lateral offset direction and amount of the vehicle body relative to the preset black guide line based on the detection signals from multiple grayscale sensors. SS63 calculates the target speed difference between the left and right drive wheels based on the lateral offset direction and offset amount; SS64 generates PWM control signals for the left and right drive wheels based on the target speed difference, driving the left and right wheels to rotate at the corresponding speeds, so that the vehicle body travels along the preset black guide line; During the execution of steps S61 to S64, the visual servo control thread and the tracking control thread run independently with different control cycles. The camera pose changes caused by the tracking control thread's adjustment of the mobile platform's driving posture are automatically compensated by the visual servo control thread through the closed-loop feedback of steps S1 to S5.

7. An autonomous aiming system based on machine vision, characterized in that, include: An embedded visual perception module is used to acquire images containing targets. The targets have a double-layer structure with a black outer border and a white inner border. The embedded visual perception module runs a target recognition algorithm, which extracts black and white connected regions in the image. After multi-level feature filtering and spatial relationship constraint verification, a matching black outer frame and a white inner frame are obtained as a valid target combination. The pixel centroid coordinates of the black outer frame and the average vertex center coordinates of the white inner frame are extracted from the valid target combination. The main control unit is communicatively connected to the embedded visual perception module. The main control unit includes a target center calculation module, a visual servo control module, and a tracking control module, wherein: The target center calculation module is used to receive the centroid coordinates of the black outer frame pixels and the average center coordinates of the white inner frame vertices output by the embedded visual perception module. It calculates the adaptive weight coefficient based on the variance of the centroid of the black outer frame pixels and the variance of the average center of the white inner frame vertices. Based on the adaptive weight coefficient, it performs weighted fusion of the pixel centroid coordinates and the average center coordinates of the vertices to obtain the target center coordinates. The visual servo control module is used to calculate the pixel deviation between the target center coordinates and the image center coordinates. The pixel deviation is input into the PID controller to calculate the azimuth axis control quantity and pitch axis control quantity of the two-dimensional gimbal. The tracking control module is used to collect detection signals from a multi-channel grayscale sensor array located at the bottom of the mobile platform vehicle body, adjust the differential speed of the left and right drive wheels in real time, and generate tracking control quantities. Among them, the visual servo control module and the tracking control module run in parallel threads in the main control unit, enabling the system to autonomously move along a preset trajectory while performing aiming. The two-dimensional gimbal is electrically connected to the main control unit and includes an azimuth axis stepper motor and a pitch axis stepper motor. It is used to receive azimuth axis control signals and pitch axis control signals and adjust the gimbal attitude accordingly. A laser pointer is fixedly mounted on the top of a two-dimensional pan-tilt unit. Its optical axis is parallel to the optical axis of the embedded vision perception module and the distance between them is fixed. It is used to emit lasers. The mobile platform is a differential wheel chassis used to carry the main control unit, embedded vision perception module, two-dimensional gimbal and laser pointer. The mobile platform includes left and right drive wheels and drive modules that drive the left and right drive wheels respectively. The drive modules adjust the differential speed of the left and right drive wheels in response to the tracking control quantity, so that the mobile platform can move autonomously along a preset trajectory.

8. The machine vision-based autonomous aiming system according to claim 7, characterized in that, The embedded vision perception module is an OpenMV embedded machine vision module.

9. The machine vision-based autonomous aiming system according to claim 7, characterized in that, The main control unit is an MSPM0G3507 microcontroller.

10. The machine vision-based autonomous aiming system according to claim 7, characterized in that, The multiple grayscale sensors are numbered S1 to S8 from left to right. S4 and S5 are located on the centerline of the vehicle body and are used to detect the tracking centering state; S3 and S6 are located on both sides of the centerline and are used for yaw detection and attitude correction; S1 is used for right-angle turn triggering; S2 and S7 are used for redundant detection.

Images