Small target image segmentation detection method and system based on micro image sensing module assembly

Abstract

The invention relates to the technical field of image processing, in particular to a small target image segmentation detection method and system based on miniature image sensing module assembly, and the method comprises the steps: obtaining a to-be-detected image comprising a small target; processing the to-be-detected image by using a small target image segmentation model to generate a pixel-level segmentation mask; performing contour optimization processing on the pixel-level segmentation mask to obtain contour data; and on the basis of the contour data, through image gradient iterative calculation, obtaining a sub-pixel precision coordinate of the target feature point. The objective of the invention is to solve the problems of low small target detection precision and high missing detection and false detection rate in the assembly process of the miniature image sensing module.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for segmenting and detecting small target images based on a miniature image sensing module assembly. Background Technology

[0002] The miniature image sensor module consists of a micro-camera and a data cable, which need to be assembled through precision welding. Automated assembly relies on a laser welding micro-assembly platform. Specifically, a multi-channel microscopic vision system is used to online identify and detect the relative positions of the micro-camera's solder pads and the data cable. A position servo-controlled robotic arm then grips the data cable for precise alignment, and finally, laser welding completes the assembly. The vision system's accuracy in identifying and detecting parts at the hundred-micron level determines the accuracy of the alignment and the stability of the assembly process.

[0003] Existing visual inspection solutions, when faced with the specific assembly scenario of miniature image sensor modules, suffer from several drawbacks. These include the susceptibility of weakly rigid data lines to random deformation and pose changes, and limited visual depth of field due to confined assembly space. Consequently, the acquired images exhibit interference such as a very small proportion of target pixels, severe occlusion of effective contour features, and blurring of key targets due to defocus. These factors lead to performance degradation in feature extraction and spatial localization for recognition methods based on traditional image processing or general deep learning models. This results in insufficient detection accuracy, high false negative and false positive rates, and an inability to meet the technical requirements for high-precision assembly of miniature image sensor modules. Summary of the Invention

[0004] To address the issues of low accuracy, high false negative and false positive rates in small target detection during the assembly of miniature image sensing modules, this invention provides a method and system for small target image segmentation and detection based on the assembly of miniature image sensing modules. The specific technical solution adopted is as follows: The first aspect of the present invention provides a method for segmenting and detecting small target images based on a miniature image sensing module assembly, the method comprising: Acquire the image to be detected, including small targets; The image to be detected is processed using a small target image segmentation model to generate a pixel-level segmentation mask; The pixel-level segmentation mask is subjected to contour optimization processing to obtain contour data; Based on contour data, sub-pixel precision coordinates of target feature points are obtained through iterative calculation of image gradients.

[0005] Furthermore, the image to be detected, including small targets, is acquired, including: A non-orthogonal multi-channel microscopic vision system is configured, including a multi-degree-of-freedom manipulator with a reflector and a refractor. Control the interaction between the reflector and the refractor to change the direction of the optical path; The non-orthogonal multi-channel microscopic vision system after optical path conversion acquires real-time images of small targets in the assembly scene to obtain the image to be detected.

[0006] Furthermore, the image to be detected is processed using a small target image segmentation model to generate a pixel-level segmentation mask, including: Construct a small target image segmentation model that includes a feature extraction network and a segmentation head network; A feature extraction network is used to extract hierarchical features from the image to be detected and perform stepless downsampling to output multi-scale feature maps. Multi-scale feature maps are input into the segmentation head network, and spatial location information is enhanced through a coordinate attention mechanism. Based on the enhanced multi-scale feature map, a pixel-level segmentation mask is generated; wherein, the small target image segmentation model is trained by a loss function that incorporates bounding box similarity measurement.

[0007] Furthermore, a feature extraction network is used to perform hierarchical feature extraction on the image to be detected, and a stepless downsampling operation is performed to output multi-scale feature maps, including: Obtain the input feature map of the previous layer of the feature extraction network; The input feature map is divided into multiple sub-feature maps according to a preset ratio in the spatial dimension; Multiple sub-feature maps are concatenated along the channel dimension to form an intermediate feature map; Non-stepping convolutional layers are used to perform convolution operations on intermediate feature maps to output multi-scale feature maps.

[0008] Furthermore, the multi-scale feature map is input into the segmentation head network, and spatial location information enhancement is performed on the multi-scale feature map through a coordinate attention mechanism, including: Obtain multi-scale feature maps output by the feature extraction network; One-dimensional global pooling is performed along the horizontal and vertical directions of the multi-scale feature map to obtain the first feature vector and the second feature vector. The first feature vector and the second feature vector are concatenated and convolved to generate an intermediate feature map that encodes the spatial relationship. The intermediate feature map is decomposed along the spatial dimension into a horizontal attention weight map and a vertical attention weight map; The horizontal and vertical attention weight maps are multiplied and weighted with the multi-scale feature map respectively to obtain the output feature map.

[0009] Furthermore, the construction of the loss function incorporating bounding box similarity metrics includes: The predicted bounding boxes and the ground bounding boxes are modeled as two-dimensional Gaussian distributions respectively; Calculate the normalized Wasserstein distance between two two-dimensional Gaussian distributions as a loss component to measure distribution similarity; Calculate the intersection-union ratio (IU) loss component between the predicted bounding box and the ground truth bounding box; The intersection-union ratio loss component and the distribution similarity metric loss component are weighted and fused to form a loss function that incorporates the bounding box similarity metric.

[0010] Further, the intersection-union ratio (IU) loss component between the predicted bounding box and the ground truth bounding box is calculated, including: Calculate the intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box; Calculate the Euclidean distance between the center points of the predicted bounding box and the true bounding box; Calculate the width difference and height difference between the predicted bounding box and the true bounding box respectively; The cross-union ratio (CUNR) loss component is calculated based on the cross-union ratio, Euclidean distance, width difference, and height difference.

[0011] Further, the pixel-level segmentation mask is subjected to contour optimization processing to obtain contour data, including: The pixel-level segmentation mask is smoothed using a Gaussian filter; Perform morphological opening operations on the smoothed pixel-level segmentation mask; Perform morphological closing operations on the pixel-level segmentation mask after morphological opening to obtain contour data.

[0012] Furthermore, based on the contour data, sub-pixel precision coordinates of the target feature points are obtained through image gradient iterative calculation, including: The initial pixel coordinates of the target feature points are extracted from the contour data, and the image gradient vector is calculated in the neighborhood of the initial pixel coordinates. The sub-pixel coordinate equation is established based on the image gradient and the perpendicularity of the image gradient direction to the contour edge line. The least squares method is used to iteratively calculate the equation for solving the sub-pixel coordinates, and the sub-pixel precision coordinates of the target feature points are output.

[0013] The second aspect of the present invention provides a small target image segmentation and detection system based on a miniature image sensing module assembly, used to execute the small target image segmentation and detection method based on a miniature image sensing module assembly described in the first aspect of the present invention, the system comprising: The image acquisition module is configured to acquire images of the target object, including small targets. The image segmentation module is configured to process the image to be detected using a small target image segmentation model to generate a pixel-level segmentation mask; The contour optimization module is configured to perform contour optimization processing on the pixel-level segmentation mask to obtain contour data. The coordinate positioning module is configured to obtain the sub-pixel precision coordinates of target feature points through image gradient iterative calculation based on contour data.

[0014] The present invention has the following beneficial effects: This invention provides a small target image segmentation and detection method based on the assembly of miniature image sensing modules, solving the problems of low detection accuracy, high false negative and false positive rates caused by complex environments in the assembly of miniature image sensing modules. First, the image to be detected is processed using a small target image segmentation model to generate a pixel-level segmentation mask, which can accurately separate small targets from the background, improving the initial recognition and contour extraction capabilities of small targets in complex backgrounds, thereby reducing false negatives and false positives. Then, the obtained pixel-level mask undergoes contour optimization processing, providing high-quality contour data for subsequent precise positioning. Finally, based on the contour data, iterative calculations using image gradient information are performed to obtain sub-pixel precision coordinates of target feature points, breaking through the accuracy limit of traditional pixel-level positioning and achieving accurate recognition and positioning of small targets in the assembly scenario of miniature image sensor modules, meeting the technical requirements of high-precision assembly. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a small target image segmentation and detection method based on a miniature image sensing module assembly according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a small target image segmentation and detection system based on a miniature image sensing module assembly, provided in one embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a small target image segmentation and detection method and system based on a miniature image sensing module assembly proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details the specific scheme of a small target image segmentation and detection method and system based on a miniature image sensing module assembly provided by the present invention.

[0020] Please see Figure 1 This document illustrates a flowchart of a small target image segmentation and detection method based on a miniature image sensing module assembly, according to an embodiment of the present invention. The method includes: Step S100: Obtain the image to be detected, including small targets; Step S100 specifically includes: Step S110: Configure a non-orthogonal multi-path microscopic vision system including a reflector and a multi-degree-of-freedom manipulator with a refractive mirror. Specifically, on the assembly platform of the miniature image sensing module, to avoid interference from existing devices such as laser welding machines and manipulators on the traditional orthogonal optical path layout, this embodiment adopts a non-orthogonal multi-path microscopic vision system, using a reflector to change the direction of the basic optical path. Simultaneously, a refractive mirror is integrated and installed at the end of the multi-degree-of-freedom manipulator used for gripping and manipulating data cables. The manipulator not only has a gripping function, but the refractive mirror at its end can also further precisely guide the optical path, thereby achieving a non-orthogonal, flexible optical path layout, ensuring that the vision sensor can image the narrow assembly area from an effective viewing angle. Step S120: Control the reflector and the refractor to cooperate in converting the optical path direction; during system operation, based on the initial spatial pose of the micro camera and data cable in the micro image sensor module, the angle of the fixed reflector and the movable refractor at the end of the robot is controlled in a coordinated manner, so that the light from the illumination source is finally received by the micro vision sensor after multiple reflections and refractions, thereby realizing the flexible conversion of the optical path direction. The purpose is to enable the optical axis of the vision system to effectively bypass the interference device on the platform and accurately focus on the micro camera pad and data cable area to be assembled. Step S130: Using a non-orthogonal multi-path microscopic vision system after optical path conversion, real-time images of small targets within the assembly scene are acquired to obtain the image to be detected. Once the optical path is correctly configured and converted, a calibrated microscopic vision sensor, such as a CCD or CMOS camera, is activated. The system captures clear real-time images of the current assembly scene, including small targets such as solder balls at the hundred-micrometer level and data lines with wire diameters of tens of micrometers on the micro-camera pads. This image is the image to be detected for subsequent algorithm processing. This embodiment utilizes a non-orthogonal multi-channel microscopic vision system to solve the problem of visual blind spots caused by equipment interference in a confined assembly space, and can stably acquire clear images of the parts to be inspected that contain all key micro-components; this step provides a high-quality input source for subsequent image segmentation and sub-pixel localization algorithms.

[0021] Step S200: Process the image to be detected using a small target image segmentation model to generate a pixel-level segmentation mask; Step S200 specifically includes: Step S210: Construct a small target image segmentation model including a feature extraction network and a segmentation head network; specifically, the small target image segmentation model is based on an improved YOLOv11 network architecture, wherein: Feature extraction network: The hierarchical structure of basic convolutional layers combined with SPD-Conv module is adopted to replace the strided convolutional module of traditional YOLOv11. The basic convolutional layer uses 3×3 convolutional kernel to achieve preliminary feature extraction, and the SPD-Conv module is used for non-strided downsampling to preserve fine-grained features of small targets. Segmentation Head Network: Includes a coordinate attention module and a bounding box regression branch. The CA module is used to enhance the spatial location information of the feature map. The bounding box regression branch integrates a loss function that combines bounding box similarity measurement to optimize the prediction accuracy of small target bounding boxes. Step S220: Use a feature extraction network to perform hierarchical feature extraction on the image to be detected, and perform stepless downsampling operation to output multi-scale feature maps; In this embodiment, the stepless downsampling operation (SPD-Conv) is used to replace the standard downsampling layer in order to retain the fine-grained information of small targets; Step S220 specifically includes: Step S221: Obtain the input feature map of the layer preceding the feature extraction network; Let the size of the input feature map be... , , For height and width, The number of channels in the input feature map.

[0022] Step S222: Divide the input feature map into multiple sub-feature maps according to a preset ratio in the spatial dimension; specifically, the preset ratio factor... Preferably, 2, with a size of The input feature map is divided into 4 regions of size 1. The sub-feature maps are denoted as follows: ; Step S223: Concatenate the multiple sub-feature maps along the channel dimension to form an intermediate feature map; The channels are concatenated along the channel dimension to form a channel count. The intermediate feature map; Step S224: A non-staggered convolutional layer is used to convolve the intermediate feature map, outputting a multi-scale feature map. A non-staggered convolution with a stride of 1 is used to compress the channel dimension while maintaining the feature size. When processing small-resolution targets, the SPD-Conv module uses spatial recombination to preserve spatial feature information in the channel dimension, retaining fine-grained features of low-resolution small targets during downsampling feature extraction. Therefore, this embodiment uses the non-staggered convolutional SPD-Conv as the feature extraction module to replace the traditional staggered convolutional module, reducing the loss of shallow feature information caused by staggered convolution, improving the segmentation accuracy of small target instances in complex environments, and enhancing model performance.

[0023] Step S230: Input the multi-scale feature map into the segmentation head network, and enhance the spatial location information of the multi-scale feature map through the coordinate attention mechanism; This sub-step applies coordinate attention (CA) to the input multi-scale feature map in the segmentation head network, enabling the model to better focus on the precise location of small targets in the image. Specifically, it includes: Step S231: Obtain the multi-scale feature map output by the feature extraction network; Step S232: Perform one-dimensional global pooling along the horizontal and vertical directions of the multi-scale feature map to obtain the first feature vector and the second feature vector; Specifically, one-dimensional global pooling is performed along the horizontal and vertical directions of the multi-scale feature map, respectively. Given the input feature map... Encode each column along the horizontal coordinate direction to obtain a width of eigenvectors At the same time, along the vertical coordinate direction Encode each line to obtain its height. eigenvectors , can be represented as:

[0024] In the formula, Represents the input feature map At height index Width index is Eigenvalues ​​at; Represents the input feature map At height index Width index is Eigenvalues ​​at; Step S233: Concatenate and convolve the first feature vector and the second feature vector to generate an intermediate feature map encoding the spatial relationship; combine the two feature vectors obtained in step S232... and The data is then concatenated and then subjected to a convolutional transformation. and nonlinear activation This generates an intermediate feature map that encodes spatial relationships. The transformation process can be represented as:

[0025] In the formula, Indicates feature concatenation operation; It is a convolution transformation function; Represents a non-linear activation function; Step S234: Decompose the intermediate feature map along the spatial dimension into a horizontal attention weight map and a vertical attention weight map; specifically, the intermediate feature map... Decomposed into two independent tensors along the spatial dimension and Their dimensions are respectively and Then use 1x1 convolutions respectively. and Processing is performed, and the Sigmoid activation function is used. Attention weights in the height direction of the generated feature map Attention weights in the width direction , can be represented as:

[0026] Step S235: Multiply and weight the horizontal and vertical attention weight maps with the multi-scale feature map respectively to obtain the output feature map; the resulting attention weights and Compared with the original input feature map Middle position Location, passage eigenvalues Weighted multiplication is performed to generate the final output feature map containing location information. eigenvalues ​​at corresponding positions , is represented as:

[0027] In the formula, Height-oriented attention weights At height The weight value at the location; Indicates attention weights in the width direction In width The weight value at the location; Step S240: Generate a pixel-level segmentation mask based on the enhanced multi-scale feature map; wherein, the small target image segmentation model is trained by combining a loss function with bounding box similarity measurement; the multi-scale feature map enhanced by the coordinate attention mechanism is input into the subsequent layers of the segmentation head network, and after a series of upsampling and convolution operations, the pixel-level segmentation mask corresponding to the size of the input image is finally output. The small target image segmentation model is trained and optimized by incorporating a loss function that combines bounding box similarity metrics, including: Step S241: Model the predicted bounding box and the ground truth bounding box as two-dimensional Gaussian distributions respectively; specifically, model the predicted bounding box and the ground truth bounding box as two-dimensional Gaussian distributions respectively; As a two-dimensional Gaussian distribution , where the mean The coordinates of the center of the bounding box, and the covariance matrix. The diagonal element and the width of the bounding box and high Related; Step S242: Calculate the normalized Wasserstein distance between the two two-dimensional Gaussian distributions as a loss component to measure distribution similarity; calculate the predicted bounding box distribution. Distribution of true bounding boxes Wasserstein distance between Then, normalization is performed to obtain the NWD loss components. , can be represented as:

[0028] in, For Wasserstein distance, The normalization constant is used to address the sensitivity issue of small target bounding boxes. In this embodiment, the normalized Wasserstein distance (NWD) is used to model the bounding box as a two-dimensional Gaussian probability distribution model with a center point probability of 1 and a boundary point probability of 0. The similarity between the predicted bounding box and the ground truth box is measured by calculating the normalized Wasserstein distance between the two distributions. Even if the overlap of the small target bounding boxes is small, the similarity between the predicted and ground truth boxes can be effectively calculated.

[0029] Step S243: Calculate the Cross-Union Ratio (CUI) loss component between the predicted bounding box and the ground truth bounding box; Calculating the CUI loss component between the predicted bounding box and the ground truth bounding box includes: Calculate the predicted bounding box With real frame intersection ratio Calculate the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. Calculate the width difference between the predicted bounding box and the ground truth bounding box respectively. and height differences ; Comprehensive calculation of EIoU loss components :

[0030] In the formula, To predict bounding boxes With real frame The intersection-union ratio is used to measure the degree of spatial overlap between two things; It is the diagonal length of the smallest bounding rectangle that covers both the predicted bounding box and the ground truth bounding box. and These are the width and height of the bounding rectangle, respectively.

[0031] Step S244: The intersection-union ratio (IU) loss component and the distribution similarity metric loss component are weighted and fused to form a loss function that combines the bounding box similarity metric; specifically, the EIoU loss component is... With NWD loss components By weighting Weighted summation is performed to construct the final hybrid loss function. This is used for model training. The core idea of ​​this embodiment, which combines the EIoU loss function of NWD, is to improve the problem of small target samples being assigned as negative samples due to low IoU caused by small overlapping areas of bounding boxes. Even if the IoU is low, the high similarity of the bounding boxes can provide more high-quality positive samples for training. Then, EIoU is used to regress the bounding box loss, thereby improving the contour localization accuracy of small targets. The final hybrid loss function can be expressed as:

[0032] In the formula, This is the final mixed loss function; The intersection-union loss component is used to optimize the localization accuracy of the predicted bounding box; The loss component is a distribution similarity metric used to address the problem of small targets being misclassified as negative samples due to low bounding box overlap. This is a weighting hyperparameter. For the application scenario described in this paper, there are many small objectives. Take the lower value, 0.2; In summary, step S200 constructs a small target image segmentation model integrating stepless downsampling and coordinate attention mechanisms. It optimizes the model using a loss function combining normalized Wasserstein distance and EIoU, addressing the issues of insufficient feature extraction and inaccurate spatial localization caused by the small size of the parts and the complexity of the background. Stepless downsampling preserves the complete features of the small target through spatial-to-depth transformation, while the coordinate attention mechanism captures accurate long-range positional dependencies using one-dimensional global pooling. The hybrid loss function collectively ensures the sample quality and bounding box regression accuracy of the small target during training, ultimately achieving highly robust and accurate pixel-level segmentation of micrometer-scale parts in a micro-image sensing module.

[0033] Step S300: Perform contour optimization processing on the pixel-level segmentation mask to obtain contour data; Step S300 specifically includes: Step S310: Smooth the pixel-level segmentation mask using a Gaussian filter; this step performs a convolution operation on the binarized pixel-level segmentation mask; specifically, select a suitable Gaussian convolution kernel. Its weight distribution follows a two-dimensional Gaussian function, with the center point having the largest weight and the surrounding weights decreasing with increasing distance; this Gaussian kernel is applied to the entire segmentation mask image. Performing sliding window convolution calculations on top can be represented as:

[0034] in, It is a smoothed mask image. These are pixel coordinates; For Gaussian kernel size parameters, This represents the actual size of the Gaussian kernel; The Gaussian kernel is in position The weight value at the location; Indicates Centered on, offset The pixel coordinates after; This embodiment is essentially a low-pass filter, which can effectively suppress random, high-frequency contour spikes caused by noise in the masked image, while smoothing jagged edges caused by inaccurate model predictions. Through Gaussian weighted averaging, the edge contours are adjusted to be smoother, providing cleaner image data for subsequent morphological processing and sub-pixel edge localization. Step S320: Perform morphological opening operation on the smoothed pixel-level segmentation mask; In some embodiments, the morphological opening operation is a composite operation of erosion followed by dilation, using a preset structuring element. First, an etching operation is performed to remove the structural elements. Smoothed mask image Slide up; only when Only the output pixel corresponding to the center point of the element is retained as foreground when it is completely covered by the target area; otherwise, it is set as background. The purpose is to shrink the target area and eliminate all isolated foreground points or fine connections smaller than the structuring element. Then, the same structuring element is used. Dilatation is performed on the eroded image. During the dilatation process, as long as... If the output pixel intersects with the target region, the output pixel corresponding to its center point is set as the foreground. This operation expands the target region, restoring the size of the main contour that has been shrunk due to erosion; Step S330: Perform morphological closing operation on the pixel-level segmentation mask after morphological opening operation to obtain contour data; In some embodiments, morphological closing is a composite operation of dilation followed by erosion, using the same structuring element as opening. First, the image after the opening operation. The process involves dilation to fill in the tiny cracks and holes inside the target contour and connecting very close breakpoints to make the contour more continuous and fuller; then the dilated image is eroded to restore the approximate original size of the target area while ensuring that the small holes filled during the dilation stage do not reappear. This embodiment optimizes the quality of the initial segmentation mask by combining Gaussian filtering with morphological opening and closing operations. Gaussian filtering first suppresses noise and contour burrs, providing a smooth edge foundation for subsequent processing. Morphological opening operations then eliminate small holes and false connections in the mask, ensuring the independence and purity of the contour. The subsequent morphological closing operation effectively fills the cracks inside the contour and connects the tiny breakpoints, significantly enhancing the continuity and overall smoothness of the contour. Finally, the pixel-level segmentation results are refined into high-quality, high-completeness contour data, providing input for sub-pixel precision coordinate positioning in step S400.

[0035] Step S400: Based on the contour data, obtain the sub-pixel precision coordinates of the target feature points through image gradient iterative calculation; Step S400 specifically includes: Step S410: Extract the initial pixel coordinates of the target feature points from the contour data, and calculate the image gradient direction in the neighborhood of the initial pixel coordinates; Extract initial coordinates: Extract corner points and curvature points from the contour data obtained in step S300 to form the feature contours of the two targets; The coordinates of these points Initially, it is an integer, that is, the initial pixel coordinates, denoted as... ,in For column coordinates, Row coordinates; In the original grayscale image Above, at the initial coordinate point Within a small neighborhood, the image gradient of each pixel is calculated; the image gradient is a vector whose direction points in the direction of the fastest change in image grayscale. On the edge of the target contour, the gradient direction at any point is perpendicular to the tangent direction of the contour edge line at that point; the gradient vector... It is possible to calculate the image in and Partial derivatives in the direction and Therefore, the gradient direction can be determined by this vector, the gradient vector. It can be represented as:

[0036] In the formula, Representing an image In the horizontal direction gradient components on; Representing an image In the vertical direction gradient components on; Step S420: Establish sub-pixel coordinates based on the image gradient and the perpendicularity of the image gradient direction to the contour edge line. Specifically, let the true coordinates of the sub-pixel contour point to be determined be... According to the principle described in step S410, at the edge point At, gradient direction Perpendicular to the contour edge line, therefore, connect the initial pixel points. With sub-pixel vector With gradient direction Parallel lines can be used to establish vector equations:

[0037] In the formula, This is a dot product operation; The coordinate vector of the initial pixel represents the initial pixel-level positioning point on the edge of the target contour; The expression for the sub-pixel coordinate vector represents the sub-pixel level contour point that needs to be precisely located; this expression describes the sub-pixel level contour point based on the gradient matrix. Initial pixel and sub-pixel The mathematical relationship between them is the theoretical constraint for sub-pixel localization; Step S430: The least squares method is used to iteratively calculate the sub-pixel coordinate solution equation, outputting the sub-pixel precision coordinates of the target feature points. In actual images, due to noise and quantization errors, the constraints on a single point are not precise enough. To stably solve for sub-pixel coordinates in noisy real-world conditions, the solution is expressed as:

[0038] In the formula, It is the gradient matrix transpose, It is a matrix The inverse matrix; in real images, noise and quantization errors can lead to insufficient precision in the constraints of the single vector equation. Therefore, the least squares method is used to stably solve for the sub-pixel coordinates. First, the transpose of the gradient vector is calculated and multiplied with the gradient vector to obtain the square matrix. Find the inverse of this matrix, and then combine the inverse matrix with... The initial pixel coordinates p are multiplied sequentially to obtain the sub-pixel coordinates to be determined. This process uses least squares fitting to reduce the influence of noise and error, and can stably output the sub-pixel precision coordinates of the target feature points in real-world scenarios. The above solution process is iterated, and the sub-pixel coordinates obtained in the first solution are... As a new initial point, repeat steps S410 to S430, that is, recalculate the gradient and construct the equation in its new neighborhood, and solve it again using the least squares method. This process is repeated until the sub-pixel coordinates are obtained. If the change is less than a preset minimum threshold, the result is considered to have converged; finally, the final sub-pixel coordinates after convergence are output. , which serves as the sub-pixel precision coordinates of the target feature point; Step S400 utilizes the geometric property that the image gradient direction is perpendicular to the contour edge to establish the sub-pixel coordinate solution equation, and uses the least squares method to iteratively optimize the information of multiple points in the neighborhood, significantly improving the positioning accuracy of the target feature point from the pixel level to the sub-pixel level. This method overcomes the quantization error of a single pixel and the interference of image noise through iterative calculation, and finally stably outputs feature point coordinates with an accuracy better than 0.1 pixels, which meets the accuracy requirements of spatial position detection for micron-level automated assembly and alignment of micro image sensor modules.

[0039] In summary, the small target image segmentation and detection method based on the assembly of miniature image sensing modules provided by this invention solves the problems of low detection accuracy, high false negative and false positive rates of small targets caused by complex environments in the assembly of miniature image sensing modules. First, the image to be detected is processed by a small target image segmentation model to generate a pixel-level segmentation mask, which can accurately separate small targets from the background, improving the initial recognition and contour extraction capabilities of small targets in complex backgrounds, thereby reducing false negatives and false positives. Then, the obtained pixel-level mask undergoes contour optimization processing, providing high-quality contour data for subsequent precise positioning. Finally, based on the contour data, iterative calculations are performed using image gradient information to obtain sub-pixel precision coordinates of target feature points, breaking through the accuracy limit of traditional pixel-level positioning and achieving accurate recognition and positioning of small targets in the assembly scenario of miniature image sensor modules, meeting the technical requirements of high-precision assembly.

[0040] Please see Figure 2 This diagram illustrates a structural schematic of a small target image segmentation and detection system based on a miniature image sensing module assembly, according to an embodiment of the present invention. The system includes: The image acquisition module is configured to acquire images of the target object, including small targets. The image segmentation module is configured to process the image to be detected using a small target image segmentation model to generate a pixel-level segmentation mask; The contour optimization module is configured to perform contour optimization processing on the pixel-level segmentation mask to obtain contour data. The coordinate positioning module is configured to obtain the sub-pixel precision coordinates of target feature points through image gradient iterative calculation based on contour data.

[0041] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0042] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for segmenting and detecting small targets based on a miniature image sensing module assembly, characterized in that, The method includes: Acquire the image to be detected, including small targets; The image to be detected is processed using a small target image segmentation model to generate a pixel-level segmentation mask; The pixel-level segmentation mask is subjected to contour optimization processing to obtain contour data; Based on contour data, sub-pixel precision coordinates of target feature points are obtained through iterative calculation of image gradients.

2. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 1, characterized in that, Acquire the image to be detected, including small targets, including: A non-orthogonal multi-channel microscopic vision system is configured, including a multi-degree-of-freedom manipulator with a reflector and a refractor. Control the interaction between the reflector and the refractor to change the direction of the optical path; The non-orthogonal multi-channel microscopic vision system after optical path conversion acquires real-time images of small targets in the assembly scene to obtain the image to be detected.

3. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 1, characterized in that, The image to be detected is processed using a small target image segmentation model to generate a pixel-level segmentation mask, including: Construct a small target image segmentation model that includes a feature extraction network and a segmentation head network; A feature extraction network is used to extract hierarchical features from the image to be detected and perform stepless downsampling to output multi-scale feature maps. Multi-scale feature maps are input into the segmentation head network, and spatial location information is enhanced through a coordinate attention mechanism. Based on the enhanced multi-scale feature map, a pixel-level segmentation mask is generated; wherein, the small target image segmentation model is trained by a loss function that incorporates bounding box similarity measurement.

4. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 3, characterized in that, A feature extraction network is used to extract hierarchical features from the image to be detected, and stepless downsampling is performed to output multi-scale feature maps, including: Obtain the input feature map of the previous layer of the feature extraction network; The input feature map is divided into multiple sub-feature maps according to a preset ratio in the spatial dimension; Multiple sub-feature maps are concatenated along the channel dimension to form an intermediate feature map; Non-stepping convolutional layers are used to perform convolution operations on intermediate feature maps to output multi-scale feature maps.

5. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 3, characterized in that, The multi-scale feature map is input into the segmentation head network, and spatial location information of the multi-scale feature map is enhanced through a coordinate attention mechanism, including: Obtain multi-scale feature maps output by the feature extraction network; One-dimensional global pooling is performed along the horizontal and vertical directions of the multi-scale feature map to obtain the first feature vector and the second feature vector. The first feature vector and the second feature vector are concatenated and convolved to generate an intermediate feature map that encodes the spatial relationship. The intermediate feature map is decomposed along the spatial dimension into a horizontal attention weight map and a vertical attention weight map; The horizontal and vertical attention weight maps are multiplied and weighted with the multi-scale feature map respectively to obtain the output feature map.

6. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 3, characterized in that, The construction of the loss function that incorporates bounding box similarity metrics includes: The predicted bounding boxes and the ground bounding boxes are modeled as two-dimensional Gaussian distributions respectively; Calculate the normalized Wasserstein distance between two two-dimensional Gaussian distributions as a loss component to measure distribution similarity; Calculate the intersection-union ratio (IU) loss component between the predicted bounding box and the ground truth bounding box; The intersection-union ratio loss component and the distribution similarity metric loss component are weighted and fused to form a loss function that incorporates the bounding box similarity metric.

7. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 6, characterized in that, Calculate the intersection-union ratio (IU) loss component between the predicted bounding box and the ground truth bounding box, including: Calculate the intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box; Calculate the Euclidean distance between the center points of the predicted bounding box and the true bounding box; Calculate the width difference and height difference between the predicted bounding box and the true bounding box respectively; The cross-union ratio (CUNR) loss component is calculated based on the cross-union ratio, Euclidean distance, width difference, and height difference.

8. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in any one of claims 1 to 7, characterized in that, The pixel-level segmentation mask is subjected to contour optimization processing to obtain contour data, including: The pixel-level segmentation mask is smoothed using a Gaussian filter; Perform morphological opening operations on the smoothed pixel-level segmentation mask; Perform morphological closing operations on the pixel-level segmentation mask after morphological opening to obtain contour data.

9. The small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in claim 8, characterized in that, Based on contour data, sub-pixel precision coordinates of target feature points are obtained through image gradient iterative calculation, including: The initial pixel coordinates of the target feature points are extracted from the contour data, and the image gradient vector is calculated in the neighborhood of the initial pixel coordinates. The sub-pixel coordinate equation is established based on the image gradient and the perpendicularity of the image gradient direction to the contour edge line. The least squares method is used to iteratively calculate the equation for solving the sub-pixel coordinates, and the sub-pixel precision coordinates of the target feature points are output.

10. A small target image segmentation and detection system based on a miniature image sensing module assembly, characterized in that, The system is used to perform the small target image segmentation and detection method based on the assembly of a miniature image sensing module as described in any one of claims 1 to 9, the system comprising: The image acquisition module is configured to acquire images of the target object, including small targets. The image segmentation module is configured to process the image to be detected using a small target image segmentation model to generate a pixel-level segmentation mask; The contour optimization module is configured to perform contour optimization processing on the pixel-level segmentation mask to obtain contour data. The coordinate positioning module is configured to obtain the sub-pixel precision coordinates of target feature points through image gradient iterative calculation based on contour data.

Images