A large image target positioning method, device, equipment and storage medium

Abstract

The application provides a large image target positioning method and device, equipment and storage medium, which are applied to the technical field of image positioning. A plurality of local sub-images covering a to-be-detected object are acquired, then coarse positioning is performed on the local sub-images to quickly determine a region of interest, fine positioning is performed in the region of interest, sub-pixel level features are extracted, and local sub-pixel level coordinates are calculated, so that the positioning precision is greatly improved. Space transformation parameters between adjacent local sub-images are calculated, and the parameters are used to map local position information to a global coordinate system. The strategy of combining local fine positioning and global unified mapping avoids information loss and precision reduction caused by scaling in traditional large image processing, solves the problems of repeated detection and inaccurate splicing that may occur in slice processing, and has the advantages of improving the accuracy and efficiency of large image target positioning.

Description

Technical Field

[0001] This application relates to the field of image positioning technology, and in particular to a method, apparatus, device, and storage medium for locating large image targets. Background Technology

[0002] The detection, positioning, and screening of high-resolution, large-image sequences is a crucial step in the inkjet-printed OLED manufacturing process, and its technological level directly impacts print quality, production efficiency, and image uniformity. While various solutions exist for existing large-image recognition and positioning technologies, they all generally face several technical challenges.

[0003] For example, an existing method and system for recognizing large, high-resolution images typically involves the following steps: first, scaling the large image; then, identifying and locating the target region using a model, while extracting key features from that region; next, cropping the located target region; and finally, inputting the cropped region back into the recognition module to further extract finer-grained feature information. Through multiple iterations, complete information about the entire original image is ultimately obtained. However, this method has significant drawbacks. When scaling a large image, some key information about the target is lost, resulting in missing target details in the scaled image, which limits the localization accuracy and may even prevent the target from being detected. Furthermore, directly cropping the large image prevents the target's positional information and global feature information from being reflected in the cropped image, thus losing the target's global information.

[0004] Another target detection algorithm, such as the SAR image intelligent target detection method incorporating a lightweight latent target activation module, typically segments the large image data and performs target detection on each slice. The extracted features are input into a specific module to determine if a target exists in the slice. Finally, the results are integrated, converting the detection results of each slice into corresponding coordinates in the large image, ultimately obtaining the target detection results for the large image. Although this method can handle large images, during the detection and stitching process, overlapping regions are often included at the cropping boundaries to ensure accuracy and seamless stitching. This leads to the cross-boundary detection problem, namely, the potential for duplicate detections in overlapping areas, and the question of how to optimally select the values ​​for these duplicate detection results.

[0005] Furthermore, current image detection methods using neural networks typically require the pre-collection of a large number of sample images, and significant time and manpower are spent on sample annotation and model pre-training. However, in practical applications, the type, quantity, and distribution of markers for target components can vary considerably between different types or batches. This makes it difficult for neural network-based models to quickly detect and locate targets, and their adaptability is poor.

[0006] Therefore, existing technologies urgently need improvement to address the aforementioned problems. Summary of the Invention

[0007] In view of the shortcomings of the prior art, this application provides a method, apparatus, device and storage medium for large image target localization, which is applied to the field of image localization technology. It solves the problems of information loss, limited localization accuracy and lack of global coordinate mapping caused by the large size of the image in large image target localization, and has the advantages of improving the accuracy and efficiency of large image target localization.

[0008] A first aspect is a method for locating targets in large images, the method comprising the steps of:

[0009] S1: Obtain multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions;

[0010] S2: Perform coarse localization processing on the local sub-image, and determine the region of interest containing the target to be detected by image segmentation or template matching;

[0011] S3: Perform fine localization processing within the region of interest, extract the edge features or centroid features of the target to be detected, calculate the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and use the sub-pixel level coordinates as local position information.

[0012] S4: Calculate spatial transformation parameters based on the overlapping regions between adjacent local sub-images;

[0013] S5: Map the local position information to the global position information of the object to be detected in the global coordinate system according to the spatial transformation parameters.

[0014] Furthermore, step S1 includes:

[0015] S11: Use multiple line scan cameras to perform regional synchronous scanning of the object to be detected, and control the multiple line scan cameras to maintain the temporal consistency of image acquisition through hardware external triggering and clock synchronization mechanism;

[0016] S12: A distributed image processing architecture is adopted, which decomposes the image processing task acquired by each line scan camera into multiple sub-tasks and allocates them to the GPU for parallel computing;

[0017] S13: The subtask includes image preprocessing of the acquired raw image to obtain multiple local sub-images covering the object to be detected; the image preprocessing includes: using grayscale correction to eliminate illumination differences, using bilateral filtering for noise reduction, and performing distortion correction based on camera intrinsic parameter calibration results.

[0018] Furthermore, step S3 includes:

[0019] S31: Use the K-means clustering algorithm to segment the region of interest, and separate the target to be detected from the background based on color or grayscale features;

[0020] S32: Perform connected component analysis on the separated image to identify and remove interference points attached to the target to be detected, or fill the voids inside the target to be detected.

[0021] S33: Perform contour fitting on the image after connected component analysis, calculate the sub-pixel level geometric features of the target to be detected by Hough transform or least squares method, obtain the sub-pixel level coordinates, and use the sub-pixel level coordinates as the local position information.

[0022] Furthermore, step S4 includes:

[0023] S41: Extract image feature points from the overlapping area of ​​two adjacent local sub-images respectively;

[0024] S42: Perform feature matching on the extracted image feature points and remove mismatched points to obtain a set of matching point pairs;

[0025] S43: Calculate the spatial transformation matrix between two adjacent local sub-images based on the set of matching point pairs, and use the spatial transformation matrix as the spatial transformation parameter.

[0026] Furthermore, step S43 includes the following:

[0027] S44: Register adjacent local sub-images using the spatial transformation matrix;

[0028] S45: The registered local sub-images are subjected to layered weighted fusion in the overlapping area using the Laplacian pyramid fusion technique to eliminate stitching seams;

[0029] S46: Histogram matching technology is used to perform global color equalization on the fused image to eliminate brightness differences between regions.

[0030] Furthermore, the plurality of local sub-images includes a reference sub-image defining the global coordinate system, and step S5 includes:

[0031] S51: Determine the adjacent path between the current local sub-image containing the target to be detected and the reference sub-image;

[0032] S52: According to the adjacent path, perform concatenated multiplication on the spatial transformation matrices between all adjacent sub-images on the path to generate the global mapping matrix of the current local sub-image relative to the global coordinate system;

[0033] S53: Use the global mapping matrix to perform coordinate transformation on the local position information to obtain the global position information.

[0034] Furthermore, step S53 includes the following:

[0035] S54: Calculate the cross-union ratio of the detected targets in the overlapping area of ​​the adjacent local sub-images. If the cross-union ratio is greater than a preset threshold, they are determined to be the same target.

[0036] S55: The region weighted nonmaximum suppression algorithm is used to filter the repeatedly detected targets, and the detection results located in the center region of the local sub-image are assigned a higher weight than those in the edge region. The detection result with the highest weight is retained as the final global location information.

[0037] Secondly, a large image target localization device, the device comprising:

[0038] Acquisition module: Acquires multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions;

[0039] Coarse localization module: performs coarse localization processing on the local sub-image, and determines the region of interest containing the target to be detected by image segmentation or template matching;

[0040] The fine localization module performs fine localization processing within the region of interest, extracts the edge features or centroid features of the target to be detected, calculates the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and uses the sub-pixel level coordinates as local position information.

[0041] Calculation module: Calculates spatial transformation parameters based on the overlapping region between adjacent local sub-images;

[0042] Mapping and positioning module: Based on the spatial transformation parameters, the local position information is mapped to the global position information of the object to be detected in the global coordinate system.

[0043] Thirdly, this application provides an electronic device including a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of any of the methods described above.

[0044] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, performs the steps of any of the above methods.

[0045] Beneficial Effects: This application proposes a method, apparatus, device, and storage medium for large image target localization. By acquiring multiple local sub-images covering the object to be detected, coarse localization is performed on the local sub-images to quickly determine the region of interest (ROI), followed by fine localization within the ROI. Subpixel-level features are extracted and local subpixel-level coordinates are calculated, significantly improving localization accuracy. By calculating spatial transformation parameters between adjacent local sub-images and using these parameters to map local position information to the global coordinate system, a seamless conversion from high-precision local localization to a unified global coordinate system is achieved. This divide-and-conquer strategy, combining precise local localization with unified global mapping, avoids information loss and accuracy degradation caused by scaling in traditional large image processing. It also solves the problems of repeated detection and inaccurate stitching that may occur in slice processing, thus improving the accuracy and efficiency of large image target localization. Attached Figure Description

[0046] Figure 1 This is a flowchart of a large image target localization method proposed in this application.

[0047] Figure 2 This is a structural diagram of a large image target positioning device proposed in this application.

[0048] Figure 3 A schematic diagram of the device provided in this application.

[0049] Labeling Explanation: 201, Acquisition Module; 202, Coarse Positioning Module; 203, Fine Positioning Module; 204, Calculation Module; 205, Mapping Positioning Module; 301, Processor; 302, Memory; 303, Communication Bus; 3, Device. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and marked in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0051] In the manufacturing process of large-size, high-precision products such as organic light-emitting diode (OLED) display panels, the rapid and accurate positioning of tiny targets on the substrate, such as positioning marks or pixel slots, is a crucial technical step. Current technologies often employ a strategy of scaling the entire large image before identification and positioning when processing high-pixel, large images. However, image scaling inevitably leads to the loss of image information, especially for micrometer-level fine targets. Key edges and texture details may become blurred or even disappear during scaling, directly limiting the final positioning accuracy and, in severe cases, even causing the target to be undetectable. Another common approach is to directly crop the large image into multiple smaller blocks for independent processing. While this method preserves local details, targets located at the boundaries of image blocks are cut off during the cropping process, resulting in the loss of their complete global geometric information and contextual position information within the original large image. In addition, to ensure the integrity of the stitching, overlapping areas are usually set between adjacent image blocks, which introduces a new technical challenge: the same target may be detected repeatedly in two or more image blocks. How to effectively filter and fuse these repeated detection results to obtain a unique and optimal localization result is a problem that existing technologies have not been able to solve properly.

[0052] To overcome the problems of limited positioning accuracy, loss of global information, and repeated detection across boundaries in the existing technologies, this application provides a large image target localization method. The core idea of ​​this method is to adopt a divide-and-conquer strategy that ultimately unifies the results, ensuring both high-precision analysis of local details and accurate unification of all local results in the global coordinate system.

[0053] Please refer to Figure 1 The proposed method for large image target localization includes the following steps in its workflow:

[0054] S1: Obtain multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions;

[0055] S2: Perform coarse localization processing on local sub-images, and determine the region of interest containing the target to be detected through image segmentation or template matching;

[0056] S3: Perform fine localization processing within the region of interest, extract edge features or centroid features of the target to be detected, calculate the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and use the sub-pixel level coordinates as local position information.

[0057] S4: Calculate the spatial transformation parameters based on the overlapping regions between adjacent local sub-images;

[0058] S5: Map the local position information to the global position information of the object to be detected in the global coordinate system according to the spatial transformation parameters.

[0059] This method fundamentally avoids the information loss caused by scaling large images by acquiring local sub-images in different regions, ensuring that the original details of each target are completely preserved, thus laying a data foundation for high-precision localization. Through a two-step strategy of coarse and fine localization, it achieves sub-pixel-level accurate calculation of target positions while maintaining efficiency. More importantly, by utilizing overlapping regions to calculate spatial transformation parameters and ultimately unifying all local localization results to the global coordinate system, this method not only solves the problem of global information loss caused by image cropping but also provides an effective solution for handling cross-boundary repeated detection problems, thereby achieving comprehensive, efficient, and high-precision localization of targets in large images.

[0060] Specifically, in some embodiments, the step of acquiring multiple local sub-images covering the object to be detected can be implemented through a collaborative multi-camera acquisition system. This system utilizes multiple line scan cameras to synchronously scan the object to be detected in different regions, and controls the multiple line scan cameras to maintain temporal consistency in image acquisition through a hardware external trigger and clock synchronization mechanism. At the data processing level, a distributed image processing architecture is adopted, decomposing the image processing task acquired by each line scan camera into multiple sub-tasks and allocating them to the graphics processor for parallel computation. These sub-tasks include a series of image preprocessing steps on the acquired raw images to obtain high-quality multiple local sub-images covering the object to be detected. The image preprocessing specifically includes: grayscale correction to eliminate illumination differences caused by uneven illumination, bilateral filtering for noise reduction to preserve target edge details while suppressing noise, and distortion correction based on camera intrinsic parameter calibration results to eliminate geometric distortion caused by the lens itself.

[0061] To illustrate this process more clearly, consider a scenario applied to the inspection of large-size organic light-emitting diode (OLED) display panel substrates. First, based on the substrate width and the effective scanning width of a single line scan camera, several industrial line scan cameras are deployed side-by-side along the substrate's width, ensuring their scanning range completely covers the entire substrate, with an overlap band of, for example, 10 to 20 millimeters between the scanning areas of adjacent cameras. As the substrate moves at a constant speed on a motion platform, an external trigger signal source synchronously sends trigger signals to all line scan cameras. Simultaneously, a clock synchronization signal, for example, implemented using Genlock technology, ensures that the internal clocks of all cameras are strictly aligned. In this way, all cameras capture data from the same line on the substrate at exactly the same time, guaranteeing strict consistency in the temporal dimension of the acquired multiple local sub-images and avoiding image misalignment or distortion caused by differences in acquisition timing.

[0062] Each camera independently transmits its acquired raw image data to a dedicated data processing thread via a high-bandwidth data interface, such as a gigabit Ethernet vision interface or a high-speed serial interface. To handle the massive amounts of image data and meet the real-time requirements of the production line, the subsequent processing flow is designed as a distributed parallel computing architecture. Specifically, for each frame of image output by the camera, the processing flow is decomposed into multiple independent subtasks, such as image preprocessing, coarse localization, and fine localization. A central task scheduler, for example, based on a high-speed cache database, dynamically allocates these subtasks to multiple graphics processing unit (GPU) nodes in the computing cluster. The inherent parallel computing capabilities of the GPUs significantly accelerate the execution speed of image processing algorithms.

[0063] In the image preprocessing subtask, grayscale correction is performed first. In industrial environments, the light source may be non-uniform, or the reflective properties of the substrate surface material may differ, leading to variations in brightness across different areas of the image. Grayscale correction algorithms, such as histogram equalization or applying a pre-calibrated brightness compensation map, can effectively eliminate these differences, resulting in a more uniform grayscale distribution across the entire image. Next, denoising is performed. During image acquisition, sensor thermal noise or environmental electromagnetic interference may introduce random noise points into the image. While ordinary Gaussian filtering can remove noise, it can also blur the edges of targets, affecting positioning accuracy. Therefore, bilateral filtering is preferred. This type of filter considers not only spatial distance but also pixel similarity when calculating pixel values, thus smoothing noise while preserving important edge information in the image to the maximum extent. Finally, distortion correction is performed. Due to design and manufacturing tolerances of optical lenses, images will exhibit a certain degree of geometric distortion, such as barrel or pincushion distortion. By calibrating the intrinsic parameters of each camera beforehand, parameters describing its lens distortion characteristics can be obtained. In the preprocessing stage, these distortion parameters can be used to perform an inverse transformation on the original image, thereby correcting geometric distortions, restoring the true geometric shape of the object, and ensuring the accuracy of subsequent localization. After this series of preprocessing steps, the quality of the output local sub-images is significantly improved, providing a reliable foundation for subsequent localization analysis.

[0064] Furthermore, the steps for fine-tuning the region of interest (ROI) can include more specific operational procedures. First, the ROI is segmented using the K-means clustering algorithm, separating the target from the complex background based on pixel color or grayscale features. Then, connected component analysis is performed on the separated binarized image to identify and remove minute interference points attached to the target, or to fill in holes formed by noise or material defects within the target. Finally, the clean target contour obtained after connected component analysis is fitted, and sub-pixel-level geometric features of the target are calculated using mathematical methods such as Hough transform or least squares, thereby obtaining its sub-pixel-level coordinates in the local sub-image coordinate system, and these coordinates are used as local position information.

[0065] Taking the location of a white circular marker as an example, this marker is located on a substrate background with complex metal wiring and residual ink stains. After coarse localization determines the approximate area containing the marker, the fine localization process begins. The first step is image segmentation. The K-means clustering algorithm is used here as an unsupervised pixel classification tool. The number of clusters is set to two: target class and background class. The algorithm randomly initializes two cluster centroids, then traverses each pixel in the region of interest, calculates the distance between its gray value and the two cluster centroids, and classifies the pixel into the closer cluster. After classifying all pixels, the mean gray value of all pixels in each cluster is recalculated, and this new mean is used as the new cluster centroid. This iterative process of classification and updating centroids is repeated until the position of the cluster centroid no longer changes significantly, at which point the clustering is complete. Since the location markers are pure white with high grayscale values, while the background interference such as metal wiring and ink stains are darker and have lower grayscale values, the K-means algorithm can effectively segment the image into white regions representing location markers and black regions representing the background and interference.

[0066] The second step is morphological processing and connected component analysis. In the segmented image, the white areas representing the location markers may not be perfect. They may have tiny black ink droplets that were not completely separated, forming internal holes; or there may be tiny white noise spots around the markers that were mistakenly identified as targets. The connected component analysis algorithm scans the entire image to find all regions composed of interconnected pixels. For white areas, the area of ​​each connected component can be calculated. By setting an area threshold, the larger location marker areas representing the main subject can be effectively preserved, while isolated white noise spots with areas much smaller than the threshold can be removed. For black areas, their connected components can also be analyzed. Those small black connected components located inside the white markers can be identified as holes that need to be filled, and these holes are filled by inverting their pixel values ​​to white. After this step, a binary image of the location markers with clear outlines and complete internal structures is obtained.

[0067] The third step is subpixel contour fitting. Using edge detection operators, such as the Canny operator, the contour point set of the clean target image is extracted. To clearly explain the method for locating and marking contour features, this application provides two detailed algorithms, as follows:

[0068] The first method involves performing line detection on the positioning markers that contain straight line features. In line detection, the formula used to transform a point (x, y) in the rectangular coordinate system to the polar coordinate parameter space is: ρ = xcosθ + ysinθ, where (x, y) are the coordinates of the point in the rectangular coordinate system, ρ is the perpendicular distance from the origin to the line, and θ is the angle between the perpendicular line and the x-axis.

[0069] A line detection threshold is set based on the length of the positioning marker to filter peak values ​​in the accumulator array. Only parameter combinations with more than the threshold are considered valid line parameters.

[0070] Iterate through the accumulator array to find all parameter combinations (ρ, θ) where the number of votes exceeds the threshold. These parameter combinations correspond to straight lines in the image. The general formula for detecting straight lines is: Ax + By + C = 0, where parameters A and B are the components of the normal vector of the line on the x-axis and y-axis, respectively, A = cosθ, B = sinθ, and C is a constant representing the distance from the origin to the line, i.e., C = -ρ.

[0071] Contour feature points arranged in a straight line at the same angle often detect multiple straight lines. By filtering out sets of straight lines with similar slopes and distances, and calculating the average value of these straight line parameters, they are merged into a single straight line, Line.

[0072] The second method involves performing circle detection on the circular positioning mark. This can be done by performing a Hough transformation on a point (x, y) in a rectangular coordinate system. The equation of the circle after the Hough transformation is: x = a + rcosθ; y = b + rsinθ. This equation describes the relationship between any point (x, y) on the circle and the center (a, b). With the center (a, b) as the reference and the radius r as a fixed distance, as the angle θ changes from 0 to 2π, the point (x, y) rotates counterclockwise around the circumference for one revolution.

[0073] Then, the circular features of the localization markers in the image space are mapped to the parameter space, and the circles are detected using an accumulator. Specifically, the distance from each non-zero pixel in the image to circles with different centers and radii is calculated, and these distances are mapped to the parameter space. The point with the highest accumulated value in the parameter space is found, and the center and radius corresponding to that point are recorded.

[0074] For contour feature points with the same center, multiple circles are often detected. By filtering out the set of circles with similar centers and radii, and calculating the average value of these circle parameters, they are merged into a single circle.

[0075] Using the detection circle and detection line of the positioning mark detected in the above steps, feature points of the positioning mark contour near the detection line Line and detection circle Circle are extracted to form a feature point set Line_set or Circle_set. This can eliminate some interfering feature points in the positioning mark target contour.

[0076] To obtain the closest straight line and circle to the actual positioning marker outline, it is necessary to perform line fitting and circle fitting on the feature point set Line_set or Circle_set selected in the above process, respectively.

[0077] In line fitting, firstly, two points are randomly selected from the feature set Line_set dataset, and the parametric model of the line formed by them is solved. Then, the distance from the remaining points in the dataset to the line is calculated. Points whose distance is less than the set distance threshold are interior points, and the number of interior points in the current line model is counted. Next, two more points are randomly selected, and the number of interior points is counted again. Iterative calculations are performed. The dataset with the most interior points is the maximum consistent set. Finally, a line is fitted from the points in the maximum consistent set using least squares.

[0078] The linear fitting equation is in slope-intercept form: y = mx + B, where m is the slope of the line and B is the y-intercept of the line.

[0079] The formula for calculating the number of iterations K is as follows: , where n is the number of randomly selected interior points, w is the probability of the true interior point group, and P is the probability of success after K iterations of the algorithm.

[0080] In circle fitting, three non-collinear points are randomly selected from the feature `Circle_set` dataset. The circle defined by these three points is a model of an interior set of points. This model is then used to test other points in the dataset. If the distance from a point to the circle is less than a certain threshold, that point is considered an interior point; otherwise, it is an exterior point. This process is repeated multiple times, selecting three different points in each iteration to construct a new circle model and validating the points in the dataset. When the preset number of iterations is reached or a sufficiently good model is found, the parameters of the optimal model are output: the center of the circle. And the radius r, the calculation formula is as follows:

[0081] .in, This represents the mean of the x-coordinates of the interior point set. This represents the mean of the ordinates of the set of interior points. Indicates each interior point relative to the center of mass. The average value of the polar angle (the angle between the polar angle and the x-axis).

[0082] Finally, the result of the fitted straight line or fitted circle calculation is used as the positioning mark to locate the secondary positioning center position P2, and the processing result is displayed on the control software interface and stored in the database.

[0083] Before mapping the target location information in each local sub-image to the global coordinate system, the spatial correspondence between adjacent local sub-images must be accurately calculated. Therefore, in some embodiments, the step of calculating spatial transformation parameters based on the overlapping region between adjacent local sub-images specifically includes: First, extracting image feature points within the overlapping region of two adjacent local sub-images. Then, performing feature matching on the extracted image feature points and using a robust algorithm to eliminate mismatched points to obtain a highly reliable set of matching point pairs. Finally, based on this set of matching point pairs, calculating the spatial transformation matrix describing the geometric transformation relationship between the two local sub-images using mathematical methods, and using this matrix as the required spatial transformation parameters.

[0084] This process is the core of image registration. The first step is feature point extraction. Within the overlapping region shared by two adjacent local sub-images, it is necessary to find some unique and stable points as the basis for subsequent matching. For example, the ORB algorithm can be used. This algorithm combines FAST corner detection and BRIEF feature description, which can quickly detect a large number of key points in the image and generate a binary string descriptor for each key point. This descriptor is invariant to image rotation and certain scale and brightness changes. Therefore, for the overlapping region of two sub-images, the ORB algorithm is executed separately to obtain two sets of key points and their corresponding descriptors.

[0085] The second step is feature matching and mismatch removal. After obtaining two sets of feature point descriptors, matching can be performed. A common method is to calculate the Hamming distance between the two sets of descriptors, which is the number of different characters at corresponding positions in the two binary strings. For each feature point in image one, the feature point with the smallest Hamming distance in image two is found. If this smallest distance is less than a preset threshold, the two feature points are considered to form an initial matching pair. However, due to the possibility of repetitive textures or noise in the images, the initial matching results will inevitably contain some incorrect matches. To improve the accuracy of registration, these mismatches must be removed. At this point, the RANSAC algorithm can be introduced. The RANSAC algorithm works as follows: A small subset of matching points, such as four pairs, is randomly selected from the initial set of matching pairs. Assuming this small subset is correct, a spatial transformation model, such as an affine transformation matrix, is calculated based on it. Then, this model is used to test all other matching pairs, counting how many pairs conform to the model (i.e., their transformation error is within a small tolerance range). These points are called inliers. This process of random sampling, modeling, and testing is repeated multiple times, and finally, the model with the most inliers is selected as the optimal model. All matching pairs that do not conform to the optimal model are considered mismatches and are discarded.

[0086] The third step is the precise calculation of the spatial transformation matrix. After filtering using the RANSAC algorithm, a clean set of matching point pairs consisting of interior points is obtained. Using all the matching point pairs in this set, a more accurate spatial transformation matrix can be recalculated through optimization methods such as least squares. This matrix can mathematically describe the mapping relationship from one local sub-image coordinate system to another, including translation, rotation, scaling, and even affine or perspective transformations. This calculated spatial transformation matrix is ​​the spatial transformation parameter required for subsequent global coordinate mapping.

[0087] After calculating the spatial transformation matrix between adjacent local sub-images, in addition to its use for coordinate transformation, it can be further used to generate a high-quality, seamlessly stitched global image, which is of great significance for manual review or global analysis. Therefore, in some preferred embodiments, after calculating the spatial transformation matrix, the method further includes the following steps: First, geometrically register and align adjacent local sub-images using the calculated spatial transformation matrix. Then, use Laplacian pyramid fusion technology to perform layered weighted fusion of the registered local sub-images in overlapping areas to smoothly eliminate stitching seams. Finally, use histogram matching technology to perform global color equalization processing on the fused entire image to eliminate brightness differences between different sub-image regions.

[0088] Specifically, the registration step involves using the previously calculated spatial transformation matrix to perform a geometric transformation on one of the local sub-images, ensuring that its pixel grid is perfectly aligned with that of the other local sub-image within the overlapping region. However, even with geometric alignment, slight differences in exposure and gain between different cameras during acquisition, or spatially incomplete uniformity of illumination, can lead to differences in brightness or hue in the overlapping region between the two images. Direct stitching will then produce a noticeable seam.

[0089] To address this issue, a Laplacian pyramid fusion technique is introduced. This technique first decomposes the two images to be fused into a series of image layers with different spatial frequencies, forming a Laplacian pyramid. The bottom layer of the pyramid contains high-frequency information, i.e., image details and edges, while the top layer contains low-frequency information, i.e., image contours and background. Then, at each layer of the pyramid, corresponding layers of the two images are fused. The fusion method typically involves weighted averaging within the overlapping region, with weights smoothly transitioning from the center of one image to the edge of the other. For example, in a left-to-right stitching, the fusion weight for pixels within the overlapping region can be designed to vary linearly with their horizontal coordinates; the weight of the left image decreases from 1 to 0 as the coordinate increases, while the weight of the right image increases from 0 to 1 as the coordinate increases. After weighted fusion is completed at all levels, the inverse transformation of the pyramid reconstructs the fused images into a single, seamless image.

[0090] Finally, even if the stitching seams are eliminated, the entire large image may still exhibit blocky brightness unevenness. In this case, histogram matching technology can be used for global color equalization. This technique first calculates a target histogram for the entire large image; this target histogram can be an ideal distribution or the histogram of a specific sub-image region. Then, the grayscale values ​​of each pixel in the image are adjusted so that the adjusted grayscale histogram of the entire image matches the target histogram as closely as possible. Through this process, macroscopic brightness differences between areas captured by different cameras can be effectively eliminated, resulting in a visually uniform and consistent global image.

[0091] Once the target location information in all local sub-images and the spatial transformation parameters between adjacent sub-images are ready, the final step is to integrate this scattered local information into a unified global coordinate system. In some embodiments, multiple local sub-images include a reference sub-image defined as the origin of the global coordinate system. The steps of mapping local location information to global location information specifically include: First, determining the physical adjacency path between the local sub-image containing the current target and the reference sub-image. Then, following this adjacency path, performing a concatenated multiplication operation on the spatial transformation matrices between all adjacent sub-images along the path to generate a global mapping matrix of the current local sub-image relative to the global coordinate system. Finally, using this global mapping matrix to perform coordinate transformation on the local location information measured in the current local sub-image, the global location information of the target in the global coordinate system can be obtained.

[0092] For ease of understanding, we can define the local sub-image captured by the top-left camera as the reference sub-image, and its coordinate system as the global coordinate system. Suppose we need to determine the position of a target located in the fifth local sub-image within the global coordinate system, and the camera arrangement is one-dimensional linear. Then, the adjacent path from the fifth sub-image to the reference sub-image is: Fifth sub-image -> Fourth sub-image -> Third sub-image -> Second sub-image -> First sub-image (reference). Since the transformation matrices between adjacent images have already been calculated, denoted as T_45 (transformation from coordinate system 5 to 4), T_34 (transformation from 4 to 3), T_23 (transformation from 3 to 2), and T_12 (transformation from 2 to 1), to obtain the global mapping matrix T_15 directly from coordinate system 5 to coordinate system 1, we need to multiply these transformation matrices sequentially according to the path order: T_15 = T_12 × T_23 × T_34 × T_45. The result T_15 of this cascaded multiplication method contains all the geometric transformation information, such as translation and rotation, required to move from the fifth local sub-image coordinate system to the global coordinate system.

[0093] Once the global mapping matrix T_15 is obtained, the coordinate transformation becomes very straightforward. The local position coordinates of the target measured in the fifth sub-image are represented as a homogeneous coordinate vector, and then this vector is multiplied on the left by the global mapping matrix T_15. The result is a new homogeneous coordinate vector, whose first two components are the precise coordinates of the target in the global coordinate system, i.e., the global position information. By performing this process on all targets detected in all local sub-images, all localization results can be integrated into a unified global map.

[0094] After mapping all targets to the global coordinate system, an important issue needs to be addressed: the problem of targets in overlapping regions being detected repeatedly. In some embodiments, after obtaining the global location information, the method further includes a filtering and fusion step: First, the intersection-union ratio (IUR) of the detected targets in the overlapping regions of adjacent local sub-images is calculated. If the IUR is greater than a preset threshold, these detection results are determined to point to the same physical target. Then, a region-weighted non-maximum suppression algorithm is used to filter these targets determined to be repeatedly detected. Specifically, the detection results located in the center region of the local sub-image are assigned a higher weight than those in the edge region, and finally only the detection result with the highest weight is retained as the final global location information of that target.

[0095] The specific operation process is as follows: After obtaining the global location information list of all targets, traverse this list. For any two target detection boxes, if they are detected in physically adjacent local sub-images, and their intersection-union ratio (IUU) in the global coordinate system (i.e., the area of ​​the intersection of the two rectangles divided by the area of ​​the union) is greater than a set threshold, such as 0.5, then these two detection results are classified into the same cluster, indicating that they may point to the same target.

[0096] After identifying multiple duplicate detection results pointing to the same target, the optimal one needs to be selected. Traditional non-maximum suppression algorithms typically make decisions based solely on the detection confidence score. However, in the scenario described in this application, we can leverage prior knowledge: the image quality in the central region of the camera's field of view is generally better than that in the edge regions, which are more susceptible to lens distortion, vignetting, and other factors, leading to decreased positioning accuracy. Therefore, this application proposes an improved region-weighted non-maximum suppression algorithm. When making decisions, this algorithm considers not only the confidence score of each detection result but also multiplies it by a weight coefficient related to its position in the original local sub-image. For example, a two-dimensional Gaussian function can be defined as the weight function, with its center at the center of the local sub-image and lower weights at the edges. The final evaluation value of a detection result is the product of its original confidence score and this positional weight coefficient. For groups of detection results judged as duplicates, the final evaluation value of each result is calculated, and the one with the highest evaluation value is retained, while all other detection results are suppressed. In this way, it can be ensured that the final global location information is not only unique, but also comes from the most reliable imaging area, thereby further improving the accuracy and robustness of the overall positioning.

[0097] Please refer to Figure 2 This application also provides a large image target localization device, the device comprising:

[0098] Acquisition module 201: Acquires multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions;

[0099] Coarse localization module 202: Performs coarse localization processing on local sub-images, and determines the region of interest containing the target to be detected through image segmentation or template matching;

[0100] Precision localization module 203: Performs precision localization processing within the region of interest, extracts edge features or centroid features of the target to be detected, calculates the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and uses the sub-pixel level coordinates as local position information;

[0101] Calculation module 204: Calculates spatial transformation parameters based on the overlapping regions between adjacent local sub-images;

[0102] Mapping and positioning module 205: Maps local position information to global position information of the object to be detected in the global coordinate system according to spatial transformation parameters.

[0103] Specifically, the acquisition module 201 may consist of one or more image acquisition devices, such as a line scan camera, an area scan camera, or a high-resolution scanner, configured to capture multiple local sub-images covering the object to be detected. The acquisition module 201 may also include an image data interface for receiving pre-acquired image data from external storage or a network.

[0104] The coarse localization module 202 can be a dedicated image processor, a programmable gate array (FPGA), or a graphics processing unit (GPU), configured to execute image segmentation algorithms (such as threshold-based segmentation, region growing, or edge detection segmentation) or template matching algorithms to identify and determine regions of interest containing the target object in local sub-images. This module can also be a software program running on a general-purpose processor, implementing the above functions by calling appropriate image processing libraries.

[0105] The fine localization module 203 can be integrated with the coarse localization module or exist as a standalone computational unit configured to perform more refined image analysis within the region of interest determined by coarse localization. For example, this module can extract edge features of the target object using sub-pixel edge detection algorithms (such as Zernike moments or Gaussian fitting), or determine centroid features by calculating a weighted average of pixel grayscale values. The output of this module is the sub-pixel-level coordinates of the target object in the local sub-image coordinate system, serving as local positional information.

[0106] The computation module 204 can be a microcontroller or a digital signal processor (DSP) configured to analyze overlapping regions between adjacent local sub-images. This module can execute feature point detection algorithms (such as SIFT, SURF, ORB) to extract image feature points within the overlapping regions and perform feature matching. Subsequently, it uses geometric transformation models (such as affine transformation and perspective transformation) to calculate spatial transformation parameters, such as spatial transformation matrices, describing the geometric relationship between adjacent local sub-images.

[0107] The mapping and localization module 205 can be a central processing unit (CPU) or an embedded processor, configured to receive local position information output by the fine localization module and spatial transformation parameters output by the calculation module. This module maps the local position information from the coordinate systems of each local sub-image to a unified global coordinate system by executing a coordinate transformation algorithm, thereby obtaining the global position information of the object to be detected. This module can also manage and store the localization results in the global coordinate system.

[0108] The large image target localization device proposed in this application represents a significant advancement compared to existing technologies. Traditional methods, involving scaling or direct cropping of large images, often result in the loss of key target information or global information, thus limiting localization accuracy. The device in this application, however, acquires local sub-images by region using the acquisition module 201, and then performs high-precision localization on these sub-images using the coarse localization module 202 and the fine localization module 203, effectively avoiding information loss caused by scaling the entire large image. Furthermore, the calculation module 204 precisely calculates the spatial transformation parameters between adjacent local sub-images, and the mapping localization module 205 uniformly maps the local position information to the global coordinate system, ensuring the accurate position of the target globally and solving the problem of global information loss. This modular design makes the entire localization process more efficient and accurate, effectively addressing the challenges of high-pixel large image detection, localization, and screening, and significantly improving the accuracy and reliability of detection and localization.

[0109] Please refer to Figure 3 , Figure 3 This application provides a schematic diagram of the structure of a device 3, comprising a processor 301 and a memory 302. The processor 301 and the memory 302 are interconnected and communicate with each other via a communication bus 303 and / or other forms of connection mechanism (not shown). The memory 302 stores computer-readable instructions executable by the processor 301. When the device 3 is running, the processor 301 executes the computer-readable instructions to perform the method in any optional implementation of the above embodiments, thereby achieving the following functions: acquiring multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions; performing coarse localization processing on the local sub-images, determining the region of interest containing the target to be detected through image segmentation or template matching; performing fine localization processing within the region of interest, extracting edge features or centroid features of the target to be detected, calculating the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and using the sub-pixel level coordinates as local position information; calculating spatial transformation parameters based on the overlapping regions between adjacent local sub-images; and mapping the local position information to global position information in the global coordinate system of the target to be detected based on the spatial transformation parameters.

[0110] This application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it performs the method in any optional implementation of the above embodiments to achieve the following functions: acquiring multiple local sub-images covering an object to be detected, wherein adjacent local sub-images have overlapping regions; performing coarse localization processing on the local sub-images, determining the region of interest containing the target to be detected through image segmentation or template matching; performing fine localization processing within the region of interest, extracting edge features or centroid features of the target to be detected, calculating the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and using the sub-pixel level coordinates as local position information; calculating spatial transformation parameters based on the overlapping regions between adjacent local sub-images; and mapping the local position information to global position information in the global coordinate system of the target to be detected based on the spatial transformation parameters.

[0111] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0112] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0114] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0115] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for target localization in large images, characterized in that, The method includes the following steps: S1: Obtain multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions; S2: Perform coarse localization processing on the local sub-image, and determine the region of interest containing the target to be detected by image segmentation or template matching; S3: Perform fine localization processing within the region of interest, extract the edge features or centroid features of the target to be detected, calculate the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and use the sub-pixel level coordinates as local position information. Step S3 includes: S31: Use the K-means clustering algorithm to segment the region of interest, and separate the target to be detected from the background based on color or grayscale features; S32: Perform connected component analysis on the separated image to identify and remove interference points attached to the target to be detected, or fill the voids inside the target to be detected. S33: Perform contour fitting on the image after connected component analysis, calculate the sub-pixel level geometric features of the target to be detected by Hough transform or least squares method, obtain the sub-pixel level coordinates, and use the sub-pixel level coordinates as the local position information; S4: Calculate spatial transformation parameters based on the overlapping regions between adjacent local sub-images; S5: Map the local position information to the global position information of the object to be detected in the global coordinate system according to the spatial transformation parameters.

2. The method for locating a large image target according to claim 1, characterized in that, Step S1 includes: S11: Use multiple line scan cameras to perform regional synchronous scanning of the object to be detected, and control the multiple line scan cameras to maintain the temporal consistency of image acquisition through hardware external triggering and clock synchronization mechanism; S12: A distributed image processing architecture is adopted, which decomposes the image processing task acquired by each line scan camera into multiple sub-tasks and allocates them to the GPU for parallel computing; S13: The subtask includes image preprocessing of the acquired raw image to obtain multiple local sub-images covering the object to be detected; the image preprocessing includes: using grayscale correction to eliminate illumination differences, using bilateral filtering for noise reduction, and performing distortion correction based on camera intrinsic parameter calibration results.

3. The method for locating a large image target according to claim 1, characterized in that, Step S4 includes: S41: Extract image feature points from the overlapping area of ​​two adjacent local sub-images respectively; S42: Perform feature matching on the extracted image feature points and remove mismatched points to obtain a set of matching point pairs; S43: Calculate the spatial transformation matrix between two adjacent local sub-images based on the set of matching point pairs, and use the spatial transformation matrix as the spatial transformation parameter.

4. The method for locating a large image target according to claim 3, characterized in that, Step S43 is followed by: S44: Register adjacent local sub-images using the spatial transformation matrix; S45: The registered local sub-images are subjected to layered weighted fusion in the overlapping area using the Laplacian pyramid fusion technique to eliminate stitching seams; S46: Histogram matching technology is used to perform global color equalization on the fused image to eliminate brightness differences between regions.

5. The method for locating a large image target according to claim 3, characterized in that, The plurality of local sub-images includes a reference sub-image that defines the global coordinate system. Step S5 includes: S51: Determine the adjacent path between the current local sub-image containing the target to be detected and the reference sub-image; S52: According to the adjacent path, perform concatenated multiplication on the spatial transformation matrices between all adjacent sub-images on the path to generate the global mapping matrix of the current local sub-image relative to the global coordinate system; S53: Use the global mapping matrix to perform coordinate transformation on the local position information to obtain the global position information.

6. The method for locating a large image target according to claim 5, characterized in that, Step S53 is followed by: S54: Calculate the cross-union ratio of the detected targets in the overlapping area of ​​the adjacent local sub-images. If the cross-union ratio is greater than a preset threshold, they are determined to be the same target. S55: The region weighted nonmaximum suppression algorithm is used to filter the repeatedly detected targets, and the detection results located in the center region of the local sub-image are assigned a higher weight than those in the edge region. The detection result with the highest weight is retained as the final global location information.

7. A large image target positioning device, characterized in that, The device includes: Acquisition module: Acquires multiple local sub-images covering the object to be detected, wherein adjacent local sub-images have overlapping regions; Coarse localization module: performs coarse localization processing on the local sub-image, and determines the region of interest containing the target to be detected by image segmentation or template matching; The fine localization module performs fine localization processing within the region of interest, extracts the edge features or centroid features of the target to be detected, calculates the sub-pixel level coordinates of the target to be detected in the local sub-image coordinate system, and uses the sub-pixel level coordinates as local position information. The fine localization module is further configured to: segment the region of interest using the K-means clustering algorithm; separate the target from the background based on color or grayscale features; perform connected component analysis on the separated image to identify and remove interference points attached to the target, or fill in holes inside the target; perform contour fitting on the image after connected component analysis; calculate the sub-pixel level geometric features of the target using Hough transform or least squares method to obtain the sub-pixel level coordinates; and use the sub-pixel level coordinates as the local position information. Calculation module: Calculates spatial transformation parameters based on the overlapping region between adjacent local sub-images; Mapping and positioning module: Based on the spatial transformation parameters, the local position information is mapped to the global position information of the object to be detected in the global coordinate system.

8. A device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1-6.

Images