Positioning method, device and storage medium

Abstract

The application provides a positioning method, device and storage medium. The method comprises the following steps: obtaining an image and a category of an object to be positioned; determining a segmentation mode corresponding to the category from multiple segmentation modes, inputting the image into a segmentation model, and segmenting the image in the segmentation mode corresponding to the category by using the segmentation model to obtain multiple candidate masks of the object to be positioned output by the segmentation model; performing hierarchical screening on the multiple candidate masks according to a pre-set geometric feature threshold to obtain a target mask of the object to be positioned; determining a local region corresponding to the target mask in the image from the image of the object to be positioned, and performing edge enhancement processing on the local region to obtain an edge contour image, wherein the edge contour image comprises an edge contour of the object to be positioned; and determining positioning information of the object to be positioned in the image according to the edge contour image. The application realizes efficient and accurate positioning of the object to be positioned.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a positioning method, device and storage medium. Background Technology

[0002] In industrial automated assembly operations, visual positioning of components is a necessary prerequisite for achieving precise assembly. Inaccurate positioning will directly lead to assembly failure, component damage, or even system downtime. Therefore, achieving accurate positioning of components is of paramount importance. Summary of the Invention

[0003] This application provides a positioning method, device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] A first aspect of this application provides a positioning method, the method comprising: Obtain the image and category of the object to be located; From multiple segmentation modes, a segmentation mode corresponding to the category is determined, and the image is input into a segmentation model. The segmentation model is used to segment the image using the segmentation mode corresponding to the category to obtain multiple candidate masks of the object to be located output by the segmentation model. The multiple candidate masks are hierarchically filtered according to a pre-set geometric feature threshold to obtain the target mask of the object to be located. From the image of the object to be located, determine the local region corresponding to the target mask in the image, and perform edge enhancement processing on the local region to obtain an edge contour image, wherein the edge contour image includes the edge contour of the object to be located; The positioning information of the object to be located in the image is determined based on the edge contour image.

[0005] In one possible implementation, the category includes a first category and a second category; the first category indicates that the position of the object to be located is fixed, and the second category indicates that the position of the object to be located is not fixed. The process of segmenting the image using a segmentation model with a segmentation pattern corresponding to the category yields multiple candidate masks for the object to be located, output by the segmentation model, including: In response to the category being the first category, the image is segmented using the segmentation model in a full-image segmentation mode to obtain multiple candidate masks output by the segmentation model; In response to the category being the second category, the image is segmented using the segmentation model in a guided segmentation mode to obtain multiple candidate masks output by the segmentation model.

[0006] In one possible implementation, segmenting the image using the segmentation model in a full-image segmentation mode to obtain multiple candidate masks output by the segmentation model includes: The image is input into the segmentation model to obtain multiple candidate masks output by the segmentation model; wherein the segmentation model performs full-image segmentation on the image to generate the multiple candidate masks.

[0007] In one possible implementation, segmenting the image using the segmentation model with a guided segmentation mode to obtain multiple candidate masks output by the segmentation model includes: Obtain the coordinates of at least one prompt point of the object to be located and the point category corresponding to each prompt point coordinate, wherein the point category includes foreground and background; The image, the coordinates of the at least one cue point, and the point category corresponding to each cue point coordinate are input into the segmentation model to obtain multiple candidate masks output by the segmentation model; wherein, the segmentation model generates the multiple candidate masks by segmenting the image based on the cue point coordinates and point categories.

[0008] In one possible implementation, the preset geometric feature thresholds include an area threshold, an aspect ratio threshold, and a fill rate threshold; The step of hierarchically filtering the multiple candidate masks according to a pre-set geometric feature threshold to obtain the target mask of the object to be located includes: Calculate the mask area and aspect ratio of the directed minimum bounding rectangle for each candidate mask; Calculate the fill rate of the area of ​​each candidate mask and the area of ​​the directed minimum bounding rectangle; The candidate masks are filtered step by step according to the area threshold, the aspect ratio threshold, and the fill rate threshold, and the candidate mask with the largest area among the filtered candidate masks is determined as the target mask.

[0009] In one possible implementation, the edge enhancement processing of the local region to obtain an edge contour image includes: Super-resolution reconstruction is performed on the local region to obtain a resolution-enhanced image; The resolution-enhanced image is subjected to adaptive contrast enhancement processing; Median filtering is applied to the contrast-enhanced image for noise reduction. The median-filtered image is locally binarized within the foreground region defined by the target mask to obtain a binarized image; A morphological opening operation is performed on the binarized image, and the difference between the morphological opening operation result and the binarized image is obtained to obtain the edge image; The edge image is subjected to connected component analysis to obtain at least one connected component, and connected components with an area smaller than a preset threshold are removed to obtain the edge contour image.

[0010] In one possible implementation, the positioning information includes the center coordinates and rotation angle of the object to be positioned; The step of determining the positioning information of the object to be located in the image based on the edge contour image includes: Calculate the directed minimum bounding rectangle of the edge contour image; Based on the directed minimum bounding rectangle, obtain the center coordinates and rotation angle of the object to be located in the image.

[0011] In one possible implementation, before segmenting the image using a segmentation model with a segmentation pattern corresponding to the category, the method further includes: The image is pre-cropped by the region of interest to obtain a cropped image; The step of segmenting the image using a segmentation model with a segmentation pattern corresponding to the category includes: The cropped image is segmented using a segmentation model with a segmentation pattern corresponding to the category.

[0012] A second aspect of this application provides a positioning device, the device comprising: The acquisition module is used to acquire the image and category of the object to be located; The segmentation module is used to determine the segmentation mode corresponding to the category from multiple segmentation modes, input the image into the segmentation model, and use the segmentation model to segment the image with the segmentation mode corresponding to the category to obtain multiple candidate masks of the object to be located output by the segmentation model. The filtering module is used to perform hierarchical filtering of the multiple candidate masks according to a pre-set geometric feature threshold to obtain the target mask of the object to be located. An edge enhancement module is used to determine the local region corresponding to the target mask in the image of the object to be located, and to perform edge enhancement processing on the local region to obtain an edge contour image, wherein the edge contour image includes the edge contour of the object to be located. An extraction module is used to determine the positioning information of the object to be located in the image based on the edge contour image.

[0013] A third aspect of this application provides an electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0014] A fourth aspect of this application provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the methods described in this application.

[0015] This application discloses a positioning method, device, and storage medium that acquires an image and category of an object to be located; determines a segmentation mode corresponding to the category from multiple segmentation modes, inputs the image into a segmentation model, and uses the segmentation model to segment the image using the segmentation mode corresponding to the category, obtaining multiple candidate masks of the object to be located output by the segmentation model; performs hierarchical filtering on the multiple candidate masks according to a pre-set geometric feature threshold to obtain a target mask of the object to be located; determines the local region corresponding to the target mask in the image of the object to be located, and performs edge enhancement processing on the local region to obtain an edge contour image, the edge contour image including the edge contour of the object to be located; and determines the positioning information of the object to be located in the image based on the edge contour image. This achieves efficient and accurate positioning of the object to be located.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0017] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0018] Figure 1 A schematic diagram illustrating the implementation flow of the positioning method provided in an embodiment of this application is shown; Figure 2 A structural diagram of the SAM model provided in an embodiment of this application is shown; Figure 3 This illustration shows a schematic diagram of image preprocessing provided in an embodiment of this application; Figure 4 This illustration shows a schematic diagram of image block grid division provided in an embodiment of this application; Figure 5This illustration shows a schematic diagram of a flattened one-dimensional representation of an image block provided in an embodiment of this application; Figure 6 This illustration shows a target mask and its corresponding local area provided in an embodiment of this application; Figure 7 A schematic diagram illustrating the implementation flow of the edge enhancement operation of the positioning method provided in this application embodiment is shown; Figure 8 This paper illustrates the effects of each stage of edge enhancement processing in the positioning method provided in the embodiments of this application. Figure 9 This paper presents a schematic diagram comparing the image cropping effects provided in an embodiment of this application. Figure 10 A schematic diagram of the composition structure of the positioning device provided in an embodiment of this application is shown; Figure 11 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0019] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] First, the application scenarios of this application are described. This application is mainly applied to plug-in assembly scenarios in industrial automated assembly operations, specifically covering automated plugging and unplugging, alignment and docking of computer (PC) I / O interfaces and various electronic connectors. This involves precise alignment operations that use vision guidance to accurately assemble connectors to their corresponding interfaces. These assembly operations require extremely high positioning accuracy; any slight deviation in position or angle can lead to assembly failure or even damage to components.

[0021] Taking the quality verification stage in the development of laptops as an example, during the overall reliability testing phase, rigorous insertion and removal durability tests must be performed on various I / O interfaces (such as USB-A, USB-C, HDMI, DC power ports, etc.) to verify the interface structural strength, electrical contact stability, and long-term reliability. This test typically requires thousands to tens of thousands of repeated insertion and removal operations over several days to weeks, representing a typical high-load, long-cycle, and standardized verification task. Currently, this testing process still largely relies on manual operation, with testers manually inserting and removing standard devices (such as USB flash drives, video cables, power plugs, etc.) repeatedly from the interface under test. Although manual operation has sufficient alignment capability in a single operation, it faces significant bottlenecks in large-scale repetitive testing scenarios: low testing efficiency, as manual operation cannot be carried out continuously for 24 hours, severely slowing down the verification cycle; high labor costs, requiring dedicated personnel, consuming valuable human resources; insufficient standardization, as insertion and removal force, speed, and pause time are difficult to strictly unify, affecting the consistency and comparability of test results; and long-term repetitive operation can easily lead to fatigue, counting errors, or missed tests, reducing test reliability.

[0022] To improve verification efficiency, ensure testing standardization, and free up human resources, the company introduced an automated plug-in / plug-out testing system. However, the automated system relies on a robotic arm to perform the plug-in / plug-out actions, but the robotic arm itself lacks the visual-tactile coordination and compliant adjustment capabilities of humans. It must rely on a highly robust vision system to precisely guide the connector alignment with the interface. Therefore, a machine vision-based automated PC I / O interface plug-in / plug-out solution was introduced. This solution relies on high-precision image acquisition and template matching algorithms to achieve the alignment and positioning of the plug and socket. However, after applying this solution in actual industrial applications, it was found that this type of solution exposed significant limitations, making it difficult to meet the requirements of high reliability, strong adaptability, and long-term stable operation. Specific defects are as follows: First, traditional solutions are highly dependent on imaging capabilities. Camera position, focal length, and exposure parameters must be kept strictly consistent; even minor mechanical vibrations, equipment drift, or fluctuations in optical parameters can cause image defocusing or feature shift, leading to positioning failure. Second, the solutions have poor generalization ability and compatibility. They often struggle to effectively identify assemblies of different models, colors, materials, and even those with wear or contamination. Typically, a dedicated visual template needs to be designed and maintained for each type of assembly, significantly limiting their applicability in multi-variety, small-batch production scenarios. Third, changes in ambient lighting pose a severe challenge to system robustness. Common factors in industrial environments, such as strong light reflection, shadow occlusion, or uneven lighting, can easily interfere with image quality, leading to inaccurate feature extraction, ultimately affecting assembly accuracy, and even causing operational interruptions or equipment damage.

[0023] A deeper problem lies in the inherent flaws of the template matching method, the core localization technique. This method is essentially a rigid comparison strategy based on pixel-level similarity, requiring the test image and the pre-stored template to be highly consistent in scale, rotation, viewpoint, and lighting conditions. However, in actual assembly processes, components often experience translation, rotation, or even slight deformation due to manufacturing tolerances, installation posture deviations, or micro-movements at the robotic arm's end effector, easily leading to a sharp drop in matching scores or even complete failure. Furthermore, template matching is extremely sensitive to background interference; common cables, metallic reflections, labels, or other structurally similar objects around the assembly area can all be misidentified as areas to be assembled, causing positioning deviations or misoperations. More critically, this method lacks semantic understanding capabilities, failing to distinguish between component types that appear similar but have vastly different functions, posing a high risk of misassembly in multi-model mixed-line production environments.

[0024] To address these issues, engineering practice often requires building a large template library covering different postures, lighting, and states for the same type of assembly. This not only significantly increases storage overhead and computational complexity for real-time matching, but also makes subsequent maintenance, updates, and expansions extremely cumbersome.

[0025] Furthermore, this application conducted experiments on the aforementioned automated PC I / O interface insertion and removal scheme based on machine vision. The experimental results verified that the performance of the template matching method is highly dependent on the clarity of the features of the parts to be assembled and the ideality of the imaging environment. Only in a few scenarios with clear feature contrast, clean background, and uniform lighting can the positioning success rate reach 92%; however, when faced with common industrial realities such as worn parts, strong surface reflection, severe background interference, or poor lighting conditions, the positioning success rate will rapidly drop to 34% to 46%. When the parts have a regular appearance, no reflection, and good imaging conditions, the success rate is about 84%; however, when the parts have strong reflections or the edge features are blurred due to deep embedding, the success rate is only 38%; in cases of severe background interference or poor lighting conditions, the success rate is even lower, at 42% and 34%, respectively. This indicates that the traditional template matching method is essentially a "fragile precision," performing reasonably well in controlled environments, but lacking robustness in real production lines or field applications, making it difficult to support reliable automated assembly operations.

[0026] In summary, traditional template-matching-based machine vision solutions exhibit increasingly prominent shortcomings when facing complex, dynamic, and ever-changing real-world industrial environments. These shortcomings include weak generalization ability, poor environmental adaptability, low fault tolerance, and high maintenance costs, which have become key bottlenecks restricting the development of automated PCI / O interface plug-in systems towards higher precision and robustness.

[0027] On the other hand, although deep learning-based small-model image segmentation or object detection methods have been proposed to address the shortcomings of traditional matching methods, these methods face a core bottleneck of data scarcity in the aforementioned plug-in durability testing scenario: industrial sites lack large-scale, high-quality assembly image datasets with pixel-level annotations, and manual collection and annotation are extremely costly, especially for degradation states such as wear, scratches, and oxidation after thousands of plug-in cycles, making it almost impossible to systematically construct training samples; in addition, the continuous introduction of new models or new interfaces further exacerbates the burden of model iteration and retraining. Therefore, to solve the above technical problems, this application provides a localization method, apparatus, device, and storage medium.

[0028] Figure 1 A schematic diagram illustrating the implementation flow of the positioning method provided in an embodiment of this application is shown.

[0029] refer to Figure 1 This application provides a positioning method, which can be executed by a visual positioning system, an industrial control computer, or an electronic device integrated with a vision processing module. The method includes: Operation 101: Obtain the image and category of the object to be located.

[0030] The positioning method described in this application is primarily applied to PC I / O plug-in assembly scenarios, specifically for accurately locating I / O interfaces and their mating components during automated plug-in durability testing. Accordingly, the objects to be located include fixed interfaces of different specifications, such as USB-A, USB-C, HDMI, RJ45, and DC power ports, as well as various pluggable devices, such as USB flash drives, Type-C adapters, video capture cards, and charging cable connectors. However, it should be noted that the application scenarios of this application are not limited to these; they can also be applied to various electronic device plug-in and industrial component alignment assembly tasks. The objects to be located can refer to any type of mating components or assembly station, such as connectors, terminals, and mating sockets. For ease of description, this application will subsequently be described using a PC I / O plug-in assembly scenario.

[0031] First, acquire images and categories of the object to be located. Images can be acquired in real time using image acquisition devices such as industrial cameras, smart cameras, or vision sensors, reflecting the actual state of the object within the current work area. Categories characterize the attributes or types of the object in the assembly scenario. For example, based on the motion characteristics or positional variability of the object during the assembly process, the objects can be categorized into those with fixed positions and those with variable positions.

[0032] Operation 102: From multiple segmentation modes, determine the segmentation mode corresponding to the category, input the image into the segmentation model, and use the segmentation model to segment the image according to the segmentation mode corresponding to the category, so as to obtain multiple candidate masks of the object to be located output by the segmentation model.

[0033] Different categories of objects to be located have different location attributes and segmentation requirements. For example, the segmentation accuracy and methods differ between I / O devices within a fixture with fixed locations and I / O interfaces on a PC with variable locations. Therefore, this application analyzes the location characteristics of existing object categories and pre-sets multiple segmentation modes suitable for different segmentation models, enabling the segmentation model to perform targeted image segmentation based on these modes.

[0034] In one embodiment of this application, given the outstanding advantage of the basic visual model in zero-shot generalization ability, the segmentation model of this application preferably adopts the Segment Anything Model, or SAM model. SAM is a basic visual segmentation model with powerful zero-shot generalization ability. It does not require additional training or fine-tuning for specific assembly types and can achieve high-precision pixel-level segmentation for any assembly. However, this application is not limited to this; other models with similar segmentation capabilities can also be applied to this application. The segmentation model, through pre-training on a large-scale dataset, already possesses the ability to extract general visual features and generate fine-grained segmentation masks. Specific training methods can be found in relevant technical literature and will not be elaborated here.

[0035] After obtaining the image and category of the object to be located, the image is input into the segmentation model, and the segmentation model is guided to perform orientation processing on the image based on the segmentation mode corresponding to the category. The segmentation model generates multiple candidate masks for the object to be located, and each candidate mask represents a possible target region.

[0036] Operation 103: Based on a pre-set geometric feature threshold, multiple candidate masks are hierarchically filtered to obtain the target mask of the object to be located.

[0037] For objects to be located, they typically possess relatively regular or predictable physical shape characteristics. Based on this prior knowledge of shape, various geometric feature thresholds can be pre-configured, such as thresholds related to size, shape proportions, and regional compactness. Furthermore, according to the constraints of each geometric feature threshold, multiple candidate masks can be filtered step by step. Invalid or low-quality masks that do not conform to the physical shape characteristics are gradually eliminated through hierarchical constraints. Finally, a unique target mask corresponding to the object to be located is selected from multiple candidate masks.

[0038] Operation 104: Determine the local region corresponding to the target mask in the image of the object to be located, and perform edge enhancement processing on the local region to obtain an edge contour image, which includes the edge contour of the object to be located.

[0039] Although the target mask generated by the segmentation model is sufficient to initially determine the target region, the edges of the target mask may still have jagged edges, blurriness, or subpixel discontinuities due to factors such as the resolution limitations of the original image, metal surface reflection, and weak texture. If subsequent precise geometric measurements are performed directly based on the target mask, significant estimation biases are easily introduced by edge noise, making it difficult to meet the stringent positioning information requirements of high-precision assembly operations.

[0040] Therefore, after obtaining the target mask, this application also performs edge enhancement processing on the local area corresponding to the target mask, so as to extract a clear and complete edge contour of the object to be located by means of improving local image quality, strengthening the gray-scale difference between the foreground and the background, and removing noise interference, and obtain an edge contour image.

[0041] Operation 105: Determine the location information of the object to be located in the image based on the edge contour image.

[0042] In one embodiment of this application, the positioning information includes the center coordinates and rotation angle of the object to be positioned; determining the positioning information of the object to be positioned in the image based on the edge contour image includes: calculating the directed minimum bounding rectangle of the edge contour image; and obtaining the center coordinates and rotation angle of the object to be positioned in the image based on the directed minimum bounding rectangle.

[0043] Specifically, the positioning information includes, but is not limited to, the center coordinates and rotation angle of the object to be positioned in the image's coordinate system. Based on the obtained edge contour image, the positioning information can be extracted by calculating its geometric features. For example, by determining the minimum bounding rectangle of the edge contour and directly solving for the center point coordinates and rotation angle based on the geometric properties of this rectangle, precise spatial pose data of the object to be positioned can be obtained. This positioning information can be directly used as a visual guidance signal, providing a reliable reference for downstream assembly actuators and ensuring the accurate execution of subsequent assembly actions.

[0044] Thus, this embodiment of the application obtains the category of the object to be located and adaptively selects the segmentation mode accordingly. Based on the segmentation model and the corresponding segmentation mode segmentation mask, and then combines a hierarchical filtering mechanism based on geometric priors with a refined edge enhancement post-processing procedure, the positioning information of the object to be located is determined. This achieves generalized and high-precision positioning of the object to be located, effectively solving the problems of weak generalization ability and insufficient positioning accuracy in related technologies.

[0045] In one embodiment of this application, the categories include a first category and a second category; the first category indicates that the position of the object to be located is fixed, and the second category indicates that the position of the object to be located is not fixed; in the above operation 102, the image is segmented using a segmentation model with a segmentation mode corresponding to the category to obtain multiple candidate masks for the object to be located, including: in response to the category being the first category, the image is segmented using a segmentation model with a full-image segmentation mode to obtain multiple candidate masks; in response to the category being the second category, the image is segmented using a segmentation model with a guided segmentation mode to obtain multiple candidate masks.

[0046] The segmentation modes in this application embodiment may include a full-image segmentation mode and a guided segmentation mode. The full-image segmentation mode performs global region segmentation of the entire image without additional guidance or prompting information. The guided segmentation mode relies on external point-based prompting information to selectively segment target regions. Given the usage requirements of these two modes, the segmentation model adopts a dual-branch fusion architecture (e.g., the SAM model) to support on-demand adaptation to different input formats, such as receiving only the image, or simultaneously receiving the image and external prompting information.

[0047] Specifically, for the first category, where the position of the object to be located in the image is relatively fixed (e.g., facing an I / O interface on a PC), the corresponding segmentation mode can be determined as the full-image segmentation mode. The image is directly input into the segmentation model, which performs full-image segmentation and automatically generates multiple candidate masks. For the second category, where the position of the object to be located in the image is somewhat predictable or can be guided by prior information (e.g., facing an I / O device placed in a fixture, such as a USB flash drive or adapter), the corresponding segmentation mode can be determined as the guided segmentation mode. At least one cue point coordinate of the object to be located and the point category (e.g., foreground or background) corresponding to each cue point coordinate are obtained. The image, at least one cue point coordinate, and point category are input into the segmentation model. The segmentation model performs directional segmentation of the image based on the cue point coordinates and point categories, thereby generating multiple candidate masks. Through the constraint of the cue points, the segmentation model can focus on locally relevant regions, effectively reducing the overhead of full-image intensive computation and improving segmentation efficiency.

[0048] In one embodiment of this application, an image is segmented using a segmentation model in a full-image segmentation mode to obtain multiple candidate masks, including: inputting the image into the segmentation model to obtain multiple candidate masks output by the segmentation model; wherein the segmentation model performs full-image segmentation on the image to generate multiple candidate masks.

[0049] Specifically, in the full-image segmentation mode, after receiving the input image, the segmentation model performs traversal feature extraction and region analysis on all regions of the image, automatically identifying and segmenting all potential independent regions in the image, and generating a candidate mask for each independent region. This process does not require manual specification of any region of interest or prompts. Based on its general visual representation capabilities learned during pre-training, the segmentation model autonomously determines the boundaries of each region in the image and generates corresponding segmentation results, ultimately outputting them in the form of multiple candidate masks.

[0050] In one embodiment of this application, an image is segmented using a segmentation model with a guided segmentation mode to obtain multiple candidate masks. This includes: obtaining the coordinates of at least one cue point of the object to be located and the point category corresponding to each cue point coordinate, where the point category includes foreground and background; inputting the image, the coordinates of at least one cue point, and the point category corresponding to each cue point coordinate into the segmentation model to obtain multiple candidate masks output by the segmentation model; wherein the segmentation model generates multiple candidate masks by segmenting the image based on the cue point coordinates and point categories.

[0051] Specifically, in guided segmentation mode, the system first acquires the coordinates of at least one user-specified or preset cue point and determines the point category (i.e., whether the point belongs to the foreground or background region) for each cue point. Then, the original image is combined with the aforementioned cue point coordinates and category information as joint input, which is provided to the segmentation model. Upon receiving this joint input, the segmentation model uses the cue points as spatial constraint references and performs targeted feature analysis and boundary extraction around the regions where the cue points are located. It selects the regions indicated by foreground cue points as segmentation targets and excludes the regions indicated by background cue points, thereby generating multiple candidate masks that match the cue point illustration. This mode, by introducing sparse cue information, effectively narrows the processing range of the segmentation model and significantly improves segmentation efficiency and the correlation between the results.

[0052] The method for obtaining the coordinates of the prompt points can be flexibly determined according to the actual application scenario, and this application does not impose specific limitations on it. For example, based on the pre-calibrated image acquisition pose, several sampling points can be pre-fixed in the image coordinate system as prompt points, and each sampling point corresponds to the possible location area of ​​the object to be located in the assembly scene.

[0053] To further understand the mask segmentation process in the embodiments of this application, the following detailed explanation uses a SAM segmentation model as an example. (Reference) Figure 2 SAM mainly consists of three core modules: Image Encoder, Prompt Encoder, and Mask Decoder.

[0054] (1) Image Encoder The image encoder employs a Vision Transformer (ViT) architecture pre-trained based on a Masked Autoencoder (MAE). It is responsible for converting the input image into a high-dimensional, semantically rich, and spatially structure-preserving feature representation for use by the subsequent cue encoder and mask decoder. The input to the image encoder is an image of the object to be localized, typically an RGB image of arbitrary size, which is resized to 1024×1024 (the default input resolution of SAM) during processing. The output is an embedding representation that preserves image information and possesses richer semantic information. This embedding representation characterizes the structured encoding of image content within the segmentation model, with dimensions as shown in the following formula:

[0055] Where H represents the height of the adjusted image in pixels; W represents the width of the adjusted image in pixels; and C represents the number of feature channels.

[0056] The image is first resized to 1024×1024, and then the pixel values ​​are normalized to the range [0, 1], such as... Figure 3 As shown.

[0057] Furthermore, the adjusted image is divided into 16×16 non-overlapping image blocks, totaling:

[0058] Each image patch is flattened into a 3×16×16=768-dimensional vector, and then mapped to 1280 dimensions through a linear projection layer to form an image patch embedding, such as... Figure 4 and Figure 5 As shown.

[0059] Then, ViT uses a learnable 2D positional encoding, which is added to the image patch embedding. The tensor shape of the image patch embedding is (4096, 1280), and the tensor shape of the 2D positional encoding is also (4096, 1280). They are fused into a single (4096, 1280) tensor using element-wise addition, as shown in the following equation:

[0060]

[0061] Where X represents the image patch embedding tensor and P represents the two-dimensional location encoding tensor.

[0062] Tensor Z is encoded by a Transformer using 32 layers of ViT blocks, outputting a token sequence with dimensions of 4096×1280. This token sequence is then reconstructed back into a 64×64×1280 spatial feature map. A lightweight convolutional projection module reduces the number of channels from 1280 to 256, ultimately outputting a 64×64×256 image embedding feature map. .

[0063] (2) Prompt Encoder The encoder operates differently in the full-image segmentation mode and the guided segmentation mode of this application embodiment, which will be described separately below.

[0064] For guided segmentation mode, the prompt encoder receives at least one prompt point coordinate and the point category corresponding to each prompt point coordinate as input. Specifically, let the input be N prompt point coordinates. and their corresponding categories (i.e., foreground or background), firstly, the coordinates of N prompt points will be... First map to Space, then downsampled to a 64×64 grid:

[0065] Then, a learnable position encoder (usually an MLP or a linear layer combining Fourier features) is used to... The position embedding is encoded as a 256-dimensional vector; further, the corresponding label embedding is selected based on the point category: if it is a foreground point, a learnable foreground embedding vector is added; if it is a background point, the corresponding background embedding vector is added. The final point embedding is represented as:

[0066] in, The final point embedding representation of a single cue point is a 256-dimensional vector. This represents the positional embedding of a 256-dimensional vector; This indicates the embedding of the label corresponding to the prompt point.

[0067] For the full image segmentation mode, the cue encoder does not need to access external cue information. It directly generates built-in global cues based on the spatial grid corresponding to the image embedding feature map output by the image encoder, and uses this as the cue embedding. Specifically, the image embedding feature map can be regarded as a 64×64 resolution grid in spatial dimension, with a total of 4096 grid points. Each grid point corresponds to a local region in the input image. The cue encoder traverses every position point on this 64×64 grid, treating each grid point as a foreground cue. For each grid point... The point encoding logic PromptEncoder is invoked to encode the data, specifically calculating the position embedding corresponding to each grid point and adding it to the label embedding to generate prompt embeddings. The generated prompt embeddings are then compared with the image embedding features. Figure 1 The data is fed into the mask decoder, and candidate masks are generated point by point for the corresponding positions.

[0068] (3) Mask Decoder The mask decoder receives the image embedding feature map output by the image encoder and the prompt embedding output by the prompt encoder: point tokens (each prompt point corresponds to a 256-dimensional vector, obtained by adding the position embedding and the label embedding), box tokens (one box corresponds to two 256-dimensional vectors, which are zero vectors in point mode), and mask tokens (one 256-dimensional vector, which is the actual mask encoding or no-mask embedding). The total prompt tokens, i.e., the set of prompt tokens, can be represented as:

[0069] in, These represent the tokens corresponding to each prompt point; These represent frame tokens; This represents a mask token.

[0070] The mask decoder is a lightweight Transformer decoder that includes the following key modules: a. Learnable "Output Tokens": Define four learnable vectors (each 256-dimensional), one of which is an Intersection over Union (IoU) token used to predict the IoU score; and three are mask tokens used to generate three candidate masks. These tokens are model parameters and are learned during training.

[0071] b. Concatenate the tokens to obtain the concatenated token sequence T: Concatenate the prompt token with the output token:

[0072] in, Represents the set of prompt tokens; These represent three mask tokens, used to generate three candidate masks; This indicates a union of tokens.

[0073] c. Cross-Attention: Query: from the concatenated token sequence T; Key / Value: from image embedding.

[0074] Through a cross-attention mechanism, each cue token or output token dynamically queries relevant regions in the image:

[0075] in, The query matrix is ​​obtained by linear transformation of the concatenated token sequence T. The key matrix is ​​obtained by linear transformation of the image embedding feature map; The value matrix is ​​also obtained by linear transformation of the image embedding feature map; This represents the scaling factor, which takes the value of the dimension of the token vector (256 in this case), and is used to prevent the softmax gradient from vanishing due to an excessively large dot product result. This represents the normalized exponential function.

[0076] This allows the mask token to focus on the target area related to the prompt information, thereby generating a segmented region that matches the object to be located.

[0077] d. MLP and LayerNorm: Cross-attention output is processed through MLP and residual connections to enhance nonlinear expressive power.

[0078] e. Upsampling Head: Each mask token (256-dimensional) is upsampled to a 256×256 candidate mask through two layers of transposed convolution (or MLP combined with reshaping), as shown in the following formula:

[0079] in, It is the sigmoid activation function.

[0080] f. Intersection over Union (IoU) Prediction Header: The IoU token predicts a scalar score using a small MLP, which can be used to evaluate the quality of each candidate mask.

[0081] in, denoted as the predicted intersection-union ratio (IU) score corresponding to the i-th candidate mask, is a scalar value, typically ranging from [0,1]. A higher value indicates a higher expected degree of overlap between the candidate mask and the actual target region. For a small multilayer perceptron used for cross-union ratio prediction, Let be the intersection-union token corresponding to the i-th candidate mask.

[0082] Through the collaborative processing of the above modules, the mask decoder finally outputs three candidate masks and their corresponding cross-union prediction scores.

[0083] In one embodiment of this application, the preset geometric feature thresholds include an area threshold, an aspect ratio threshold, and a fill rate threshold. The above operation 103, which performs hierarchical filtering of multiple candidate masks according to the preset geometric feature thresholds to obtain the target mask of the object to be located, includes: calculating the mask area and aspect ratio of the directed minimum bounding rectangle of each candidate mask; calculating the fill rate of the mask area and the area of ​​the directed minimum bounding rectangle of each candidate mask; filtering the multiple candidate masks step by step according to the area threshold, aspect ratio threshold, and fill rate threshold, and determining the candidate mask with the largest area among the filtered candidate masks as the target mask.

[0084] Based on the SAM segmentation results, three types of discriminative geometric features can be defined for each candidate mask: (1) mask area; (2) aspect ratio of the directed minimum bounding rectangle; and (3) the ratio of the mask area to the area of ​​the bounding rectangle (fill rate). By setting reasonable threshold ranges for these three features, a lightweight rule classifier can be constructed to efficiently and accurately filter out the target mask corresponding to the object to be located.

[0085] Specifically, let Sam automatically generate the first The candidate mask is The corresponding attributes are as follows: : Mask pixel area; : The directed minimum bounding rectangle (Rotated Bounding Box), where, With the center coordinates, Width and height ( ), The rotation angle; The aspect ratio is used. Area of ​​the circumscribed rectangle; Area ratio (i.e., fill ratio).

[0086] The threshold filtering rules are as follows: Level 1: Mask Area Filtering Retain candidate targets that meet the area range:

[0087] Level 2: Aspect Ratio Filtering exist Based on this, further filter objects with reasonable aspect ratios:

[0088] Level 3: Fill Rate Filtering exist Based on this, retain a compact, non-fragmented mask:

[0089] in, The area threshold is a preset upper and lower limit for the area based on the physical size of the object to be located, such as the I / O interface. The aspect ratio range set according to the interface geometry of the object to be located (e.g., USB-A is a long and narrow rectangle), i.e., the aspect ratio threshold; The fill rate threshold is used to remove edge-broken or noise masks.

[0090] After the above three-level filtering, if multiple target masks still exist, the one with the largest area (usually the main interface) is selected as the target mask. The target mask and its corresponding local region in the image can be found in [reference needed]. Figure 6 , Figure 6 (a) shows the white rectangle as the target mask, and (b) shows the target mask corresponding to a local region in the image.

[0091] Thus, through the three-level filtering based on area threshold, aspect ratio threshold, and fill rate threshold, a target mask matching the geometric characteristics of the object to be located can be automatically selected from the large number of candidate masks output by the segmentation model. This hierarchical filtering mechanism effectively eliminates abnormal segmentation regions caused by image noise, background interference, reflection, or occlusion, ensuring the accuracy and reliability of the target mask and laying a high-quality input foundation for subsequent edge enhancement processing and localization information extraction based on the target mask.

[0092] Figure 7 A schematic diagram illustrating the implementation flow of the edge enhancement operation of the positioning method provided in this application embodiment is shown.

[0093] refer to Figure 7 In one embodiment of this application, the above operation 104, which performs edge enhancement processing on the local region corresponding to the target mask in the image to obtain an edge contour image, includes: Operation 201 performs super-resolution reconstruction on the local region to obtain a resolution-enhanced image.

[0094] Super-resolution reconstruction is an image processing method that reconstructs high-resolution details from low-resolution images. Its core objective is to recover high-frequency information lost due to physical limitations of the imaging system (such as sensor resolution, optical blur, compression distortion, etc.), thereby generating high-resolution images with clearer textures and sharper edges. This technique is not a simple interpolation upscaling, but rather reconstructs the geometry at the sub-pixel level by learning the prior distribution of natural images or using deep neural networks to model the nonlocal correlations between pixels.

[0095] By performing super-resolution reconstruction on the local area corresponding to the target mask, the detail representation of the target area can be effectively improved, providing richer geometric information for subsequent edge extraction.

[0096] Operation 202 performs adaptive contrast enhancement processing on the resolution-enhanced image.

[0097] After obtaining the resolution-enhanced image, adaptive contrast enhancement processing is performed. This processing aims to strengthen the grayscale difference between the edge region of the object to be located and its surrounding background. Especially for metallic surfaces that are prone to reflection and have low contrast, adaptively adjusting local contrast effectively improves the recognizability of edge features and suppresses visual interference caused by uneven lighting or reflection. The effect after adaptive contrast enhancement processing is shown below. Figure 8 As shown in Figure a.

[0098] Operation 203 performs median filtering to denoise the contrast-enhanced image.

[0099] Contrast enhancement may amplify random noise in an image to some extent. Therefore, median filtering is applied after enhancement. Median filtering is a non-linear filtering method that replaces the gray value of each pixel in the image with the median of the gray values ​​of all pixels within its neighborhood window. This smooths the image, removes random noise, and preserves edge structure relatively well, avoiding false contours or edge breaks in subsequent processing. The effect of median filtering is shown below. Figure 8 As shown in b.

[0100] Operation 204 performs local binarization on the median-filtered image within the foreground region defined by the target mask to obtain a binarized image.

[0101] Since the target mask precisely defines the area containing the object to be located, local binarization can be performed on the median-filtered image within this foreground constraint range. By employing an adaptive thresholding method, the binarization threshold is dynamically determined based on the grayscale distribution characteristics of the local region, transforming the image into a binary image containing only the foreground and background. Because the background region has been effectively removed by the target mask, the binarization process can be performed in a relatively clean local sub-image, thus stably segmenting the boundary region of the object to be located and significantly improving the continuity and clarity of the contour. The effect of adaptive threshold binarization is shown below. Figure 8 As shown in c.

[0102] Operation 205 performs a morphological opening operation on the binarized image, and the difference between the morphological opening operation result and the binarized image is used to obtain the edge image.

[0103] After obtaining the binarized image, a morphological opening operation is first performed on it. Morphological opening is a combination of erosion and dilation operations, which effectively removes isolated noise points, small spikes, and disconnected micro-fragments from the binary image while preserving the overall shape of the main region. Subsequently, a difference operation is performed between the opened image and the original binarized image; that is, the opening result is subtracted from the original binarized image. This extracts the edge details and local protrusions removed during the opening process, resulting in a preliminary edge image. The effect of the morphological opening operation is shown below. Figure 8 As shown in d, the effect of subtracting the original binary image from the opening operation is as follows: Figure 8 As shown in e.

[0104] Operation 206: Perform connected component analysis on the edge image to obtain at least one connected component, and remove connected components with an area smaller than a preset threshold to obtain the edge contour image.

[0105] Connected component analysis is performed on the above edge image to further remove residual invalid pixel blocks and retain valid edge regions with complete structure and clear contours. The specific implementation of connected component area filtering is as follows: Let the input edge image be... The foreground pixel value is 255, and the background pixel value is 0.

[0106] Through 8-neighborhood connected component analysis, it can be found that... Decomposed into A series of non-overlapping connected regions ,in For the background.

[0107] For each foreground connected component Its pixel area is defined as:

[0108] in, Represents connected components Pixel coordinates within; This represents the area attribute value in the statistics of the i-th connected component returned by the connected component analysis function.

[0109] Given a minimum area threshold Define the filtered output mask. for:

[0110] in, This indicates the area of ​​some foreground connected region i. Greater than the preset threshold And pixels It belongs to the connected component.

[0111] Equivalently, this can be represented as a set operation:

[0112] Where U represents the union operation, i.e., the output mask. It consists of the union of all foreground connected regions that satisfy the above area conditions.

[0113] By using the connected component area filtering process described above, isolated pixel blocks with areas smaller than a preset threshold can be effectively removed from the edge image, retaining only structurally complete and contour-continuous effective connected components. The final result is an edge contour image containing the clear edge structure of the object to be located. The effect of connected component analysis after pixel block removal is shown below. Figure 8 As shown in f.

[0114] In one embodiment of this application, after obtaining the image in operation 101, the image is further pre-cropped in terms of the region of interest to obtain a cropped image; correspondingly, in the above operation 102, the image is segmented using a segmentation model in a segmentation mode corresponding to the category, including: segmenting the cropped image using a segmentation model in a segmentation mode corresponding to the category.

[0115] In practical industrial applications, the raw images acquired often have high resolution and contain a large number of background areas unrelated to the localization task (such as work surfaces, cables, fixtures, or other equipment structures). Directly inputting such raw images into a segmentation model for full-image processing results in a massive number of redundant candidate masks, many of which correspond to irrelevant objects or noise in the background. This not only significantly increases the computational burden of subsequent candidate mask selection and prolongs the overall processing time but may also introduce false detections, reducing the accuracy of target mask recognition.

[0116] Therefore, based on the actual image acquisition environment and the typical placement area of ​​the object to be located, the original image can be pre-cropped with regions of interest. Specifically, based on the fixed pose of the image acquisition device and the usual location of the object to be located in the image, one or more regions of interest can be pre-marked, retaining only the effective areas of the image containing the potential object to be located, and cropping away the rest of the irrelevant background. Figure 9 As shown, this strategy effectively reduces the size range of the input image and significantly reduces interference from irrelevant background information, thereby significantly improving the processing efficiency of the segmentation model and the reliability of the segmentation results while ensuring the integrity of the segmentation.

[0117] In one embodiment of this application, in operation 104 above, determining the local region in the image corresponding to the target mask from the image of the object to be located is achieved through the following operations: Specifically, let the coordinates of the centroid of the target mask in the local coordinate system of the mask be... (Taking the top-left corner of the target mask bounding box as the origin), the coordinates of the top-left corner of the bounding box corresponding to this target mask in the cropped image are: The top-left corner coordinates of the cropped region in the original image are: If the target mask is located in the original image, the corresponding region can be located using the following coordinate mapping relationship:

[0118] in, Let be the horizontal coordinate of the centroid of the target mask in the cropped image coordinate system. Let be the vertical coordinates of the centroid of the target mask in the cropped image coordinate system. Let $\mathbf{ ... The global vertical coordinates of the target mask centroid in the original image coordinate system.

[0119] The centroid coordinates of the target mask can be calculated from the average of the pixel coordinates within the mask region:

[0120] in, A represents the binary matrix of the target mask, where A = |M| represents the area of ​​the target mask (i.e., the total number of non-zero pixels). These are the pixel coordinates within the target mask region. Through this mapping, the local region corresponding to the target mask in the original image can be accurately obtained, and this local region can then be used as input for subsequent edge enhancement processing.

[0121] Figure 10 A schematic diagram of the composition structure of the positioning device provided in the embodiments of this application is shown.

[0122] refer to Figure 10 This application also provides a positioning device, which includes: The acquisition module 301 is used to acquire the image and category of the object to be located; The segmentation module 302 is used to determine the segmentation mode corresponding to the category from multiple segmentation modes, input the image into the segmentation model, and use the segmentation model to segment the image with the segmentation mode corresponding to the category to obtain multiple candidate masks of the object to be located output by the segmentation model. The filtering module 303 is used to perform hierarchical filtering on multiple candidate masks according to a pre-set geometric feature threshold to obtain the target mask of the object to be located. The edge enhancement module 304 is used to determine the local region corresponding to the target mask in the image of the object to be located, and to perform edge enhancement processing on the local region to obtain an edge contour image, which includes the edge contour of the object to be located. The extraction module 305 is used to determine the positioning information of the object to be located in the image based on the edge contour image.

[0123] It should be noted that the description of the device in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, therefore it will not be repeated. For any technical details not covered in the positioning device provided in this application embodiment, please refer to... Figures 1 to 9 The meaning is understood in accordance with the description of any of the accompanying drawings.

[0124] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0125] Figure 11 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0126] like Figure 11As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0127] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the positioning method. For example, in some embodiments, the positioning method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the positioning method by any other suitable means (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0133] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0134] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0135] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0136] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A positioning method, characterized in that, The method includes: Obtain the image and category of the object to be located; From multiple segmentation modes, a segmentation mode corresponding to the category is determined, and the image is input into a segmentation model. The segmentation model is used to segment the image using the segmentation mode corresponding to the category to obtain multiple candidate masks of the object to be located output by the segmentation model. The multiple candidate masks are hierarchically filtered according to a pre-set geometric feature threshold to obtain the target mask of the object to be located. From the image of the object to be located, determine the local region corresponding to the target mask in the image, and perform edge enhancement processing on the local region to obtain an edge contour image, wherein the edge contour image includes the edge contour of the object to be located; The positioning information of the object to be located in the image is determined based on the edge contour image.

2. The method according to claim 1, characterized in that, The categories include a first category and a second category; the first category indicates that the position of the object to be located is fixed, and the second category indicates that the position of the object to be located is not fixed. The process of segmenting the image using a segmentation model with a segmentation pattern corresponding to the category yields multiple candidate masks for the object to be located, output by the segmentation model, including: In response to the category being the first category, the image is segmented using the segmentation model in a full-image segmentation mode to obtain multiple candidate masks output by the segmentation model; In response to the category being the second category, the image is segmented using the segmentation model in a guided segmentation mode to obtain multiple candidate masks output by the segmentation model.

3. The method according to claim 2, characterized in that, The segmentation model is used to segment the image in a full-image segmentation mode to obtain multiple candidate masks output by the segmentation model, including: The image is input into the segmentation model to obtain multiple candidate masks output by the segmentation model; wherein the segmentation model performs full-image segmentation on the image to generate the multiple candidate masks.

4. The method according to claim 2, characterized in that, The segmentation model is used to segment the image using a guided segmentation mode, resulting in multiple candidate masks output by the segmentation model, including: Obtain the coordinates of at least one prompt point of the object to be located and the point category corresponding to each prompt point coordinate, wherein the point category includes foreground and background; The image, the coordinates of the at least one cue point, and the point category corresponding to each cue point coordinate are input into the segmentation model to obtain multiple candidate masks output by the segmentation model; wherein, the segmentation model generates the multiple candidate masks by segmenting the image based on the cue point coordinates and point categories.

5. The method according to claim 1, characterized in that, The preset geometric feature thresholds include area threshold, aspect ratio threshold, and fill rate threshold; The step of hierarchically filtering the multiple candidate masks according to a pre-set geometric feature threshold to obtain the target mask of the object to be located includes: Calculate the mask area and aspect ratio of the directed minimum bounding rectangle for each candidate mask; Calculate the fill rate of the area of ​​each candidate mask and the area of ​​the directed minimum bounding rectangle; The candidate masks are filtered step by step according to the area threshold, the aspect ratio threshold, and the fill rate threshold, and the candidate mask with the largest area among the filtered candidate masks is determined as the target mask.

6. The method according to claim 1, characterized in that, The process of performing edge enhancement processing on the local region to obtain an edge contour image includes: Super-resolution reconstruction is performed on the local region to obtain a resolution-enhanced image; The resolution-enhanced image is subjected to adaptive contrast enhancement processing; Median filtering is applied to the contrast-enhanced image for noise reduction. The median-filtered image is locally binarized within the foreground region defined by the target mask to obtain a binarized image; A morphological opening operation is performed on the binarized image, and the difference between the morphological opening operation result and the binarized image is obtained to obtain the edge image; The edge image is subjected to connected component analysis to obtain at least one connected component, and connected components with an area smaller than a preset threshold are removed to obtain the edge contour image.

7. The method according to claim 1, characterized in that, The positioning information includes the center coordinates and rotation angle of the object to be positioned. Determining the positioning information of the object to be located in the image based on the edge contour image includes: Calculate the directed minimum bounding rectangle of the edge contour image; Based on the directed minimum bounding rectangle, obtain the center coordinates and rotation angle of the object to be located in the image.

8. The method according to claim 1, characterized in that, Before segmenting the image using a segmentation model with a segmentation pattern corresponding to the category, the method further includes: The image is pre-cropped by the region of interest to obtain a cropped image; The step of segmenting the image using a segmentation model with a segmentation pattern corresponding to the category includes: The cropped image is segmented using a segmentation model with a segmentation pattern corresponding to the category.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

Images