Positioning method and device and electronic equipment

Abstract

The invention discloses a positioning method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the following steps: carrying out key point extraction processing on a target object in a first image to obtain a first pixel coordinate of each key point in the first image and affine transformation information of the target object in the first image relative to a target object in a reference image at a pixel level; determining and extracting a second image from the first image according to each first pixel coordinate; determining first sub-pixel edge information in the first image according to the second image, the pixel-level affine transformation information and the reference image; according to the first sub-pixel edge information and the reference sub-pixel edge information, a positioning result of the target object in the first image is obtained, and the positioning result comprises radiation transformation information at the sub-pixel level compared with the target object in the reference image, so that the influence of factors such as image materials, contrast and noise on positioning can be effectively reduced, and the positioning accuracy is improved. And the robustness and the accuracy of sub-image level positioning are improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a positioning method, device and electronic equipment. Background Art

[0002] In tasks such as alignment, bonding, and inspection in modern intelligent production lines, accurate positioning of target objects is very important. With the development of computer vision and pattern recognition technology, traditional pixel-level positioning can no longer meet the accuracy requirements in practical applications, while sub-pixel positioning can achieve higher accuracy.

[0003] Related technologies use moment-based methods to directly perform sub-pixel positioning. However, factors such as image material (color, texture, reflectivity, transparency), contrast, and noise have a great impact on imaging. Moment-based methods cannot adapt to changes in these factors and are prone to problems such as poor edge detection robustness and low positioning accuracy. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a positioning method, device and electronic device, which can effectively reduce the influence of factors such as image material, contrast and noise on the positioning target object, and improve the robustness (or anti-interference ability) and accuracy of sub-pixel level positioning.

[0005] In a first aspect, the present application provides a positioning method, the method comprising:

[0006] Performing key point extraction processing on the target object in the first image to obtain first pixel coordinates of each key point of the target object in the first image and pixel-level affine transformation information of the target object in the first image compared with the target object in the reference image;

[0007] Determine and extract a second image from the first image according to each first pixel coordinate;

[0008] First sub-pixel edge information of the target object in the first image is determined according to the second image, the pixel-level affine transformation information, and the reference image.

[0009] According to the positioning method of the present application, by performing key point extraction processing on the target object in the first image, the first pixel coordinates of each key point of the target object in the first image and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level are obtained, so as to roughly detect the edge information of the target object, effectively reduce the influence of factors such as image material (color, texture, reflectivity, transparency), contrast and noise on the positioning target object, and improve the anti-interference ability of positioning.

[0010] In addition, according to each first pixel coordinate, the second image is determined and extracted from the first image; according to the second image, the affine transformation information and the reference image, the first sub-pixel edge information of the target object in the first image is determined; according to the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, the positioning result of the target object in the first image is obtained, and the positioning result includes the radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level. Compared with performing sub-pixel edge detection and sub-pixel positioning in the entire first image, performing sub-pixel detection in the second image reduces the range of high-precision sub-pixel edge detection, which can effectively reduce the delay of high-precision edge detection. In addition, compared with performing sub-pixel edge detection in the second pixel area, performing sub-pixel in the second image appropriately expands the range of sub-pixel edge detection, which can better capture the edge information in the image and improve the sub-pixel positioning accuracy and robustness.

[0011] In addition, from a hardware perspective, regardless of whether the controller or processor of the device has a GPU, the above-mentioned sub-pixel edge detection method can be executed, and it can also be run in a CPU without a GPU, which can effectively reduce costs.

[0012] According to an embodiment of the present application, determining first sub-pixel edge information of a target object in a first image according to a second image, pixel-level affine transformation information, and a reference image includes:

[0013] Performing sub-pixel edge extraction processing on the second image according to the pixel-level affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image;

[0014] The second sub-pixel edge information is mapped to the first image to obtain first sub-pixel edge information of the target object in the first image.

[0015] According to an embodiment of the present application, sub-pixel edge extraction processing is performed on the second image according to pixel-level affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image, including:

[0016] Calling a first operator of a trained sub-pixel edge model to perform interpolation processing and sub-pixel gradient detection processing on the second image to obtain a first gradient map;

[0017] Calling a second operator of the sub-pixel edge model to perform sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-image, and pixel-level affine transformation information, to obtain second sub-pixel edge information of the target object in the second image;

[0018] The reference sub-image is a sub-image extracted from the reference image based on sub-pixel edge information of the target object in the reference image; the reference gradient image is a gradient image obtained after interpolation processing and gradient detection processing are performed on the reference image.

[0019] According to an embodiment of the present application, the size of the first gradient map is larger than the size of the reference gradient map; calling the second operator of the sub-pixel edge model to perform sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-map, and pixel-level affine transformation information, includes: performing the following operations by the second operator:

[0020] Transforming the first gradient map according to pixel-level affine transformation information to obtain a second gradient map;

[0021] Normalizing the second gradient map and the reference gradient map;

[0022] A sliding window is set in the second gradient map to traverse the second gradient map, and the sliding window and the reference gradient map have the same size; wherein, for the current sliding window currently traversed, the current sliding window and the reference gradient map are gradient matched to obtain a matching degree between the sliding window and the reference gradient map;

[0023] Determine a first sub-pixel region corresponding to the sliding window of maximum matching degree in the interpolated second image;

[0024] Sub-pixel boundary information of the first pixel region is determined from the interpolated second image; the sub-pixel boundary information of the first pixel region is second sub-pixel edge information of the target object.

[0025] According to one embodiment of the present application, mapping the second sub-pixel edge information to the first image to obtain the first sub-pixel edge information of the target object in the first image includes:

[0026] Performing interpolation processing on the first image to obtain an interpolated first image;

[0027] The second sub-pixel edge information is mapped to the interpolated first image to obtain first sub-pixel edge information of the target object in the first image.

[0028] According to one embodiment of the present application, performing key point extraction processing on a target object in a first image to obtain first pixel coordinates of each key point of the target object in the first image and pixel-level affine transformation information of the target object in the first image compared with the target object in the reference image includes:

[0029] Calling a trained target detection model to detect a target object in the first image, and obtaining a labeled image output by the target detection model after annotating a real bounding box for the target object in the first image;

[0030] Calling the first sub-model of the trained key point detection model to perform key point extraction processing on the target object in the area corresponding to the real bounding box of the annotated image, and obtaining the first pixel coordinates of each key point of the target object output by the first sub-model corresponding to the first image; the key point detection model includes the reference pixel coordinates corresponding to each key point of the target object in the reference image;

[0031] The second submodel of the key point detection model is called to obtain the pixel-level affine transformation information of the target object in the first image compared to the target object in the reference image output by the second submodel based on the first pixel coordinates in the first image and the reference pixel coordinates of the target object in the reference image.

[0032] According to an embodiment of the present application, determining and extracting a second image from a first image according to each first pixel coordinate includes:

[0033] Determine in the first image a second pixel region with the first pixel coordinates as boundaries;

[0034] In the first image, pixel expansion is performed on the second pixel region along the target direction to obtain a second image.

[0035] In a second aspect, the present application provides a positioning device, the device comprising:

[0036] A first processing module is used to perform key point extraction processing on the target object in the first image to obtain first pixel coordinates of each key point of the target object in the first image and affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level;

[0037] A second processing module, used for determining and extracting a second image from the first image according to each first pixel coordinate;

[0038] A third processing module, configured to determine first sub-pixel edge information of the target object in the first image according to the second image, the pixel-level affine transformation information and the reference image;

[0039] The fourth processing module is used to obtain a positioning result of the target object in the first image based on the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, wherein the positioning result includes the radiation transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level.

[0040] According to the positioning device of the present application, by performing key point extraction processing on the target object in the first image, the first pixel coordinates of each key point of the target object in the first image and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level are obtained, thereby achieving rough detection of the edge information of the target object, and can effectively reduce the influence of factors such as image material (color, texture, reflectivity, transparency), contrast and noise on the positioning of the target object, thereby improving the anti-interference ability of positioning.

[0041] In addition, according to each first pixel coordinate, the second image is determined and extracted from the first image; according to the second image, the affine transformation information and the reference image, the first sub-pixel edge information of the target object in the first image is determined; according to the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, the positioning result of the target object in the first image is obtained, and the positioning result includes the radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level. Compared with performing sub-pixel edge detection and sub-pixel positioning in the entire first image, performing sub-pixel detection in the second image reduces the range of high-precision sub-pixel edge detection, which can effectively reduce the delay of high-precision edge detection. In addition, compared with performing sub-pixel edge detection in the second pixel area, performing sub-pixel in the second image appropriately expands the range of sub-pixel edge detection, which can better capture the edge information in the image and improve the sub-pixel positioning accuracy and robustness.

[0042] In addition, from a hardware perspective, regardless of whether the controller or processor of the device has a GPU, the above-mentioned sub-pixel edge detection method can be executed, and it can also be run in a CPU without a GPU, which can effectively reduce costs.

[0043] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the positioning method provided in the first aspect is implemented.

[0044] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the positioning method provided in the first aspect described above is implemented.

[0045] In a fifth aspect, the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a program or instruction to implement the positioning method provided in the first aspect.

[0046] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the positioning method provided in the first aspect above.

[0047] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0048] By performing key point extraction processing on the target object in the first image, the first pixel coordinates of each key point of the target object in the first image are obtained, and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level is obtained, so as to roughly detect the edge information of the target object, effectively reduce the influence of factors such as image material (color, texture, reflectivity, transparency), contrast and noise on the positioning of the target object, and improve the anti-interference ability of positioning.

[0049] In addition, according to each first pixel coordinate, the second image is determined and extracted from the first image; according to the second image, the affine transformation information and the reference image, the first sub-pixel edge information of the target object in the first image is determined; according to the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, the positioning result of the target object in the first image is obtained, and the positioning result includes the radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level. Compared with performing sub-pixel edge detection and sub-pixel positioning in the entire first image, performing sub-pixel detection in the second image reduces the range of high-precision sub-pixel edge detection, which can effectively reduce the delay of high-precision edge detection. In addition, compared with performing sub-pixel edge detection in the second pixel area, performing sub-pixel in the second image appropriately expands the range of sub-pixel edge detection, which can better capture the edge information in the image and improve the sub-pixel positioning accuracy and robustness.

[0050] In addition, from a hardware perspective, regardless of whether the controller or processor of the device has a GPU, the above-mentioned sub-pixel edge detection method can be executed, and it can also be run in a CPU without a GPU, which can effectively reduce costs.

[0051] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0053] Figure 1 is a schematic diagram of imaging images under different image materials provided in an embodiment of the present application;

[0054] Figure 2 This is one of the flowcharts of the positioning method provided in the embodiment of the present application;

[0055] Figure 3 is a schematic diagram of key points marked on a target object in two second sample images provided by an embodiment of the present application;

[0056] Figure 4 This is the second flow chart of the positioning method provided in the embodiment of the present application;

[0057] Figure 5 is a schematic diagram of the structure of a positioning device provided in an embodiment of the present application;

[0058] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0060] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0061] The positioning tool is mainly used to detect the edge information of the target object in the image, which is used for subsequent tasks such as alignment, fitting, and detection. In 2D edge detection, traditional positioning encounters bottlenecks. Factors such as image material, contrast, and noise lead to insufficient stability. At the same time, the traditional algorithm has a high threshold for parameter adjustment, such as Figure 1 As shown, an embodiment of the present application provides a schematic diagram of imaging images under different image materials. The target objects in the circles of Image 1 and Image 2 are the target objects to be positioned. It can be seen that the imaging effects of the two target objects are completely different due to the difference in materials, which increases the difficulty of positioning.

[0062] In addition, positioning tools are usually run in closed factory workshops and cannot be connected to the Internet, so each positioning tool needs to run on a small industrial computer. Due to high cost requirements, industrial computers cannot be equipped with graphics processing units (GPUs). In addition, industrial scenarios also have very high requirements for production efficiency, and the time tolerance (delay) is within 100ms. However, some positioning tools will increase delays or detection errors due to changes in the aforementioned factors. These problems are likely to cause product scrapping.

[0063] To solve at least one of the above problems, embodiments of the present application provide a positioning method, a positioning device, an electronic device, and a readable storage medium.

[0064] The positioning method, positioning device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0065] The positioning method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0066] The positioning method provided in the embodiments of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the positioning method. The electronic devices mentioned in the embodiments of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The positioning method provided in the embodiments of the present application is described below using an electronic device as an example of an execution subject.

[0067] like Figure 2 As shown, the positioning method includes: step 210, step 220, step 230 and step 240.

[0068] Step 210: perform key point extraction processing on the target object in the first image to obtain the first pixel coordinates of each key point of the target object in the first image, and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level.

[0069] In the embodiment of the present application, the first image and the reference image are both images containing a target object, wherein the reference image refers to a standard image or template image used for comparison or registration, and the reference image can be regarded as an image without rotation angle, scaling, and translation. The first image is the image to be processed, and the target object in the first image may be at least one of rotation, scaling, and translation compared to the target object in the reference image.

[0070] The target object of the embodiment of the present application can be any object, such as a person or an object, and can be set according to the business field or business scenario. For example, in image processing and computer vision, the target object can be an industrial component.

[0071] The target object of the embodiment of the present application includes multiple key points. Key points refer to the feature points used to describe and identify objects or scenes in images in the field of computer vision and image processing. These feature points are usually points with high contrast, texture changes or specific shapes in the image (usually corners or mutations), which can help algorithms identify, match and track targets in different images. For the human body, key points are palms, joints, heads, etc.

[0072] The number of key points of the target object in the embodiment of the present application is a first target number, and the first target number can be set according to actual conditions, such as 10.

[0073] After obtaining the first image, the embodiment of the present application can first perform key point extraction processing on the target object in the first image to determine the key points of the target object in the first image, and obtain the first pixel coordinates of the key points of the target object in the first image, and the affine transformation information of the target object in the first image compared to the target object in the reference image at the sub-pixel level.

[0074] Affine Transformation is a commonly used geometric transformation in image processing, which can scale, rotate, translate and shear images. Affine transformation maintains the relative position relationship between points, so that straight lines remain straight lines and parallel lines remain parallel. Affine changes can be considered from the overall perspective of the target object.

[0075] Pixel-level affine transformation information uses integer coordinates to transform images, which is computationally simple and efficient. However, since sub-pixel accuracy is not considered, it may cause distortion and aliasing after image transformation.

[0076] The affine transformation information of the target object in the first image at the pixel level compared to the target object in the reference image may be at least one of scaling information, rotation information, translation information and shearing information of the target object in the first image at the pixel level compared to the target object in the reference image.

[0077] Specifically, the real bounding box of the target object in the first image may be determined first, and then key point extraction processing may be performed on the target object in the area corresponding to the real bounding box to obtain first pixel coordinates corresponding to key points of the target object in the first image.

[0078] The Ground Truth Bounding Box refers to a rectangular box used to represent the actual position and size of the target in the image.

[0079] Furthermore, according to the first pixel coordinates in the first image and the reference pixel coordinates in the reference image, the affine transformation information at the pixel level of the target object in the first image output by the second sub-model compared to the target object in the reference image can be obtained.

[0080] Step 220: Determine and extract the second image from the first image according to the first pixel coordinates.

[0081] The first pixel coordinate output by key point positioning is a relatively accurate coordinate, but the positioning accuracy often exceeds 10 pixels, while the accuracy of sub-pixel edge positioning is required to reach an accuracy within 1 pixel. Therefore, the first pixel coordinate is usually not used as an edge detection result, but sub-pixel positioning can be performed based on the first pixel coordinate.

[0082] In some embodiments, determining and extracting the second image from the first image according to each first pixel coordinate includes:

[0083] Determine in the first image a second pixel region with the first pixel coordinates as boundaries;

[0084] In the first image, pixel expansion is performed on the second pixel region along the target direction to obtain a second image.

[0085] Specifically, a second pixel area with each first pixel coordinate as a boundary can be determined in the first image. In the first image, the second pixel area is expanded along the target direction to obtain a second image in which the number of pixel points is greater than the number of pixel points contained in the second pixel area. The target direction can be above, below, left, and right, etc., and a target number of pixel rows or pixel columns can be expanded at each boundary of the target area. The target number is a preset threshold, such as 5.

[0086] Step 230: Determine first sub-pixel edge information of the target object in the first image according to the second image, the pixel-level affine transformation information, and the reference image.

[0087] The sub-pixel edge information of the embodiment of the present application may be the sub-pixel coordinates and / or the sub-pixel contour map corresponding to the edge points of the target object, and there is no limitation on this. The sub-pixel edge information of the target object in the first image is referred to as the first sub-pixel edge information.

[0088] After obtaining the second image, the embodiment of the present application can perform sub-pixel edge extraction processing on the second image according to the affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image, and then map the second sub-pixel edge information to the first image to obtain first sub-pixel edge information of the target object in the first image.

[0089] Step 240, obtaining a positioning result of the target object in the first image based on the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, wherein the positioning result includes radiation transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level.

[0090] The embodiment of the present application can predetermine the baseline sub-pixel edge information of the target object in the baseline image. After obtaining the first sub-pixel edge information of the target object in the first image, the positioning result of the target object in the first image can be obtained based on the first sub-pixel edge information and the baseline sub-pixel edge information of the target object in the baseline image. The positioning result includes the radial transformation information of the target object in the first image compared with the target object in the baseline image at the sub-pixel level. The accuracy of the radial transformation information at the sub-pixel level is within 1 pixel, which belongs to precise positioning.

[0091] The embodiment of the present application performs key point extraction processing on the target object in the first image to obtain the first pixel coordinates of each key point of the target object in the first image, and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level, thereby roughly detecting the edge information of the target object, and can effectively reduce the influence of factors such as image material (color, texture, reflectivity, transparency), contrast and noise on the positioning of the target object, thereby improving the anti-interference capability of positioning.

[0092] In addition, according to each first pixel coordinate, the second image is determined and extracted from the first image; according to the second image, the affine transformation information and the reference image, the first sub-pixel edge information of the target object in the first image is determined; according to the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, the positioning result of the target object in the first image is obtained, and the positioning result includes the radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level. Compared with performing sub-pixel edge detection and sub-pixel positioning in the entire first image, performing sub-pixel detection in the second image reduces the range of high-precision sub-pixel edge detection, which can effectively reduce the delay of high-precision edge detection. In addition, compared with performing sub-pixel edge detection in the second pixel area, performing sub-pixel in the second image appropriately expands the range of sub-pixel edge detection, which can better capture the edge information in the image and improve the sub-pixel positioning accuracy and robustness.

[0093] In addition, from a hardware perspective, regardless of whether the controller or processor of the device has a GPU, the above-mentioned sub-pixel edge detection method can be executed, and it can also be run in a CPU without a GPU, which can effectively reduce costs.

[0094] In some embodiments, determining first sub-pixel edge information of a target object in a first image according to the second image, pixel-level affine transformation information, and a reference image includes:

[0095] Performing sub-pixel edge extraction processing on the second image according to the pixel-level affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image;

[0096] The second sub-pixel edge information is mapped to the first image to obtain first sub-pixel edge information of the target object in the first image.

[0097] In the embodiment of the present application, the reference image is a template image, and the sub-pixel edge information of the target object in the reference image is determined. In order to improve the positioning accuracy, the second image can be subjected to sub-pixel edge extraction processing according to the pixel-level affine transformation information to obtain the second sub-pixel edge information of the target object in the second image.

[0098] Further, after obtaining the second sub-pixel edge information of the target object in the second image, the second sub-pixel edge information may be mapped to the first image to obtain the first sub-pixel edge information of the target object in the first image.

[0099] In some embodiments, performing key point extraction processing on the target object in the first image to obtain first pixel coordinates of each key point of the target object in the first image and pixel-level affine transformation information of the target object in the first image compared with the target object in the reference image includes:

[0100] Calling a trained target detection model to detect a target object in the first image, and obtaining a labeled image output by the target detection model after annotating a real bounding box for the target object in the first image;

[0101] Calling the first sub-model of the trained key point detection model to perform key point extraction processing on the target object in the area corresponding to the real bounding box of the annotated image, and obtaining the first pixel coordinates of each key point of the target object output by the first sub-model corresponding to the first image; the key point detection model includes the reference pixel coordinates corresponding to each key point of the target object in the reference image;

[0102] The second submodel of the key point detection model is called to obtain the pixel-level affine transformation information of the target object in the first image output by the second submodel compared with the target object in the reference image based on the first pixel coordinates in the first image and the reference pixel coordinates in the reference image.

[0103] Before performing sub-pixel edge detection, the embodiment of the present application trained a target detection model and a key point detection model. The target detection model is used to detect the real bounding box of the target object in the image, and the key point detection model is used to detect the pixel coordinates of the key points of the target object in the image.

[0104] The training process of the target detection model is as follows:

[0105] Obtain a first training sample set, input a first sample image in the first training sample set into an initial target detection model for at least one training, until a training stop condition of the target detection model is met (the first loss function converges or the number of training times reaches a target number of times), and obtain a trained target detection model; each first sample image contains a target object; and the training label of each first sample image is a real bounding box corresponding to the target object;

[0106] Among them, the following operations are performed during each training process of the target detection model:

[0107] Determine a first loss error based on a predicted bounding box, a true bounding box, and a first target loss function corresponding to the target object output by the target detection model;

[0108] And adjust the model parameters of the target detection model based on the first loss error.

[0109] In practical applications, the target detection model can be trained by referring to the architecture of models such as the rtmdet model, the R-CNN model, the YOLO model, and the SSD model, without any restriction.

[0110] The key point detection model of the embodiment of the present application includes a first sub-model and a second sub-model. The first sub-model is used to determine the pixel coordinates corresponding to each key point of the target object in the image; the second sub-model is used to detect the affine transformation information of the target object in the image to be processed compared with the target object in the reference image at the pixel level.

[0111] Among them, the training process of the first sub-model is as follows:

[0112] The second sample images in the second training sample set are input into the initial first sub-model for training at least once until the training stop condition of the first sub-model is met (the second loss function converges or the number of training times reaches the target number of times), thereby obtaining a trained first sub-model; each second sample image includes a real bounding box of the target object; the training label of each second sample image is a labeled key point set for the target object in the area of ​​the real bounding box; the labeled key point can be labeled by a preset labeling tool or manually, without limitation.

[0113] The following operations are performed during each training of the first submodel:

[0114] Determine a second loss error based on the predicted key points, the labeled key points, and the second target loss function set for the target object within the area of ​​the true bounding box output by the first sub-model;

[0115] Model parameters of the first sub-model are adjusted based on the second loss error.

[0116] The purpose of marking the key points of the target object is to train the first sub-model, such as Figure 3 As shown, an embodiment of the present application provides a schematic diagram of key points annotated for a target object in two second sample images (sample image A and sample image B). In each second sample image, 10 key points can be annotated around the target object, namely P0, P1, P2, P3, P4, ..., P9. The target objects in different images should have key points indicating the same position. For example, if a thumb corresponds to a key point, then each image should have a key point corresponding to the thumb.

[0117] It is best to have more than 100 labeled second sample images. Of course, different models have different requirements for the number of labeled images. There is no specific rule for selecting key points. Just mark the key points at corners or mutations.

[0118] The training process of the second sub-model is as follows:

[0119] The predicted key points obtained by the first sub-model are input into the initial second sub-model,

[0120] Inputting the second sample image in the second training sample set into the initial second sub-model for at least one training until the convergence condition of the second sub-model is met, thereby obtaining a trained second sub-model;

[0121] During each training of the second sub-model, the following operations are performed:

[0122] The third loss error is determined based on the pixel coordinates of the predicted key points output by the first sub-model, the preset pixel coordinates and the third target loss function; the preset pixel coordinates can be randomly set pixel coordinates during the training process, or they can be the reference pixel coordinates of the target object in the reference image, and there is no restriction on this. After the application stage of the key point detection model, it is necessary to register the reference image to the key point detection model so that the key point detection model has the reference pixel coordinates of the target object in the reference image.

[0123] Model parameters of the second sub-model are adjusted based on the third loss error.

[0124] In practical applications, the key point detection model can be trained by referring to the architecture of the rtmpose model and the dekr_hrnet model.

[0125] After obtaining the trained target detection model and key point detection model, the embodiment of the present application calls the trained target detection model to detect the target object in the first image, and obtains the labeled image output by the target detection model after marking the real bounding box for the target object in the first image.

[0126] Furthermore, the first sub-model of the trained key point detection model is called to perform key point extraction processing on the target object in the corresponding area of ​​the real bounding box of the annotated image, and the first pixel coordinates corresponding to each key point of the target object output by the first sub-model are obtained; the key point detection model includes the reference pixel coordinates corresponding to each key point of the target object in the reference image; the second sub-model of the key point detection model is called to obtain the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level output by the second sub-model based on the first pixel coordinates in the first image and the reference pixel coordinates of the target object in the reference image.

[0127] In some embodiments, performing sub-pixel edge extraction processing on the second image according to the pixel-level affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image includes:

[0128] Calling a first operator of a trained sub-pixel edge model to perform interpolation processing and sub-pixel gradient detection processing on the second image to obtain a first gradient map;

[0129] Calling a second operator of the sub-pixel edge model to perform sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-image, and pixel-level affine transformation information, to obtain second sub-pixel edge information of the target object in the second image;

[0130] The reference sub-image is a sub-image extracted from the reference image based on sub-pixel edge information of the target object in the reference image; the reference gradient image is a gradient image obtained after interpolation processing and gradient detection processing are performed on the reference image.

[0131] Before performing sub-pixel edge detection, the embodiment of the present application also trains a sub-pixel edge model. The sub-pixel edge model is based on the geometric positioning of edge information. Because it is based on sub-pixel edges, it can achieve a positioning accuracy within 1 pixel. Therefore, the sub-pixel edge model is a high-precision model.

[0132] After obtaining the trained sub-pixel edge model, the embodiment of the present application registers a reference image with the sub-pixel edge model. The registered reference image is used to output the corresponding scaling and rotation result values ​​during subsequent edge detection. Because affine transformations such as scaling and rotation are all relative to a specific reference, the registered reference is to search the reference image with the key point detection model and the sub-pixel edge model to obtain the reference information of the target object in the reference image. The reference information represents that the scaling and angle are both 0.

[0133] The sub-pixel edge model of the embodiment of the present application includes multiple operators, wherein the multiple operators include a first operator and a second operator, the first operator is used to perform interpolation processing and sub-pixel gradient detection processing on the image, and the first operator can be a Sobel operator, and the second operator is used to perform sub-pixel edge detection on the image, and the second operator can be a Canny operator.

[0134] The first operator of the trained sub-pixel edge model can be called to perform interpolation processing and sub-pixel gradient detection processing on the second image to obtain a first gradient map of the second image, the first gradient map including a horizontal sub-pixel gradient map in the horizontal direction and a vertical sub-pixel gradient map in the vertical direction. That is, the input of the first operator is an image, and the output is a sub-pixel gradient map of the image. The gradient map includes a gradient vector of each sub-pixel point in the image.

[0135] The second operator can be called to perform sub-pixel edge detection on the second image according to the first gradient map and the baseline gradient map of the baseline sub-map, and pixel-level affine transformation information to obtain second sub-pixel edge information of the target object in the second image; the baseline sub-map is a sub-map extracted from the baseline image based on the sub-pixel edge information of the target object in the baseline image, the baseline gradient map is a gradient map obtained after interpolation processing and gradient detection processing are performed on the baseline image, the size of the baseline sub-map is smaller than the size of the second image, and then the size of the baseline gradient map is smaller than the size of the first gradient map, the input of the second operator is the sub-pixel gradient map of the image, and the output is the sub-pixel edge information of the image.

[0136] like Figure 4 As shown, the embodiment of the present application provides a second flow chart of an edge detection method, including:

[0137] Model training phase:

[0138] The target detection model, the key point detection model and the sub-pixel edge model are trained to obtain the trained target detection model, the key point detection model and the sub-pixel edge model. The detailed training process of each model is shown in the aforementioned embodiment.

[0139] Registering reference images to the object detection model, keypoint detection model, and sub-pixel edge model;

[0140] Positioning phase:

[0141] Inputting the first image to be processed into the target detection model to obtain a labeled image output by the target detection model after annotating a true bounding box for the target object in the first image;

[0142] Input the labeled image into the key point detection model, obtain the first pixel coordinates of each key point of the target object output by the key point detection model in the first image, and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level;

[0143] Determine and extract a second image from the first image according to each first pixel coordinate;

[0144] Inputting the second image and the pixel-level affine transformation information into a sub-pixel edge model to obtain second sub-pixel edge information of the target object in the second image;

[0145] Mapping the second sub-pixel edge information to the first image to obtain first sub-pixel edge information of the target object in the first image;

[0146] According to the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, a positioning result of the target object in the first image is obtained, and the positioning result includes radiation transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level.

[0147] The detailed processing process executed by each model is shown in the above embodiment and will not be described again here.

[0148] In some embodiments, the size of the first gradient map is larger than the size of the reference gradient map; calling the second operator of the sub-pixel edge model to perform sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-map, and pixel-level affine transformation information, includes: performing the following operations by the second operator:

[0149] Transforming the first gradient map according to pixel-level affine transformation information to obtain a second gradient map;

[0150] Normalizing the second gradient map and the reference gradient map;

[0151] A sliding window is set in the second gradient map to traverse the second gradient map, and the sliding window and the reference gradient map have the same size; wherein, for the current sliding window currently traversed, the current sliding window and the reference gradient map are gradient matched to obtain a matching degree between the sliding window and the reference gradient map;

[0152] Determine a first sub-pixel region corresponding to the sliding window of maximum matching degree in the interpolated second image;

[0153] Sub-pixel boundary information of the first pixel region is determined from the interpolated second image; the sub-pixel boundary information of the first pixel region is second sub-pixel edge information of the target object.

[0154] In order to further improve the accuracy of edge detection, it is necessary to transform the first gradient map according to the pixel-level affine transformation information through the second operator to obtain the second gradient map. Specifically, the influence of the pixel-level affine transformation information (translation, rotation, scaling, shearing) is propagated to the first gradient map to change the direction and amplitude of the gradient. Through inverse mapping and matrix multiplication, the gradient value of each pixel in the transformed image can be calculated, and the second gradient map is obtained after adjustment.

[0155] In addition, in order to eliminate scale differences and facilitate comparison, the second gradient map and the reference gradient map are normalized.

[0156] In order to determine the sub-pixel edge information of the target object from the second image, a sliding window is set in the second gradient map of the second image to traverse the second gradient map. The sliding window and the reference gradient map have the same size, so that the matching degree between the sliding window and the reference gradient map can be calculated. Specifically, for the current sliding window currently traversed, the current sliding window and the reference gradient map are gradient matched to obtain the matching degree between the sliding window and the reference gradient map.

[0157] In the embodiment of the present application, the matching degree between the sliding window and the reference gradient map can be determined by the following formula (1):

[0158]

[0159] Among them, score represents the matching degree, n represents the reference gradient or the number of gradient vectors in the sliding window, (pi,qj) represents the gradient vector of the pixel point in the reference image, (p′i,q′i) represents the gradient vector of the pixel point in the sliding window, and the gradient vector consists of amplitude and direction. Pi represents the amplitude of the pixel point in the reference image, qi represents the direction of the pixel point in the reference image, p′i represents the amplitude of the pixel point in the reference image, and q′i) represents the direction of the pixel point in the reference image.

[0160] After obtaining the matching degree between each sliding window and the reference gradient map, the sliding window with the maximum matching degree can be determined. The sliding window with the maximum matching degree is the target window to be queried. The first pixel area corresponding to the target window in the second image after interpolation can be determined. The sub-pixel boundary information of the first pixel area can be determined from the second image after interpolation. The sub-pixel boundary information of the first pixel area is the second sub-pixel edge information of the target object.

[0161] In some embodiments, mapping the second sub-pixel edge information to the first image to obtain the first sub-pixel edge information of the target object in the first image includes:

[0162] The first image is interpolated to obtain an interpolated first image; the second sub-pixel edge information is mapped to the interpolated first image to obtain first sub-pixel edge information of the target object in the first image.

[0163] When performing sub-pixel gradient detection on the second image, the second image is interpolated to obtain an interpolated second image, in which two adjacent pixels in the interpolated second image are spaced apart by a second target number of sub-pixels; in order to map the second sub-pixel edge information to the first image, the first image needs to be interpolated to obtain an interpolated first image; in the interpolated first image, two adjacent pixels are also spaced apart by a second target number of sub-pixels, that is, the first image and the second image are interpolated in the same manner, and the specific interpolation manner may be bilinear interpolation, bicubic interpolation, etc., or other interpolation manners, which are not limited thereto.

[0164] After obtaining the interpolated first image, the second sub-pixel edge information is mapped to the interpolated first image to obtain the first sub-pixel edge information of the target object in the first image.

[0165] The positioning method provided in the embodiment of the present application can be executed by a positioning device. In the embodiment of the present application, the positioning method executed by the positioning device is taken as an example to illustrate the positioning device provided in the embodiment of the present application.

[0166] The embodiment of the present application also provides a positioning device.

[0167] like Figure 5 As shown, the positioning device includes: a first processing module 510 , a second processing module 520 , a third processing module 530 and a fourth processing module 540 .

[0168] A first processing module 510 is used to perform key point extraction processing on the target object in the first image to obtain first pixel coordinates of each key point of the target object in the first image and affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level;

[0169] A second processing module 520, for determining and extracting a second image from the first image according to each first pixel coordinate;

[0170] A third processing module 530, configured to determine first sub-pixel edge information of the target object in the first image according to the second image, the pixel-level affine transformation information and the reference image;

[0171] The fourth processing module 540 is used to obtain a positioning result of the target object in the first image based on the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, wherein the positioning result includes the radiation transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level.

[0172] According to the positioning device provided in the embodiment of the present application, by performing key point extraction processing on the target object in the first image, the first pixel coordinates of each key point of the target object in the first image and the affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level are obtained, so as to roughly detect the edge information of the target object, effectively reduce the influence of factors such as image material (color, texture, reflectivity, transparency), contrast and noise on the positioning of the target object, and improve the anti-interference ability of positioning.

[0173] In addition, according to each first pixel coordinate, the second image is determined and extracted from the first image; according to the second image, the affine transformation information and the reference image, the first sub-pixel edge information of the target object in the first image is determined; according to the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, the positioning result of the target object in the first image is obtained, and the positioning result includes the radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level. Compared with performing sub-pixel edge detection and sub-pixel positioning in the entire first image, performing sub-pixel detection in the second image reduces the range of high-precision sub-pixel edge detection, which can effectively reduce the delay of high-precision edge detection. In addition, compared with performing sub-pixel edge detection in the second pixel area, performing sub-pixel in the second image appropriately expands the range of sub-pixel edge detection, which can better capture the edge information in the image and improve the sub-pixel positioning accuracy and robustness.

[0174] In addition, from a hardware perspective, regardless of whether the controller or processor of the device has a GPU, the above-mentioned sub-pixel edge detection method can be executed, and it can also be run in a CPU without a GPU, which can effectively reduce costs.

[0175] In some embodiments, the third processing module 530:

[0176] An edge extraction submodule, used to perform sub-pixel edge extraction processing on the second image according to the pixel-level affine transformation information and the reference image, so as to obtain second sub-pixel edge information of the target object in the second image;

[0177] The mapping submodule is used to map the second sub-pixel edge information to the first image to obtain the first sub-pixel edge information of the target object in the first image.

[0178] In some embodiments, the edge extraction submodule specifically includes:

[0179] A first processing unit is used to call a first operator of a trained sub-pixel edge model to perform interpolation processing and sub-pixel gradient detection processing on the second image to obtain a first gradient map;

[0180] A second processing unit is used to call a second operator of the sub-pixel edge model to perform sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-image, and pixel-level affine transformation information, so as to obtain second sub-pixel edge information of the target object in the second image;

[0181] The reference sub-image is a sub-image extracted from the reference image based on sub-pixel edge information of the target object in the reference image; the reference gradient image is a gradient image obtained after interpolation processing and gradient detection processing are performed on the reference image.

[0182] In some embodiments, the size of the first gradient map is larger than the size of the reference gradient map; the second processing unit is specifically configured to perform the following operations through the second operator:

[0183] Transforming the first gradient map according to pixel-level affine transformation information to obtain a second gradient map;

[0184] Normalizing the second gradient map and the reference gradient map;

[0185] A sliding window is set in the second gradient map to traverse the second gradient map, and the sliding window and the reference gradient map have the same size; wherein, for the current sliding window currently traversed, the current sliding window and the reference gradient map are gradient matched to obtain a matching degree between the sliding window and the reference gradient map;

[0186] Determine a first sub-pixel region corresponding to the sliding window of maximum matching degree in the interpolated second image;

[0187] Sub-pixel boundary information of the first pixel region is determined from the interpolated second image; the sub-pixel boundary information of the first pixel region is second sub-pixel edge information of the target object.

[0188] In some embodiments, the mapping submodule is specifically configured to:

[0189] Performing interpolation processing on the first image to obtain an interpolated first image;

[0190] The second sub-pixel edge information is mapped to the interpolated first image to obtain first sub-pixel edge information of the target object in the first image.

[0191] In some embodiments, the first processing module 510 is specifically configured to:

[0192] Calling a trained target detection model to detect a target object in the first image, and obtaining a labeled image output by the target detection model after annotating a real bounding box for the target object in the first image;

[0193] Calling the first sub-model of the trained key point detection model to perform key point extraction processing on the target object in the area corresponding to the real bounding box of the annotated image, and obtaining the first pixel coordinates of each key point of the target object output by the first sub-model corresponding to the first image; the key point detection model includes the reference pixel coordinates corresponding to each key point of the target object in the reference image;

[0194] The second submodel of the key point detection model is called to obtain the pixel-level affine transformation information of the target object in the first image compared to the target object in the reference image output by the second submodel based on the first pixel coordinates in the first image and the reference pixel coordinates of the target object in the reference image.

[0195] In some embodiments, the second processing module 520 is specifically used to:

[0196] Determine in the first image a second pixel region with the first pixel coordinates as boundaries;

[0197] In the first image, pixel expansion is performed on the second pixel region along the target direction to obtain a second image.

[0198] The positioning device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0199] The positioning device in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0200] The positioning device provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0201] In some embodiments, Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the above-mentioned positioning method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0202] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices described above. The electronic device may be a terminal or other devices other than a terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It may also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.

[0203] The embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned positioning method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0204] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0205] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above positioning method when executed by a processor.

[0206] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0207] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned positioning method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0208] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0209] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0210] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0211] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0212] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0213] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A positioning method, characterized in that: include: Performing key point extraction processing on the target object in the first image to obtain first pixel coordinates of each key point of the target object in the first image and affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level; Determine and extract a second image from the first image according to each of the first pixel coordinates; Determining first sub-pixel edge information of a target object in the first image according to the second image, the pixel-level affine transformation information, and the reference image; Based on the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, a positioning result of the target object in the first image is obtained, and the positioning result includes radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level.

2. The positioning method according to claim 1, characterized in that: The determining, according to the second image, the pixel-level affine transformation information and the reference image, first sub-pixel edge information of the target object in the first image comprises: Performing sub-pixel edge extraction processing on the second image according to the pixel-level affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image; The second sub-pixel edge information is mapped to the first image to obtain first sub-pixel edge information of the target object in the first image.

3. The positioning method according to claim 2, characterized in that: The step of performing sub-pixel edge extraction processing on the second image according to the pixel-level affine transformation information and the reference image to obtain second sub-pixel edge information of the target object in the second image includes: Calling a first operator of a trained sub-pixel edge model to perform interpolation processing and sub-pixel gradient detection processing on the second image to obtain a first gradient map; Calling a second operator of the sub-pixel edge model to perform sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-image, and the pixel-level affine transformation information, to obtain second sub-pixel edge information of the target object in the second image; The reference sub-image is a sub-image extracted from the reference image based on sub-pixel edge information of a target object in the reference image; the reference gradient image is a gradient image obtained after interpolation processing and gradient detection processing are performed on the reference image.

4. The positioning method according to claim 3, characterized in that: The size of the first gradient map is larger than the size of the reference gradient map; the second operator calling the sub-pixel edge model performs sub-pixel edge detection on the second image according to the first gradient map and the reference gradient map of the reference sub-map, and the pixel-level affine transformation information, including: performing the following operations by the second operator: Transforming the first gradient map according to the pixel-level affine transformation information to obtain a second gradient map; Normalizing the second gradient map and the reference gradient map; A sliding window is set in the second gradient map to traverse the second gradient map, and the sliding window and the reference gradient map have the same size; wherein, for a current sliding window currently traversed, gradient matching is performed on the current sliding window and the reference gradient map to obtain a matching degree between the sliding window and the reference gradient map; Determine a first sub-pixel region corresponding to the sliding window of maximum matching degree in the interpolated second image; The sub-pixel boundary information of the first pixel region is determined from the interpolated second image; the sub-pixel boundary information of the first pixel region is the second sub-pixel edge information of the target object.

5. The positioning method according to claim 2, characterized in that: Mapping the second sub-pixel edge information to the first image to obtain first sub-pixel edge information of the target object in the first image includes: Performing interpolation processing on the first image to obtain an interpolated first image; The second sub-pixel edge information is mapped to the interpolated first image to obtain first sub-pixel edge information of the target object in the first image.

6. The positioning method according to claim 1, characterized in that: The step of performing key point extraction processing on the target object in the first image to obtain first pixel coordinates of each key point of the target object in the first image and affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level includes: Calling a trained target detection model to detect a target object in the first image, and obtaining a labeled image output by the target detection model after annotating a real bounding box for the target object in the first image; Calling a first sub-model of a trained key point detection model to perform key point extraction processing on the target object in the area corresponding to the real bounding box of the annotated image, and obtaining first pixel coordinates corresponding to each key point of the target object in the first image output by the first sub-model; the key point detection model includes reference pixel coordinates corresponding to each key point of the target object in the reference image; The second sub-model of the key point detection model is called to obtain the pixel-level affine transformation information of the target object in the first image compared with the target object in the reference image output by the second sub-model based on the first pixel coordinates in the first image and the reference pixel coordinates of the target object in the reference image.

7. The positioning method according to any one of claims 1 to 6, characterized in that: The step of determining and extracting a second image from the first image according to the first pixel coordinates includes: Determining, in the first image, a second pixel region with the first pixel coordinates as boundaries; In the first image, pixel expansion is performed on the second pixel region along a target direction to obtain the second image.

8. A positioning device, characterized in that: include: A first processing module, configured to perform key point extraction processing on the target object in the first image, obtain first pixel coordinates of each key point of the target object in the first image, and affine transformation information of the target object in the first image compared with the target object in the reference image at the pixel level; A second processing module, configured to determine and extract a second image from the first image according to each of the first pixel coordinates; a third processing module, configured to determine first sub-pixel edge information of a target object in the first image according to the second image, the pixel-level affine transformation information and the reference image; A fourth processing module is used to obtain a positioning result of the target object in the first image based on the first sub-pixel edge information and the reference sub-pixel edge information of the target object in the reference image, wherein the positioning result includes radial transformation information of the target object in the first image compared with the target object in the reference image at the sub-pixel level.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the positioning method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the positioning method according to any one of claims 1 to 7 is implemented.

Images