Method, device, computer-readable medium and program product for estimating initial pose

Abstract

The present application provides a method, device, computer-readable medium and program product for estimating an initial pose. The method comprises: acquiring an actual image of an object to be tracked; adjusting the pose of a pre-established three-dimensional model of the object to be tracked based on the actual image; extracting characteristic control line segments of the three-dimensional model based on the three-dimensional model after the pose adjustment, searching for matching corresponding line segments in the actual image based on the characteristic control line segments; and determining the initial pose of the object to be tracked based on the corresponding line segments. The method of the present application improves the accuracy of the correspondence between the two-dimensional image and the three-dimensional scene, thereby improving the accuracy of the initial pose estimation.

Description

Technical Field

[0001] Embodiments of the present application relate to an initial pose estimation method, and more particularly to an initial pose estimation method, device, computer-readable medium, and program product. Background Art

[0002] Restoring the three-dimensional information (6-DOF pose) of the target object from the two-dimensional image for real-time tracking has always been a research hotspot in the field of computer vision. The current tracking algorithm is mainly based on the slight differences between image frames. That is, when the pose of the previous frame is known, since the difference between the two frames is very small, the previous frame information can be used as prior information and only incremental updates are performed to obtain the current frame pose, thereby achieving real-time tracking. It can be seen that the tracking algorithm strongly relies on the previous frame information to perform real-time pose tracking. For the initial frame, it is necessary to calculate the pose of the object without the previous frame information. Therefore, it is particularly important to accurately obtain the initial frame, that is, the initial pose estimation problem. The initial pose estimation is the premise and basis for the operation of the tracking algorithm. In the prior art, the estimation of the initial pose is usually carried out by establishing the correspondence between the two-dimensional image and the three-dimensional scene, and then solving the simultaneous equations to determine it.

[0003] However, the existing technology usually includes all lines in the image in establishing the correspondence between the two-dimensional image and the three-dimensional scene, which cannot guarantee the accuracy of the correspondence, resulting in low accuracy of the established correspondence, and thus low accuracy of the initial pose estimation. Summary of the invention

[0004] The present application provides an initial pose estimation method, device, computer readable medium and program product, which are used to solve the problem in the prior art of low accuracy of initial pose estimation due to inaccurate correspondence between established two-dimensional images and three-dimensional scenes.

[0005] On the one hand, the present application provides a method for estimating an initial posture, comprising:

[0006] Acquire actual images of the object to be tracked;

[0007] Based on the actual image, the pre-established three-dimensional model of the object to be tracked is adjusted in position and posture;

[0008] Based on the three-dimensional model after posture adjustment, extract the characteristic control line segments of the three-dimensional model;

[0009] Based on the feature-controlled line segments, a matching corresponding line segment is searched in the actual image;

[0010] Determine the initial pose of the object to be tracked based on the corresponding line segments.

[0011] In a second aspect, an embodiment of the present application provides an initial posture estimation device, comprising:

[0012] An acquisition module, used for acquiring an actual image of the object to be tracked;

[0013] An adjustment module, used for adjusting the position and posture of a pre-established three-dimensional model of an object to be tracked based on an actual image;

[0014] An extraction module extracts characteristic control line segments of the three-dimensional model based on the three-dimensional model after the posture adjustment; and searches for matching corresponding line segments in the actual image based on the characteristic control line segments;

[0015] The determination module determines the initial pose of the object to be tracked according to the corresponding line segments.

[0016] In a third aspect, an embodiment of the present application provides an initial posture estimation device,

[0017] Including: memory, processor;

[0018] Wherein, the memory is used to store executable instructions;

[0019] The processor is configured to run the instructions stored in the memory to perform any initial pose estimation method of the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement any initial pose estimation method of the first aspect.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any method for estimating the initial pose of the first aspect.

[0022] The present application provides a method, device, computer-readable medium and program product for estimating an initial pose. The method collects an actual image of an object to be tracked; based on the actual image, adjusts the pose of a pre-established three-dimensional model of the object to be tracked; based on the three-dimensional model after the pose adjustment, extracts the characteristic control line segments of the three-dimensional model; based on the characteristic control line segments, searches for matching corresponding line segments in the actual image; and determines the initial pose of the object to be tracked based on the corresponding line segments. The method of the present application improves the accuracy of the correspondence between the two-dimensional image and the three-dimensional scene, thereby improving the accuracy of the initial pose estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 A flow chart of a method for estimating an initial posture provided in an embodiment of the present application;

[0025] Figure 2 A flow chart of another method for estimating an initial posture provided in an embodiment of the present application;

[0026] Figure 3 A flow chart of another method for estimating initial posture provided in an embodiment of the present application;

[0027] Figure 4 A flow chart of another method for estimating an initial posture provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of searching for a corresponding straight line segment in a neighborhood according to an embodiment of the present application;

[0029] Figure 6 A schematic diagram of an initial posture estimation device provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the technical solution of the present invention more clearly presented, the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail with specific embodiments below. Obviously, the described embodiments are only a part of the embodiments, rather than all the embodiments, and are not used to limit this invention. The same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0032] First, the terms involved in this application are explained:

[0033] LSD: Line Segment Detector is a line segment detection algorithm. This algorithm can obtain high-precision line segment detection results in a short time. The LSD line detection algorithm first calculates the gradient size and direction of all points in the image, and then regards the points with small gradient direction changes and adjacent points as a connected domain. Then, according to the rectangularity of each domain, it is determined whether it needs to be disconnected according to the rules to form multiple domains with larger rectangularity. Finally, all the generated domains are improved and screened, and the domains that meet the conditions are retained, which is the final line detection result. The advantages of this algorithm are fast detection speed, no parameter adjustment, and the error control method is used to improve the accuracy of line detection.

[0034] GL environment: Open Graphics Library Translation: Open Graphics Library or "Open Graphics Library" is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. This interface is used to draw simple graphics to complex three-dimensional scenes.

[0035] CAD: Computer Aided Design, refers to the use of computers and their graphic equipment to help designers perform design work.

[0036] Pixels: Pixels are made up of small squares in an image. These small squares have a clear position and an assigned color value. The color and position of the small squares determine how the image appears.

[0037] Grayscale: A grayscale digital image is an image with only one sampled color per pixel. This type of image is usually displayed as a grayscale from the darkest black to the brightest white. Grayscale images are different from black and white images. In the field of computer graphics, black and white images only have two colors, black and white, while grayscale images have many levels of color depth between black and white.

[0038] Least squares: Least squares method (also known as least square method) is a mathematical optimization technique. It finds the best function matching data by minimizing the sum of squares of errors. In addition, it can easily obtain unknown data and minimize the sum of squares of errors between the obtained data and the actual data.

[0039] The application scenarios of the embodiments of the present application are applicable to any scenario where the target object needs to be tracked, for example: it can be applied to augmented reality technology, by calculating the position of the camera image in real time, to achieve a "seamless" connection between real-world information and virtual-world information. For another example: it can also be applied to the field of robot vision, by performing position recognition and state analysis of obstacles and road signs, to achieve visual navigation of mobile robots. It is understandable that this is only used as an example of two specific usage scenarios, but it does not limit the present application.

[0040] Posture initialization is the premise for real-time tracking of the target object’s posture. This application provides a method for estimating the initial posture, which improves the accuracy of the correspondence between the two-dimensional image and the three-dimensional scene by pre-adjusting the posture of the three-dimensional model and searching for corresponding line segments in the neighborhood, thereby improving the accuracy of the initial posture estimation. This enables more accurate real-time tracking of the target object.

[0041] The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0042] Figure 1 A flow chart of an initial posture estimation method provided by an embodiment of the present invention, such as Figure 1 As shown, including:

[0043] S101: Acquire an actual image of an object to be tracked.

[0044] The actual image of the object to be tracked is collected by an image acquisition device, such as a camera, so that the object to be tracked appears in the picture. The image collected by the camera is a dynamic two-dimensional image.

[0045] S102: Based on the actual image, adjust the position and posture of the pre-established three-dimensional model of the object to be tracked.

[0046] By adjusting the presentation posture of the three-dimensional model, it is made to be similar to the posture of the object to be tracked captured by the camera. In this embodiment, the three-dimensional model is a three-dimensional CAD model. It can be understood that the CAD model pre-established in this application can also be other types of three-dimensional models, so the CAD model cannot be used as a limitation to this application. The specific operation of posture adjustment can be translation or rotation, etc., and the specific method of posture adjustment can be performed on the touch screen of the mobile device using gestures such as rotation and scaling, or on the PC side using peripherals such as a mouse.

[0047] S103: extracting characteristic control line segments of the three-dimensional model based on the three-dimensional model after posture adjustment.

[0048] After confirming that the posture adjustment of the 3D CAD model is completed, the characteristic control line segments of the adjusted 3D CAD model are extracted. Before extracting the characteristic control line segments of the 3D CAD model, the 3D CAD model is projected in an optional manner, and its projected straight line segments are extracted from the projection diagram. Further, the characteristic control line segments of the 3D CAD model are determined by the projected straight line segments. It should be noted that the projected straight line segments obtained at this time are two-dimensional line segments, and the characteristic control line segments obtained according to the projected straight line segments are three-dimensional space line segments.

[0049] Among them, the method of extracting the projected straight line segment can use various existing algorithms, for example, the LSD algorithm can be used to detect the straight line segment in the projection image of the 3D CAD model. The goal of the LSD algorithm is to detect the local straight contour in the image. The contour is some special area in the image. In these areas, the grayscale of the image changes dramatically from black to white or from white to black. By using the LSD algorithm, the purpose of quickly detecting the straight line segment in the image is achieved.

[0050] S104 . Based on the feature control line segment, search for a matching corresponding line segment in the actual image.

[0051] After obtaining the control straight line segments of the 3D CAD model, the LSD algorithm is similarly used to extract straight line segments in the actual image captured by the camera, and a neighborhood search is performed among these straight line segments to obtain corresponding straight line segments that match the control straight line segments.

[0052] S105: Determine the initial position and posture of the object to be tracked according to the corresponding line segment.

[0053] If the matched control straight line segment-corresponding straight line segment meets the set quantity threshold, the initial posture is determined according to the corresponding straight line segment that matches the control straight line segment. Optionally, a nonlinear optimization method can be used to solve the problem of minimizing the Euclidean distance between the projection of the control straight line segment and its corresponding straight line segment. Since the optimization process involves iterative posture solution, the iterative initial value uses the posture parameter after the three-dimensional model is adjusted in step S102. The posture parameter includes rotation and translation, with a total of 6 degrees of freedom.

[0054] The embodiment of the present application provides a method for estimating an initial posture, which collects an actual image of an object to be tracked; based on the actual image, adjusts the posture of a pre-established three-dimensional model of the object to be tracked; based on the three-dimensional model after the posture adjustment, extracts the characteristic control line segments of the three-dimensional model; based on the characteristic control line segments, searches for matching corresponding line segments in the actual image; and determines the initial posture of the object to be tracked based on the corresponding line segments. The method of this embodiment improves the accuracy of the correspondence between the two-dimensional image and the three-dimensional scene, thereby improving the accuracy of the initial posture estimation.

[0055] Figure 2 A flowchart of another embodiment of a method for estimating an initial posture provided by an embodiment of the present invention, such as Figure 2 As shown, in Figure 1 On the basis of the illustrated embodiment, the posture adjustment of the pre-established three-dimensional model of the object to be tracked based on the actual image of the object to be tracked can be achieved by the following steps, including:

[0056] S1021, setting the background of the rendering layer of the three-dimensional model of the object to be tracked to be transparent, setting it as the top layer, and superimposing and aligning the rendering layer of the three-dimensional model with the actual image.

[0057] S1022: adjusting the posture of the three-dimensional model based on the actual image of the object to be tracked.

[0058] Based on the CAD model of the existing target object, the CAD model is loaded and superimposed and rendered to the top layer of the screen. This is similar to the application of layers in image processing. A layer can be understood as a completely transparent paper, and different contents can be stored on each paper. In the present application, it can be understood that the CAD model occupies a layer, in which the layer background of the CAD model is rendered and set to be transparent. The picture taken by the camera is in another layer, and the CAD model and the image taken by the camera can be displayed on the same screen at the same time by merging or superimposing different layers. The rendering layer of the CAD model is loaded to the top layer of the screen, so that the picture taken by the camera is used as the environmental background, which facilitates the adjustment of the posture of the three-dimensional model. During the adjustment process, it should be noted that the adjustment is based on the actual image of the object to be tracked.

[0059] Further, such as Figure 3 As shown, the characteristic control line segments of the three-dimensional model after posture adjustment are extracted, and based on the control line segments, matching corresponding line segments are searched in the actual image.

[0060] in,

[0061] Extracting the three-dimensional model feature control line segments in S103 can be specifically implemented by the following steps, including:

[0062] S1031, performing off-screen rendering on the three-dimensional model to obtain a model projection image, extracting all projection line segments in the model projection image and obtaining pixel coordinates of the line segment endpoints.

[0063] After confirming that the CAD model posture adjustment is completed, the CAD model is automatically rendered off-screen into a blank image in the current posture to obtain a grayscale image. At this time, there is only the projection of the model in the image. All the projected straight line segments in the image (i.e., on the projection of the model) are extracted from the image using the LSD algorithm, which can be recorded as {l}. It should be noted that the projected straight line segments extracted at this time are two-dimensional line segments, and the pixel coordinates of the endpoints of the line segments are obtained at the same time.

[0064] The purpose of off-screen rendering is to obtain an image with a blank background. This image only contains the CAD model without the background (i.e. the image captured by the camera). This ensures that when extracting lines, only the lines of the CAD model are extracted without any interference from other external environments. Secondly, it is also to put the process into the background without affecting the interactive experience.

[0065] S1032. Back-project the pixel coordinates of the endpoints of the line segment onto the three-dimensional model to obtain the spatial point coordinates in the three-dimensional model coordinate system.

[0066] According to the pixel coordinates of the projected straight line segment obtained in S1031, reverse projection is performed on the CAD model using the pixel coordinates of the line segment endpoints in the GL environment, and the spatial point coordinates in the three-dimensional model coordinate system are obtained by calculating the intersection of the projected straight line segment and the three-dimensional model.

[0067] GL refers to OpenGL, a professional 3D programming interface, which is used to call the underlying 3D graphics library. A window environment can be built to use OpenGL to finally obtain the control line segment. S1033. According to the spatial point coordinates in the three-dimensional model coordinate system, the two spatial endpoints are connected to determine the characteristic control line segment of the three-dimensional model.

[0068] Connect the two end points of the three-dimensional model space to determine a three-dimensional line segment. Through this step, the three-dimensional line segment on the surface of the target object is automatically extracted and generated, which is called the control line segment. A set of control line segments is obtained, denoted as {L}. It should be noted that the control line segment is a three-dimensional space segment.

[0069] Correspondingly, determining the matching corresponding line segment in the actual image based on the control line segment in S104 can also be specifically implemented by the following steps, including:

[0070] S1041. Adjust the posture of the actual image acquisition device so that the three-dimensional model approaches the object to be tracked in the actual image.

[0071] The user adjusts the image acquisition device, which can be a certain device or camera, or a camera, etc. The user adjusts the camera posture to make the object to be tracked in the camera shooting picture closer to the CAD model. Only when the posture is close, the lines searched in the field are more likely to be the correct corresponding lines, thereby reducing the mismatch rate. Since the posture of the CAD model has been adjusted to be similar to the target object in S102, only a small amount of movement is required during the actual adjustment process in this step.

[0072] S1042: extract all line segments of the object to be tracked in the actual image and obtain the pixel coordinates of the endpoints of the line segments.

[0073] Similarly, the LSD algorithm is used to extract all straight line segments in the image captured by the camera, denoted as {l image}, and obtain the endpoint coordinates at the same time.

[0074] S1043. Based on each characteristic control line segment, use its two-dimensional control line segment in the model projection image to search for a matching corresponding line segment in the actual image.

[0075] As described above, the projected straight line segment obtained by projecting the CAD model corresponds to the control straight line segment obtained according to the projected straight line segment, that is, the projected straight line segment can also be understood as a two-dimensional control straight line segment.

[0076] For the projected straight line segments {l} obtained by extracting the CAD model after off-screen rendering in S1031, for each straight line segment l i , that is, control straight line segment L i The projection in the image, all straight line segments in the actual image captured by the camera {l image}Search for the corresponding straight line segment l i .

[0077] It should be noted that the corresponding straight line segment l of the neighborhood search i ', the following conditions should be met:

[0078] Specific:

[0079] Condition 1: l i and l i The parallelism of ′ is within the preset threshold (20°);

[0080] Condition 2: l i and l i ′The distance dist(l i The midpoint to l i ′) is within the preset threshold (15% of the image width);

[0081] Condition 3: put l i The two end points of ′ are projected vertically to l i or its extension line, the corresponding line segment is i There is overlap;

[0082] Condition 4: l i With l i The length ratio of ' is within the preset threshold (0.67<l i :l i '<1.5) range;

[0083] Condition 5: If {l image If multiple straight line segments in} meet the above conditions, the one with the highest parallelism in condition 1 is selected.

[0084] Among them, condition 5 can be a further supplement to condition 1, that is, if there are multiple straight line segments l i In addition, the contents in brackets in the above conditions are only examples of possible angles, distances or lengths, which may have different settings under different application conditions and are not intended to limit the present application.

[0085] Furthermore, Figure 4 A flowchart of another embodiment of an initial posture estimation method provided by an embodiment of the present invention, such as Figure 4 As shown, in S105, the initial position and posture of the object to be tracked is determined according to the corresponding line segment, including:

[0086] If the number of corresponding straight line segments matching the characteristic control line segment meets the set number, the pose that makes the distance between the projection line segment of the control line segment and the matching corresponding straight line segment satisfy the least squares is taken as the initial pose.

[0087] Specifically:

[0088] If the proportion of corresponding straight line segments found in the projected straight line segment {l} is higher than a preset threshold (for example, 50%) and the number of projected straight line segment-corresponding straight line segment pairs is ≥ 6, then the number of corresponding straight line segments matching the feature control line segment satisfies the set number and the posture is solved. Otherwise, this frame is skipped and the process returns to S1042.

[0089] Pose solving process: Since the projected straight line segment-corresponding straight line segment meets the set number, the projected straight line segment {l [R|t] The Euclidean distance between the corresponding straight line segment in S106 satisfies the least squares, that is,

[0090]

[0091] Therefore, this pose is used as the initial pose of the object to be tracked. Where dist is the projected straight line segment Proj(L i ,[R|t]) to l i ′ distance.

[0092] like Figure 5 This formula can be solved by nonlinear optimization methods (such as Levenberg-Marquardt). The optimization process involves iterative posture solving. By pre-adjusting the posture of the CAD model in S102, an initial value closer to the optimal solution has been obtained. Therefore, the iterative initial value uses the posture parameters adjusted in S102, so that the optimization can converge to the global optimal solution.

[0093] This application improves the accuracy of the correspondence between the two-dimensional image and the three-dimensional scene by pre-adjusting the posture of the three-dimensional model and searching for corresponding line segments in the neighborhood, thereby improving the accuracy of the initial posture estimation. After solving the initial posture of the target object, it is equivalent to obtaining the previous frame information. Using the previous frame information as prior information, only incremental updates are needed to obtain the current frame posture, thereby achieving real-time tracking of the target object. As for the specific implementation method of incremental update, it is not the research focus of this application. Therefore, it will not be elaborated here.

[0094] Figure 6 This is a schematic diagram of an embodiment of an initial posture estimation device provided in the present application, the device comprising:

[0095] An acquisition module 61 is used to acquire an actual image of the object to be tracked;

[0096] An adjustment module 62, configured to adjust the position and posture of a pre-established three-dimensional model of the object to be tracked based on the actual image;

[0097] The extraction module 63 extracts the characteristic control line segments of the three-dimensional model based on the three-dimensional model after the posture adjustment; and searches for matching corresponding line segments in the actual image based on the characteristic control line segments;

[0098] The determination module 64 determines the initial position and posture of the object to be tracked according to the corresponding line segment.

[0099] Figure 7 This is a schematic diagram of the structure of the initial posture estimation device provided by the present application. As shown in FIG7 , the electronic device may include: at least one processor 71 and a memory 72 . Figure 7 An electronic device is shown using a processor as an example.

[0100] The memory 72 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0101] The memory 72 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0102] The processor 71 is used to execute the computer execution instructions stored in the memory 72 to implement the above-mentioned initial posture estimation method;

[0103] Among them, the processor 71 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The processor 71 determines the initial posture by running the instructions stored in the memory 72.

[0104] Optionally, in a specific implementation, if the communication interface, the memory 72 and the processor 71 are implemented independently, the communication interface, the memory 72 and the processor 71 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0105] Optionally, in a specific implementation, if the communication interface, the memory 72 and the processor 71 are integrated on a chip, the communication interface, the memory 72 and the processor 71 can communicate through an internal interface.

[0106] The present application also provides a computer-readable storage medium, which may include: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program information, and the program information is used for the above-mentioned initial pose estimation method.

[0107] An embodiment of the present application also provides a program, which, when executed by a processor, is used to execute the initial posture estimation method provided in the above method embodiment.

[0108] An embodiment of the present application also provides a program product, such as a computer-readable storage medium, in which instructions are stored. When the program product is run on a computer, the computer executes the initial posture estimation method provided by the above method embodiment.

[0109] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for estimating an initial pose, characterized in that: include: Acquire actual images of the object to be tracked; Based on the actual image, adjusting the posture of the pre-established three-dimensional model of the object to be tracked; Perform off-screen rendering on the three-dimensional model to obtain a model projection image, extract all projection line segments in the model projection image and obtain pixel coordinates of the line segment endpoints; reversely project the pixel coordinates of the line segment endpoints to the three-dimensional model to obtain spatial point coordinates in a three-dimensional model coordinate system; connect the two spatial endpoints according to the spatial point coordinates in the three-dimensional model coordinate system to determine a characteristic control line segment of the three-dimensional model; The characteristic control line segment represents a three-dimensional control line segment of the surface of the object to be tracked; Adjusting the posture of the actual image acquisition device so that the three-dimensional model approaches the object to be tracked in the actual image; Extracting all line segments of the object to be tracked in the actual image acquired by the actual image acquisition device after the posture is adjusted and obtaining the pixel coordinates of the endpoints of the line segments; Based on each of the characteristic control line segments, using its two-dimensional control line segment in the model projection image, searching for a matching corresponding line segment in the actual image; An initial position and posture of the object to be tracked is determined according to the corresponding line segment.

2. The method according to claim 1, characterized in that The step of adjusting the position and posture of the pre-established three-dimensional model of the object to be tracked based on the actual image comprises: Setting the background of the rendering layer of the three-dimensional model of the object to be tracked to be transparent, setting it as the top layer, and superimposing and aligning the rendering layer of the three-dimensional model with the actual image; The posture of the three-dimensional model is adjusted based on the actual image of the object to be tracked.

3. The method according to claim 1, characterized in that Based on each of the feature control line segments, using the two-dimensional control line segment in the model projection image, searching for a matching corresponding line segment in the actual image includes: The two-dimensional control line segment and the matching corresponding line segment satisfy the following conditions: The parallelism between the two-dimensional control line segment and the corresponding line segment is within a preset parallelism threshold range; The two-dimensional control line segment and the corresponding line segment are within a threshold range of a preset image width; When the two end points of the corresponding line segment are projected vertically onto the matching two-dimensional control line segment or its extension line, the corresponding line segment overlaps with the two-dimensional control line segment; The length ratio of the two-dimensional control line segment to the corresponding line segment is within a preset threshold range.

4. The method according to any one of claims 1 to 3, characterized in that: The determining the initial position and posture of the object to be tracked according to the corresponding line segment comprises: If the number of corresponding line segments matching the characteristic control line segment meets the set number, the posture that makes the distance between the projection line segment of the control line segment and the matching corresponding line segment satisfy the least squares is used as the initial posture.

5. An initial posture estimation device, comprising: An acquisition module, used for acquiring an actual image of the object to be tracked; An adjustment module, configured to adjust the position and posture of the pre-established three-dimensional model of the object to be tracked based on the actual image; An extraction module is used to perform off-screen rendering on the three-dimensional model to obtain a model projection diagram, extract all projection line segments in the model projection diagram and obtain pixel coordinates of the line segment endpoints; reversely project the pixel coordinates of the line segment endpoints to the three-dimensional model to obtain spatial point coordinates in the three-dimensional model coordinate system; and connect the two spatial endpoints to determine a characteristic control line segment of the three-dimensional model according to the spatial point coordinates in the three-dimensional model coordinate system; The characteristic control line segment represents a three-dimensional control line segment of the surface of the object to be tracked; Adjusting the posture of the actual image acquisition device so that the three-dimensional model approaches the object to be tracked in the actual image; Extracting all line segments of the object to be tracked in the actual image acquired by the actual image acquisition device after the posture is adjusted, and obtaining the pixel coordinates of the endpoints of the line segments; Based on each of the characteristic control line segments, using its two-dimensional control line segment in the model projection image, searching for a matching corresponding line segment in the actual image; A determination module determines an initial position and posture of the object to be tracked according to the corresponding line segment.

6. An initial pose estimation device, comprising: Memory, processor; Wherein, the memory is used to store executable instructions; The processor is configured to: run the instructions stored in the memory to execute the initial pose estimation method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the initial pose estimation method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, which, when executed by a processor, implements the initial pose estimation method according to any one of claims 1 to 4.

Images