Method, device, computer-readable medium and program product for real-time posture tracking

Abstract

The present application provides a method, device, computer-readable medium and program product for real-time tracking of posture. The method comprises: collecting the current frame of the object to be tracked, extracting the characteristic straight line segment in the current frame; calculating the projection straight line segment of the control straight line segment in the previous frame through the control straight line segment of the three-dimensional model and the object posture obtained by solving the previous frame, and the projection straight line segment represents the projection position of the control straight line segment in the previous frame; based on the projection straight line segment, searching for matching characteristic straight line segments in the current frame, and the matching characteristic straight line segments represent the projection position of the control straight line segment in the current frame; based on the difference in the projection position between the current frame and the previous frame, solving and updating the posture parameters of the object to be tracked, and realizing real-time tracking of the posture of the object to be tracked. The method of the present application improves the accuracy of the corresponding relationship between the two-dimensional image and the three-dimensional scene, thereby improving the accuracy of real-time tracking of the posture of the target object.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of target tracking technology, and in particular to a method, device, computer-readable medium and program product for real-time posture tracking. Background Art

[0002] Restoring the three-dimensional information of the target object (6-Degree of Freedom, 6-DOF posture for short) from the two-dimensional image for real-time tracking has always been a research hotspot in the field of computer vision. The currently commonly used tracking algorithm estimates the relative posture of the current frame relative to the previous frame by analyzing the edge displacement between consecutive frames, and obtains the posture of each frame of the object to be tracked through incremental updates to achieve real-time tracking.

[0003] In weak texture scenes, for target objects with obvious straight line edge features, the existing technology mostly uses the RAPID (Real-time Attitude and Position Determination) method, which searches for strong gradient points in the neighborhood.

[0004] However, the existing technology is prone to mistakenly match detected noise points or interference points as corresponding points during the search, resulting in inaccurate correspondence between the two-dimensional image and the three-dimensional scene, making it impossible to accurately track the position and posture of the target object in real time. Summary of the invention

[0005] The present application provides a real-time posture tracking method, device, computer-readable medium and program product to solve the problem that the prior art easily mismatches detected noise points or interference points as corresponding points during search, resulting in inaccurate correspondence between two-dimensional images and three-dimensional scenes, and thus failing to accurately track the posture of the target object in real time.

[0006] In a first aspect, the present application provides a real-time posture tracking method, comprising:

[0007] The current frame of the object to be tracked is collected, and the characteristic straight line segments in the current frame are extracted; the projection straight line segments of the control straight line segments in the previous frame are calculated through the control straight line segments of the three-dimensional model and the object posture calculated in the previous frame, and the projection straight line segments represent the projection positions of the control straight line segments in the previous frame; based on the projection straight line segments, matching characteristic straight line segments are searched in the current frame, and the matching characteristic straight line segments represent the projection positions of the control straight line segments in the current frame; based on the difference in projection positions between the current frame and the previous frame, the posture parameters of the object to be tracked are solved and updated to realize real-time tracking of the posture of the object to be tracked.

[0008] In a second aspect, an embodiment of the present application provides a real-time posture tracking device, comprising:

[0009] An acquisition module is used to acquire the current frame of the object to be tracked and extract the characteristic straight line segments in the current frame;

[0010] A first calculation module is used to calculate the projection straight line segment of the control straight line segment in the previous frame through the control straight line segment of the three-dimensional model and the object posture obtained by solving the previous frame, where the projection straight line segment represents the projection position of the control straight line segment in the previous frame;

[0011] A search module, used for searching for a matching feature straight line segment in the current frame based on the projected straight line segment, wherein the matching feature straight line segment represents a projection position of the control straight line segment in the current frame;

[0012] The second calculation module calculates and updates the posture parameters of the object to be tracked based on the difference in projection position between the current frame and the previous frame, thereby achieving real-time tracking of the posture of the object to be tracked.

[0013] In a third aspect, an embodiment of the present application provides a real-time posture tracking device,

[0014] Including: memory, processor;

[0015] Memory; Memory for storing processor executable instructions;

[0016] Wherein, the processor is configured to run instructions stored in the memory to execute any one of the real-time posture tracking methods of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement any one of the real-time posture tracking methods of the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any one of the real-time posture tracking methods of the first aspect.

[0019] The present application provides a method, device, computer-readable medium and program product for real-time tracking of posture. The current frame of the object to be tracked is collected, and the characteristic straight line segments in the current frame are extracted; the projection straight line segments of the control straight line segments in the previous frame are calculated through the control straight line segments of the three-dimensional model and the object posture obtained by solving the previous frame, and the projection straight line segments represent the projection positions of the control straight line segments in the previous frame; based on the projection straight line segments, matching characteristic straight line segments are searched in the current frame, and the matching characteristic straight line segments represent the projection positions of the control straight line segments in the current frame; based on the difference in projection positions between the current frame and the previous frame, the posture parameters of the object to be tracked are solved and updated, so as to realize real-time tracking of the posture of the object to be tracked. The method of the present application improves the accuracy of the corresponding relationship between the two-dimensional image and the three-dimensional scene, thereby improving the accuracy of real-time tracking of the posture of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A flow chart of a real-time posture tracking method provided in an embodiment of the present application;

[0022] Figure 2 A flow chart of another method for real-time posture tracking provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of searching for corresponding points in a neighborhood according to an embodiment of the present application;

[0024] Figure 4 A flow chart of another method for real-time posture tracking provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of searching for a corresponding straight line segment in a neighborhood according to an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a real-time posture tracking device provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] First, the terms involved in this application are explained:

[0030] RAPiD: Real-time Attitude and Position Determination is a model-based real-time tracking algorithm that uses observed image features to determine the position of a known 3D object relative to the camera, thereby achieving real-time tracking.

[0031] Image edge: refers to the discontinuity of local image characteristics, such as grayscale mutation, color mutation, texture structure mutation, etc. It is an important feature that image segmentation relies on.

[0032] Noise point: refers to the random error and variance of the measured variable, specifically image noise in this application.

[0033] Grayscale processing: It is the process of converting a color image into a grayscale image.

[0034] Euclidean distance: refers to the Euclidean metric, which is a commonly used distance definition. The Euclidean distance in two-dimensional and three-dimensional space is the actual distance between two points.

[0035] Satisfying least squares: The least squares method is a mathematical optimization technique, which is used for curve fitting in this application. It can easily obtain the result, and the result is more accurate.

[0036] Hough transform: A basic method for identifying geometric shapes from images in image processing. The basic principle of Hough transform is to transform the straight line in the image space into the parameter space, and determine the description parameters of the straight line by detecting the extreme points in the parameter space, thereby extracting the straight line in the image.

[0037] LSD: Line Segment Detector is a line segment detection algorithm. This algorithm can obtain high-precision line segment detection results in a short time. The LSD line detection algorithm first calculates the gradient size and direction of all points in the image, and then regards the points with small gradient direction changes and adjacent to each other as a connected domain. Then, according to the rectangularity of each domain, it is determined whether it is necessary to disconnect it according to the rules to form multiple domains with larger rectangularity. Finally, all the generated domains are improved and screened, and the domains that meet the conditions are retained, which is the final line detection result. The advantages of this algorithm are fast detection speed, no parameter adjustment, and the error control method is used to improve the accuracy of line detection.

[0038] The application scenarios of the embodiments of the present application are applicable to any scenario where it is necessary to track a target object, which usually has multiple straight line edge features. For example: it can be applied to the field of intelligent video surveillance, by performing motion recognition on the vehicle and detecting surrounding objects, and using the straight line features of the object to achieve real-time tracking of the position. For another example: it can also be applied to the field of robot vision, by performing position recognition and state analysis on obstacles and road signs, calculating the motion trajectory of the photographed object, and realizing visual navigation of the mobile robot. It is understandable that this is only used as an example of two specific usage scenarios, but it does not limit the present application.

[0039] The target object is tracked in real time. The relative pose of the current frame relative to the previous frame, i.e., the incremental pose, is estimated by analyzing the edge displacement between consecutive frames. The pose of each frame of the object to be tracked is obtained by incremental updating. That is, assuming that the pose of the target object in the previous frame has been obtained, if the pose change of the current frame relative to the previous frame, i.e., the incremental pose, is calculated, the pose of the current frame can be obtained by incremental updating. Furthermore, by taking the pose of the current frame as the pose of the previous frame, the pose of the next frame is calculated by the same method, and so on and so forth, the real-time tracking of the target object is finally achieved.

[0040] As mentioned above, in order to achieve real-time tracking of the target object's posture, it is necessary to first obtain the incremental posture.

[0041] In this application, it is assumed that the first frame pose has been obtained. As for the specific solution process of the first frame pose, since it is not the research focus of this application, it will not be elaborated here. At the same time, since sampling is performed on the three-dimensional model of the object to be tracked in advance, by collecting the starting and ending points of the straight line segments on the three-dimensional model, the control straight line segment set of the three-dimensional model is obtained, that is, the control straight line segment of the three-dimensional model has been obtained. This process can be understood as the preparation stage before calculating the incremental pose.

[0042] The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0043] Figure 1A flow chart of a real-time posture tracking method provided by an embodiment of the present invention, such as Figure 1 As shown, including:

[0044] S101, collecting the current frame of the object to be tracked, and extracting the characteristic straight line segments in the current frame.

[0045] The current frame of the object to be tracked is captured by an image acquisition device, such as a camera, and the current frame is a two-dimensional image, so that the object to be tracked appears in the picture, and all characteristic straight line segments in the current frame are extracted by any existing algorithm that can extract straight lines.

[0046] S102, calculating the projection straight line segment of the control straight line segment in the previous frame through the control straight line segment of the three-dimensional model and the object posture obtained by solving the previous frame, where the projection straight line segment represents the projection position of the control straight line segment in the previous frame.

[0047] In the above preparation stage, the three-dimensional straight line edges have been sampled in advance on the three-dimensional model of the object to be tracked. The control straight line segment set can be obtained by collecting the first and last points of the straight line segment on the three-dimensional model, and it is assumed that the posture of the previous frame has been obtained. According to the collected control straight line segment set and the posture of the object to be tracked obtained by solving the previous frame, its projected straight line segment is calculated by a certain algorithm, and the projected straight line segment represents the projection position of the control straight line segment in the previous frame.

[0048] S103 . Based on the projected straight line segment, search for a matching feature straight line segment in the current frame, where the matching feature straight line segment represents a projection position of the control straight line segment in the current frame.

[0049] According to the calculated projected straight line segment, a neighborhood search is performed on the projected straight line segment, and a matching feature straight line segment is searched in the current frame. The matching feature straight line segment is referred to as a corresponding straight line segment in this embodiment, and the corresponding straight line segment indicates the projection position of the control straight line segment in the current frame.

[0050] S104, based on the difference in projection position between the current frame and the previous frame, the pose parameters of the object to be tracked are solved and updated to achieve real-time tracking of the pose of the object to be tracked. After obtaining the control straight line segment-corresponding straight line segment that meets a certain number of thresholds, the incremental pose is solved based on the difference in projection position between the current frame and the previous frame, and the pose parameters of the object to be tracked are solved and updated according to the incremental pose to achieve real-time tracking of the pose of the object to be tracked.

[0051] The embodiment of the present application provides a method for real-time tracking of posture, which collects the current frame of the object to be tracked and extracts the characteristic straight line segment in the current frame; calculates the projection straight line segment of the control straight line segment in the previous frame through the control straight line segment of the three-dimensional model and the object posture obtained by solving the previous frame, and the projection straight line segment represents the projection position of the control straight line segment in the previous frame; based on the projection straight line segment, searches for matching characteristic straight line segments in the current frame, and the matching characteristic straight line segments represent the projection position of the control straight line segment in the current frame; based on the difference in projection position between the current frame and the previous frame, solves and updates the posture parameters of the object to be tracked, and realizes real-time tracking of the posture of the object to be tracked. The method of the present application improves the accuracy of the correspondence between the two-dimensional image and the three-dimensional scene, thereby improving the accuracy of real-time tracking of the posture of the target object.

[0052] Figure 2 A flow chart of another method for real-time posture tracking provided by an embodiment of the present invention is as follows: Figure 2 As shown, in Figure 1 On the basis of the embodiment shown, the current frame of the object to be tracked is captured in step S101, and the characteristic straight line segments in the current frame are extracted, which can be specifically implemented by the following steps, including:

[0053] S1011 , collecting the current frame of the object to be tracked, and performing grayscale preprocessing on the current frame.

[0054] The current frame is actually captured by an image acquisition device, such as a camera, and is a color image. A color image is an image in which each pixel is composed of R, G, and B components. Since a color image contains many pixels and occupies many channels, the subsequent image processing requires a large amount of calculation and a slow processing speed. Therefore, the captured color image needs to be preprocessed first. Optionally, the preprocessing method can be grayscale processing.

[0055] Grayscale preprocessing is performed on color images, that is, the RGB value of each pixel is unified into the same value. The grayscale image will change from three channels to a single channel, and single-channel data processing will be simpler. Grayscale processing of images is performed to prepare for other operations on the image.

[0056] S1012. Based on a straight line extraction algorithm, extract characteristic straight line segments in the grayscale current frame.

[0057] Extract all straight lines in the current frame. The extraction method can use any existing straight line extraction algorithm, for example, using Hough transform or LSD algorithm. All straight line segments in the grayscale image are obtained by Hough transform or LSD algorithm, denoted as {l image}, and obtain the coordinates of the endpoints of the straight line segment at the same time.

[0058] According to the control straight line segment of the three-dimensional model and the object posture obtained by solving the previous frame in S102, the projected straight line segment of the control straight line segment in the previous frame is calculated. The obtained control straight line segment of the three-dimensional model is recorded as {L}, and the projected straight line segment of the control straight line segment in the previous frame can be calculated, which can specifically include:

[0059] S1021. Determine the projection coordinates of the two end points of each control straight line segment through the control straight line segments of the three-dimensional model and the object posture obtained by solving the previous frame.

[0060] In this embodiment, the three-dimensional model is a three-dimensional CAD model. It is understandable that the three-dimensional CAD model in this application can also be other types of three-dimensional models. According to the control straight line segments of the acquired three-dimensional CAD model, each control straight line L is calculated using formula 1. i The projection coordinates of the two endpoints.

[0061] Formula 1:

[0062]

[0063] Among them, (X w ,Y w ,Z w ) is the three-dimensional coordinate of the control point E, [R|t] is the pose parameter including rotation and displacement, (u0,v0) is the coordinate of the center point of the image plane in the pixel coordinate system, α x =f / d x , α y =f / d y are the ratios of the camera focal length to the physical size of the pixel in the x and y directions, respectively. c Z w The coordinate value in the camera coordinate system.

[0064] S1022. Determine a projected straight line segment of the control straight line segment in the previous frame according to the projection coordinates of the two end points of each control straight line segment, where the projected straight line segment represents a projection position of the control straight line segment in the previous frame.

[0065] The projection coordinates of the two end points of each control straight line segment are obtained according to formula 1. Connecting the two end points of the projection can determine a projection straight line segment, which is denoted as l i It can be understood that as long as the two endpoints of any control straight line segment are determined, the corresponding projection straight line segment can be obtained.

[0066] Further, in S103, based on the projected straight line segment, a matching feature straight line segment is searched in the current frame. In this embodiment, the matching feature straight line segment is the corresponding straight line segment. It should be noted that the corresponding straight line segment represents the projection position of the control straight line segment in the current frame.

[0067] By projecting each straight line segment l i , that is, control straight line segment L i Projection in the image, in the current frame all straight line segments {l image} to perform a neighborhood search to find the corresponding straight line segment that matches the projected straight line segment.

[0068] The corresponding straight line segment l′ of the neighborhood search i , the following conditions can be met:

[0069] Specific:

[0070] Condition 1: l i and l′ i The parallelism is within the preset threshold (20°);

[0071] Condition 2: l i and l′ i The distance dist(l i midpoint to l′ i The distance) is within the preset threshold (15% of the image width);

[0072] Condition 3: L′ i The two end points are projected vertically to l i or its extension, l′ i With l i There is overlap;

[0073] Condition 4: l i and l′ i The length ratio is within the preset threshold (0.67 <l i :l′ i <1.5)

[0074] Condition 5: If {l image If multiple straight line segments in} meet the above conditions, the one with the closest distance in condition 2 is selected.

[0075] Among them, condition 5 can be a further supplement to condition 2, that is, if there are multiple straight line segments l′ i If all of them meet condition 2, the one with the shortest distance can be selected. In addition, the contents in brackets in the above conditions are only examples of possible angles, distances or lengths, which may have different settings under different application conditions and are not intended to limit the present application.

[0076] In the prior art, when searching for strong gradient points along the gradient direction, noise points or interference points are introduced, so that the corresponding points searched by the control points are not accurate strong gradient points, resulting in wrong matching results. Figure 3 As shown, where e ij is the projection point, e′ ij1 , e′ij2 and e′ ij3 In this embodiment, the corresponding straight line segment is searched in the projection neighborhood of the control straight line segment. Usually, noise mostly appears in the form of points, and the probability of appearing in the form of line segments is low. Therefore, in this embodiment, noise interference and mismatch rate are further reduced.

[0077] In addition, since the prior art identifies target objects of arbitrary edge shapes by searching for corresponding points along the gradient direction, and arbitrary edge shapes do not have the characteristic of having an explicit analytical expression that is unique to straight line segments, searching for corresponding points along the gradient direction is an approximate means. In fact, the projection point of the control point in the current frame does not necessarily fall in the gradient direction, resulting in inaccurate calculated distances between the projection points and the corresponding points. In this embodiment, by searching straight line segments in the neighborhood, the calculated distance between the projected straight line segment and the corresponding straight line segment is more accurate, thereby reducing the mismatch rate.

[0078] Furthermore, Figure 4 A flow chart of another method for real-time posture tracking is provided in an embodiment of the present invention. To facilitate understanding of this embodiment, the letter expressions involved in this embodiment are first explained. [R|t] represents the posture of the target object in the previous frame, which has been obtained through assumptions in this application. [ΔR·R|t+Δt] represents the posture of the current frame, and [ΔR|Δt] represents the posture change of the current frame relative to the previous frame, that is, the incremental posture.

[0079] like Figure 4 As shown, according to S104, the position and posture of the object to be tracked is tracked in real time based on the matched characteristic straight line segments.

[0080] Specifically, real-time posture tracking can be achieved through the following steps, including:

[0081] S1041. If the number of feature straight line segments matching the projected straight line segment meets the set number, the incremental pose is obtained by minimizing the Euclidean distance between the projected straight line segment and the matching feature straight line segment.

[0082] If the number of projected straight line segments and corresponding straight line segments is greater than a certain value, for example, the number is ≥ 6 pairs, the distance between the new projected position and the corresponding straight line segment can be solved by solving Formula 2, so as to minimize the distance between the projected straight line segment and the corresponding straight line segment and solve the incremental pose. If the projected straight line segment and the corresponding straight line segment do not meet the set number, then end.

[0083] Formula 2:

[0084]

[0085] Where dist is Proj(L i,[ΔR·R|t+Δt]) to l′ i The distance, such as Figure 5 This equation can be solved by nonlinear optimization method or by linear equations after linear approximation of ΔR.

[0086] S1042: Calculate and update the position and posture parameters of the object to be tracked to achieve real-time tracking of the position and posture of the object to be tracked.

[0087] After obtaining the incremental pose [ΔR|Δt], substitute it into [ΔR·R|t+Δt] and obtain the current frame pose of the object to be tracked by incremental updating.

[0088] After obtaining the current frame pose, the current frame pose is passed to the next frame, and the current frame pose is used as the "previous frame pose" in this application to solve the new "current frame pose" (i.e., the next frame pose). This cycle is repeated to achieve real-time tracking of the pose of the target object.

[0089] Figure 6 This is a schematic diagram of an embodiment of a real-time posture tracking device provided in the present application, the device comprising:

[0090] The acquisition module 61 is used to acquire the current frame of the object to be tracked and extract the characteristic straight line segments in the current frame;

[0091] A first calculation module 62 is used to calculate the projection straight line segment of the control straight line segment in the previous frame through the control straight line segment of the three-dimensional model and the object posture obtained by solving the previous frame, where the projection straight line segment represents the projection position of the control straight line segment in the previous frame;

[0092] A search module 63, for searching for a matching feature straight line segment in the current frame based on the projected straight line segment, wherein the matching feature straight line segment represents a projection position of the control straight line segment in the current frame;

[0093] The second calculation module 64 calculates and updates the posture parameters of the object to be tracked based on the difference in projection position between the current frame and the previous frame, so as to achieve real-time tracking of the posture of the object to be tracked.

[0094] Figure 7 This is a schematic diagram of the structure of the real-time posture tracking device provided by the present application. As shown in FIG7 , the electronic device may include: at least one processor 71 and a memory 72 . Figure 7 An electronic device is shown using a processor as an example.

[0095] The memory 72 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0096] The memory 72 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0097] The processor 71 is used to execute the computer execution instructions stored in the memory 72 to implement the real-time posture tracking method;

[0098] Among them, the processor 71 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The processor 71 realizes real-time posture tracking by running the instructions stored in the memory 72.

[0099] Optionally, in a specific implementation, if the communication interface, the memory 72 and the processor 71 are implemented independently, the communication interface, the memory 72 and the processor 71 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0100] Optionally, in a specific implementation, if the communication interface, the memory 72 and the processor 71 are integrated on a chip, the communication interface, the memory 72 and the processor 71 can communicate through an internal interface.

[0101] The present application also provides a computer-readable storage medium, which may include: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program information, and the program information is used for the real-time posture tracking method.

[0102] An embodiment of the present application also provides a program, which, when executed by a processor, is used to execute the real-time posture tracking method provided in the above method embodiment.

[0103] An embodiment of the present application also provides a program product, such as a computer-readable storage medium, in which instructions are stored. When the program product is run on a computer, the computer executes the real-time posture tracking method provided in the above embodiment.

[0104] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time posture tracking method, characterized in that: include: Acquire a current frame of the object to be tracked, and extract characteristic straight line segments in the current frame; Determine the projection coordinates of the two end points of each control line segment through the control line segment of the 3D model and the object posture obtained by solving the previous frame; Determine, according to the projection coordinates of the two end points of each control straight line segment, a projected straight line segment of the control straight line segment in the previous frame, wherein the projected straight line segment represents the projection position of the control straight line segment in the previous frame; Based on the projected straight line segments, a neighborhood search is performed on each projected straight line segment in the feature straight line segments in the current frame to determine a matching feature straight line segment, wherein the matching feature straight line segment represents a projection position of the control straight line segment in the current frame; Based on the difference in projection position between the current frame and the previous frame, the posture parameters of the object to be tracked are solved and updated to achieve real-time tracking of the posture of the object to be tracked.

2. The method according to claim 1, characterized in that: The step of acquiring a current frame of the object to be tracked and extracting a characteristic straight line segment in the current frame includes: Acquire a current frame of the object to be tracked, and perform grayscale preprocessing on the current frame; Based on a straight line extraction algorithm, characteristic straight line segments in the grayscale current frame are extracted.

3. The method according to claim 1, characterized in that Based on the projected straight line segment, the neighborhood search is performed, and the projected straight line segment and the matching characteristic straight line segment meet the following conditions: The parallelism between the projected straight line segment and the characteristic straight line segment is within a preset parallelism threshold range; The distance between the projected straight line segment and the characteristic straight line segment is within a threshold range of a preset image width; When the two end points of the characteristic straight line segment are projected vertically onto the matching projection straight line segment or its extension line, the characteristic straight line segment overlaps with the projection straight line segment; The length ratio of the projected straight line segment to the characteristic straight line segment is within a preset threshold range.

4. The method according to any one of claims 1 to 3, characterized in that: The method of calculating and updating the posture parameters of the object to be tracked based on the difference in projection position between the current frame and the previous frame to achieve real-time tracking of the posture of the object to be tracked includes: If the number of characteristic straight line segments matching the projected straight line segment meets the set number, the incremental pose is obtained by minimizing the Euclidean distance between the projected straight line segment and the matching characteristic straight line segment; According to the incremental posture, the posture parameters of the object to be tracked are solved and updated to achieve real-time tracking of the posture of the object to be tracked.

5. A real-time posture tracking device, comprising: An acquisition module, used for acquiring a current frame of the object to be tracked, and extracting characteristic straight line segments in the current frame; A first calculation module is used to determine the projection coordinates of the two end points of each control straight line segment through the control straight line segments of the three-dimensional model and the object posture obtained by solving the previous frame; according to the projection coordinates of the two end points of each control straight line segment, determine the projection straight line segment of the control straight line segment in the previous frame, wherein the projection straight line segment represents the projection position of the control straight line segment in the previous frame; A search module is used to perform a neighborhood search for each projected straight line segment in the feature straight line segments in the current frame based on the projected straight line segments to determine a matching feature straight line segment, wherein the matching feature straight line segment represents a projection position of the control straight line segment in the current frame; The second calculation module calculates and updates the posture parameters of the object to be tracked based on the difference in projection position between the current frame and the previous frame, so as to achieve real-time tracking of the posture of the object to be tracked.

6. A real-time posture tracking device, comprising: Memory, processor; The memory is used to store executable instructions; The processor is configured to: run the instructions stored in the memory to execute the real-time posture tracking method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the real-time posture tracking method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, which, when executed by a processor, implements the real-time posture tracking method described in any one of claims 1 to 4.

Images