A method and device for positioning three-dimensional objects

Abstract

The present invention discloses a method for locating a three-dimensional object, comprising: using a single camera to capture an image of a three-dimensional object, identifying the posture and position of the three-dimensional object, wherein the outer surface of the three-dimensional object is divided into multiple regions, and the colors of adjacent regions are different; displaying a virtual object corresponding to the three-dimensional object; and after detecting that the posture and / or position of the three-dimensional object in three-dimensional space has changed, adjusting the virtual object according to the amount of change in the posture and / or position. Accordingly, an embodiment of the present invention provides a device for locating a three-dimensional object, which solves the problem in the prior art that it is impossible to adaptively adjust the changes of virtual objects according to the dynamic changes of three-dimensional objects in a single camera scenario.

Description

Technical Field

[0001] The present invention belongs to the technical field of augmented reality, and in particular, relates to a method and device for positioning a three-dimensional object. Background Art

[0002] In current augmented reality applications, three-dimensional objects can be captured by a camera, identified and displayed in a virtual scene. For example, an image of a hexahedron can be captured and a corresponding virtual object, such as a virtual hexahedron, a virtual character, a virtual globe, etc., can be displayed on a display screen.

[0003] In the prior art, virtual objects can be displayed by capturing three-dimensional objects, but the changes of virtual objects cannot be adaptively adjusted according to the dynamic changes of the three-dimensional objects in a single camera usage scenario. The cost is high and it is difficult to finely identify the position and posture of the three-dimensional object. Summary of the invention

[0004] The present invention provides a method and device for locating a three-dimensional object, which solves the problem in the prior art that it is impossible to adaptively adjust the changes of a virtual object according to the dynamic changes of the three-dimensional object in a single camera scene.

[0005] In order to achieve the above object, the present invention provides a method for positioning a three-dimensional object, comprising:

[0006] Using a single camera to capture an image of a three-dimensional object, the posture and position of the three-dimensional object are identified, wherein the outer surface of the three-dimensional object is divided into a plurality of regions, and the colors of adjacent regions are different;

[0007] displaying a virtual object corresponding to the three-dimensional object;

[0008] After detecting that the posture and / or position of the three-dimensional object in the three-dimensional space has changed, the virtual object is adjusted according to the amount of change in the posture and / or position.

[0009] In one embodiment, adjusting the virtual object includes:

[0010] Rotating the virtual object in a virtual space, wherein the rotation angle and angular velocity of the virtual object correspond to the amount of change in the posture of the three-dimensional object, or,

[0011] The position of the virtual object is moved in the virtual space, and the displacement of the virtual object corresponds to the position change of the three-dimensional object.

[0012] In one embodiment, after adjusting the virtual object, the method further includes:

[0013] When the camera captures different combinations of different color surfaces of the three-dimensional object, it enters different virtual scenes according to preset instructions, or,

[0014] When the rotational angular velocity of the virtual object exceeds a first preset threshold, a first virtual scene is displayed, or,

[0015] When the rotational angular velocity of the virtual object is lower than a second preset threshold, a second virtual scene is displayed, and the first preset threshold is greater than the second preset threshold.

[0016] In one embodiment, adjusting the virtual object according to the change in the posture and / or position includes:

[0017] The size of the virtual object is adjusted according to the depth of field distance between the three-dimensional object and the camera.

[0018] In one embodiment, identifying the posture and position of the three-dimensional object includes:

[0019] Performing color block segmentation on the image to decompose the image into a plurality of regions of different colors;

[0020] Taking the average value of the color of each of the regions, traversing all pairs of adjacent color blocks;

[0021] Using a table lookup method to screen the color block pairs, and screen out areas that match a preset model;

[0022] The orientation data of the matching area is calculated, and the position and posture of the three-dimensional object are obtained.

[0023] In one embodiment, after calculating the orientation data of the matching area, the method further includes:

[0024] Determine a candidate solution corresponding to the matching area;

[0025] Compare the compatibility of the candidate solutions pairwise, and discard any one of the two compatible candidate solutions;

[0026] Using an edge detection algorithm to identify edge pixels between the color block pairs;

[0027] The position and posture of the three-dimensional object are optimized using an optimization formula, which is:

[0028]

[0029] Where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q w ,q x ,q y ,qz ); f is a projection function used to calculate the point X on the surface of the three-dimensional object when the three-dimensional object is in posture P i The position of the projection position; E is the cost function, which calculates the difference between the projection position and the observation position; x i With θ i is the edge pixel detected in the image, x i is the coordinate of the edge point in the image, θ i is the tangent angle at the edge point.

[0030] In one embodiment, after performing color block segmentation on the image, the method further includes:

[0031] Building a model of the three-dimensional object;

[0032] Traversing adjacent faces in the model, and recording the color pairs and face orientations of the adjacent faces;

[0033] Record the coordinate information of the boundary lines of all adjacent faces.

[0034] The embodiment of the present invention further provides a method for locating a three-dimensional object, comprising:

[0035] Using a single camera to capture an image of a three-dimensional object, the outer surface of the three-dimensional object is divided into a plurality of regions, and the colors of adjacent regions are different;

[0036] Recording two or more adjacent faces of the three-dimensional object;

[0037] Performing color block segmentation on the image to decompose the image into a plurality of regions of different colors;

[0038] Taking the average value of the color of each of the regions, traversing all pairs of adjacent color blocks;

[0039] Using a table lookup method to screen the color block pairs, and screen out areas that match a preset model;

[0040] Calculating the orientation data of the matching area and obtaining the position and posture of the three-dimensional object;

[0041] A virtual object corresponding to the three-dimensional object is displayed.

[0042] In one embodiment, after calculating the orientation data of the matching area, the method further includes:

[0043] Determine a candidate solution corresponding to the matching area;

[0044] Compare the compatibility of the candidate solutions pairwise, and discard any one of the two compatible candidate solutions;

[0045] Using an edge detection algorithm to identify edge pixels between the color block pairs;

[0046] The position and posture of the three-dimensional object are optimized using an optimization formula, which is:

[0047]

[0048] Where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q w ,q x ,q y ,q z ); f is a projection function used to calculate the point X on the surface of the three-dimensional object when the three-dimensional object is in posture P i The position of the projection position; E is the cost function, which calculates the difference between the projection position and the observation position; x i With θ i is the edge pixel detected in the image, x i is the coordinate of the edge point in the image, θ i is the tangent angle at the edge point.

[0049] An embodiment of the present invention also provides a three-dimensional object positioning device, which includes a processor and a memory for storing a computer program that can be run on the processor; wherein the processor is used to execute the above-mentioned three-dimensional object positioning method when running the computer program.

[0050] An embodiment of the present invention further provides a computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions being used to execute the above-mentioned method for locating a three-dimensional object.

[0051] An embodiment of the present invention provides a method and device for locating a three-dimensional object. The method uses a single camera to identify different color combination areas of a three-dimensional object, determines the spatial posture and position of the three-dimensional object, and displays the corresponding virtual object. At the same time, the spatial posture and position of the three-dimensional object can be determined in detail. In addition, the depth of field of the three-dimensional object can be measured by a single camera, with low cost and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of a three-dimensional object positioning method according to an embodiment of the present invention;

[0053] Figure 2 is a flow chart of a method for identifying the posture and position of a three-dimensional object in an embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of three-dimensional object recognition in an embodiment of the present invention;

[0055] Figure 4 is another schematic diagram of three-dimensional object recognition in an embodiment of the present invention;

[0056] Figure 5 It is a schematic diagram of the principle of optimizing the posture of a three-dimensional object color block edge in an embodiment of the present invention;

[0057] Figure 6 is a schematic structural diagram of a three-dimensional object positioning device in an embodiment of the present invention;

[0058] Figure 7 It is a schematic diagram of the structure of a three-dimensional object positioning device in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0060] To achieve the above objectives, Figure 1 As shown, an embodiment of the present invention provides a method for recommending an application program, the method comprising:

[0061] S101, using a single camera to capture an image of a three-dimensional object, and identifying the posture and position of the three-dimensional object, wherein the outer surface of the three-dimensional object is divided into a plurality of regions, and the colors of adjacent regions are different;

[0062] In an embodiment of the present invention, the three-dimensional object positioning and virtual object display can be realized by a three-dimensional object positioning device. Optionally, the positioning device includes a single camera, a processing unit and a display unit. The image of the three-dimensional object is collected in the detection area by the single camera, and the image processing is realized by the processing unit and the posture and position of the three-dimensional object are finally determined, and a virtual object identical to or corresponding to the three-dimensional object is created in the virtual scene with the help of augmented reality AR technology. The three-dimensional object can be a polyhedron, such as a sphere, a cylinder, a tetrahedron, a hexahedron, etc. Among them, the color of each face can be different. In an embodiment of the present invention, during the rotation or movement of the three-dimensional object, the direction and posture of the three-dimensional object facing the camera can be determined by the color combination of each face. For the convenience of explanation, the embodiments of the present invention are all explained by taking a hexahedron as an example, and the remaining three-dimensional objects (with different color face combinations) are also within the protection scope of the embodiments of the present invention.

[0063] like Figure 2 As shown, in one embodiment, the posture and position of the three-dimensional object are identified, specifically, as follows:

[0064] S201, performing color block segmentation on an image to decompose the image into a plurality of regions of different colors;

[0065] like Figure 3 As shown, the image may be a yellow color area, a green color area, a blue color area, a black color area, a white color area, and a red color area.

[0066] In addition, after the color block segmentation, the model of the three-dimensional object can be established; the adjacent faces in the model can be traversed to record the color pairs and face orientations of the adjacent faces; and the coordinate information of the boundary lines of all adjacent faces can be recorded. That is, in the embodiment of the present invention, the adjacent faces in the three-dimensional object can be traversed to record the color pairs (such as "green-red", "red-yellow", etc.) and face orientations of the adjacent faces, and a table can be built for use by subsequent algorithms; at the same time, the coordinate information of the boundary lines of all adjacent faces can be recorded (represented by a series of discrete sampling points) for subsequent precise measurement.

[0067] When the camera can see two or more faces at the same time, it can use the color combination to preliminarily determine the position and posture (i.e., orientation) of the marker, and then use an iterative algorithm to obtain more accurate position and posture data.

[0068] S202, taking an average value of the color of each of the regions, and traversing all pairs of adjacent color blocks;

[0069] S203, using a table lookup method to screen the color block pairs to select areas that match a preset model;

[0070] Use the table lookup method to screen the color block pairs. For example, "red-white" is consistent / matched with the three-dimensional object model, but "red-blue" and "green-purple" do not match the model (the former is because red and blue are not adjacent in the model, and the latter is because there is no purple in the model). Discard the inconsistent color block pairs and screen out the areas (color block pairs) that match the preset model.

[0071] S204: Calculate the orientation data of the matching area, and obtain the position and posture of the three-dimensional object.

[0072] After screening, some candidate color block pairs will be left, such as Figure 4 You will get "red-white", "red-black", "black-white", etc. You can use the table lookup method to find out the orientation data of these faces, so as to know the approximate orientation of the camera relative to the marker; at the same time, since there are data from two faces at the same time, the approximate rotation angle of the camera can also be calculated (the rotation angle with the line connecting the camera and the marker as the axis), and then the approximate position and posture of the three-dimensional object can be inferred.

[0073] In the above embodiment, the approximate position and posture of the three-dimensional object can be estimated, and the result obtained may not be accurate. In the embodiment of the present invention, the position and posture can be further optimized to obtain the posture and position of the three-dimensional object in space in a refined manner. The specific method is as follows:

[0074] After calculating the orientation data of the matching area, the embodiment of the present invention further includes:

[0075] S2041, calculating candidate solutions corresponding to the matching area;

[0076] S2042, comparing the compatibility of the candidate solutions in pairs, and discarding any one of the two compatible candidate solutions;

[0077] When the positions and postures of the markers obtained by two candidate solutions are close (the distance and angle differences are less than a certain threshold), the two candidate solutions can be considered compatible, that is, they are both color block pairs on the same marker. At this time, one of the solutions can be discarded.

[0078] S2043, using an edge detection algorithm to identify edge pixels between the color block pairs;

[0079] S2044, using an optimization formula to optimize the position and posture of the three-dimensional object, the optimization formula is:

[0080]

[0081] Where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q w ,q x ,q y ,q z ); f is a projection function used to calculate the point X on the surface of the three-dimensional object when the three-dimensional object is in posture P i The position of the projection position; E is the cost function, which calculates the difference between the projection position and the observation position; x i With θ i is the edge pixel detected in the image, x i is the coordinate of the edge point in the image, θ i is the tangent angle of the edge point. After iterative optimization of the above formula using the Levenberg-Marquardt algorithm, a better P can be obtained. * The value is the optimized position and posture of the marker.

[0082] After completing the posture optimization through the iterative algorithm, the accurate posture data of the three-dimensional object can be obtained.

[0083] In addition, for video data, Kalman filtering can be used to smooth the postures of consecutive frames, and finally a relatively stable posture sequence can be obtained.

[0084] S102, displaying a virtual object corresponding to the three-dimensional object;

[0085] In the embodiment of the present invention, after the posture and position of the three-dimensional object are identified, the virtual object corresponding to the three-dimensional object can be displayed in the virtual scene by means of AR technology. For example, a hexahedron with the same size and shape as the three-dimensional object is displayed, and the color of each face is consistent with the actual three-dimensional object.

[0086] S103: After detecting that the posture and / or position of the three-dimensional object in the three-dimensional space has changed, adjust the virtual object according to the amount of change in the posture and / or position.

[0087] In one embodiment, the adjusting the virtual object may specifically be:

[0088] Rotating the virtual object in a virtual space, wherein the rotation angle and angular velocity of the virtual object correspond to the amount of change in the posture of the three-dimensional object, or,

[0089] The position of the virtual object is moved in the virtual space, and the displacement of the virtual object corresponds to the position change of the three-dimensional object.

[0090] In one embodiment, after adjusting the virtual object, the method further includes:

[0091] When the camera captures different combinations of different color surfaces of the three-dimensional object, it enters different virtual scenes according to preset instructions, or,

[0092] When the rotational angular velocity of the virtual object exceeds a first preset threshold, a first virtual scene is displayed, or,

[0093] When the rotational angular velocity of the virtual object is lower than a second preset threshold, a second virtual scene is displayed, and the first preset threshold is greater than the second preset threshold.

[0094] In one embodiment, adjusting the virtual object according to the change in the posture and / or position includes:

[0095] The size of the virtual object is adjusted according to the depth of field distance between the three-dimensional object and the camera.

[0096] For example, in the field of games, the three-dimensional object can be made into a polyhedral shape of a "magic wand / magic staff". Through the above three-dimensional positioning method, the combination of different face colors can be identified to determine which specific posture is currently facing the camera. While the user operates the magic wand, the virtual magic wand can also change adaptively. For example, when the user rotates the magic wand, the virtual magic wand also rotates synchronously. When the user moves the magic wand, the virtual magic wand also moves synchronously. At the same time, the depth of field of the actual magic wand relative to the camera can be measured by a single camera according to the distance. The size of the virtual magic wand varies with the depth of field. Users can use this feature to have different game experiences. For example, different magic wands are operated in a three-dimensional space. The game effects produced by being closer to the camera and farther away from the camera are different, which can improve the user's game experience and reduce the manufacturing cost (TOF cameras or multi-camera solutions are used on the market to measure the depth of field, which is expensive).

[0097] At the same time, users can also design different gaming experiences based on the different characteristics of the magic wand. For example, by rotating the magic wand and turning it to different sides in front of the camera (i.e., different color combinations captured by the camera are different), new interactive experiences can be designed. For example, turning to side A will enter a certain game, and turning to side B will enter another game. When turning from side A to side B, an interactive virtual scene will be generated, and when turning from side B to side C, another scene will be generated. At the same time, different games can also be designed when the rotation speed and angle are different, that is, different angles and different speeds are used to rotate, different sides correspond to different games, or the corresponding game play is different.

[0098] An embodiment of the present invention provides a method and device for locating a three-dimensional object. The method uses a single camera to identify different color combination areas of a three-dimensional object, determines the spatial posture and position of the three-dimensional object, and displays the corresponding virtual object. At the same time, the spatial posture and position of the three-dimensional object can be determined in detail. In addition, the depth of field of the three-dimensional object can be measured by a single camera, with low cost and high accuracy.

[0099] In addition, the embodiment of the present invention also provides a method for locating a three-dimensional object. The positioning algorithm pre-enters the marked information, including the coordinates of all vertices, the parametric equations of the edges, and the colors of the faces. There are the following points when entering the model:

[0100] 1. Traverse the adjacent faces in the model, record the color pairs of the adjacent faces (such as "green-red", "red-yellow", etc.) and the orientation of the faces, and build a table for subsequent algorithm use;

[0101] 2. Record the coordinate information of the boundary lines of all adjacent faces (represented by a series of discrete sampling points),

[0102] For subsequent accurate measurements.

[0103] When the camera can see two or more faces at the same time, it can use the color combination to preliminarily determine the position and posture (that is, the orientation) of the marker, and then use an iterative algorithm to obtain more accurate position and posture data.

[0104] The method specifically includes:

[0105] S501, using a single camera to collect an image of a three-dimensional object, wherein the outer surface of the three-dimensional object is divided into a plurality of regions, and colors of adjacent regions are different;

[0106] S502, recording two or more adjacent faces of the three-dimensional object;

[0107] S503, performing color block segmentation on the image, decomposing the image into a plurality of regions of different colors;

[0108] S504, taking an average value of the color of each of the regions, and traversing all pairs of adjacent color blocks;

[0109] For example, "red-white" is consistent with the model, but "red-blue" and "green-purple" are not consistent with the model (the former is because red and blue are not adjacent in the model, and the latter is because there is no purple in the model).

[0110] S505, using a table lookup method to screen the color block pairs to select areas that match a preset model;

[0111] for example Figure 4 You will get "red-white", "red-black", "black-white", etc. You can use the table lookup method to find out the orientation data of these faces, so as to know the approximate orientation of the camera relative to the marker; at the same time, since there are data from two faces at the same time, the approximate rotation angle of the camera can also be calculated (the rotation angle with the line connecting the camera and the marker as the axis), and then the approximate position and posture of the marker can be inferred.

[0112] S506, calculating the orientation data of the matching area, and obtaining the position and posture of the three-dimensional object;

[0113] When the positions and postures of the markers obtained by two candidate solutions are close (the distance and angle differences are less than a certain threshold), the two candidate solutions can be considered compatible, that is, they are both color block pairs on the same marker. At this time, one of the solutions can be discarded.

[0114] S507: Display a virtual object corresponding to the three-dimensional object.

[0115] In one embodiment, after calculating the orientation data of the matching area, the method further includes:

[0116] S5061, calculating candidate solutions corresponding to the matching area;

[0117] S5062, comparing the compatibility of the candidate solutions in pairs, and discarding any one of the two compatible candidate solutions;

[0118] S5063, using an edge detection algorithm to identify edge pixels between the color block pairs;

[0119] S5064: Use an optimization formula to optimize the position and posture of the three-dimensional object. The optimization formula is:

[0120]

[0121] Where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q w ,q x ,q y ,q z ), where the attitude angle is expressed in quaternion; f is the projection function used to calculate the point X on the marker when it is in attitude P. i Where is it located on the image captured by the camera? E is the cost function, which calculates the difference between the projection position and the observation position. The greater the difference, the greater the cost. i With θ i is the edge pixel detected in the image, x i is the coordinate of the edge point in the image, θ i is the tangent angle of the edge point. After iterative optimization of the above formula using the Levenberg-Marquardt algorithm, a better P can be obtained. * The value is the optimized position and posture of the marker.

[0122] In addition, for video data, Kalman filtering can be used to smooth the postures of consecutive frames, and finally a relatively stable posture sequence can be obtained.

[0123] Figure 5 : is a schematic diagram of the principle of optimizing the posture of a three-dimensional object color block edge in an embodiment of the present invention, wherein the dotted line is the color block edge inferred from the current estimated posture, and the solid line is the color block edge actually captured. During optimization, the dotted line is brought closer to the solid line by adjusting the posture.

[0124] In addition, in an embodiment of the present invention, it can also be combined with a remote control module to perform human-computer interaction on virtual objects. The traditional remote control has a patterned operation instruction set. Pressing a button generates a command signal and transmits the signal so that terminals such as televisions and air conditioners respond to the instruction. In an embodiment of the present invention, a series of operation protocols can be set up in a targeted manner to combine remote control instructions with the spatial posture / position of three-dimensional objects to form a new set of human-computer interaction protocols. For example, a series of instruction sets can be set in the remote control module, and the interactions generated in different spatial postures / positions can be different. After pressing a button to move to the right, if the position of the three-dimensional object is not detected to be updated, the virtual object moves one grid to the right. If the position of the three-dimensional object is detected to be updated, it is also possible to press a button to move to the right to move three grids.

[0125] Figure 6 is a schematic diagram of a three-dimensional positioning device in an embodiment of the present invention; a user can operate an application on a device such as a mobile phone, a tablet or a computer. The device may include information input devices such as a camera, a microphone, application props and / or an AR device. The device may also include output devices such as a display device and a speaker.

[0126] An embodiment of the present invention further provides a storage medium on which computer instructions are stored. When the instructions are executed by a processor, the above-mentioned method for positioning a three-dimensional object is implemented.

[0127] Figure 7 It is a schematic diagram of a system structure provided by an embodiment of the present invention. The system may include one or more central processing units (CPU) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) for storing application programs or data. Among them, the memory 620 and the storage medium 630 can be short-term storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the device. Furthermore, the central processing unit 610 can be configured to communicate with the storage medium 630 and execute a series of instruction operations in the storage medium 630 on the system. The system may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, and one or more input and output interfaces 660. The steps performed by the above-mentioned three-dimensional positioning method embodiment may be based on this. Figure 7 The system structure shown.

[0128] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] Those of ordinary skill in the art will appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0130] Each part of this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0131] Finally, it should be noted that the above are only preferred embodiments of the technical solution of the present application and are not intended to limit the scope of protection of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. If these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, any modifications, equivalent substitutions, improvements, etc. made shall be included in the scope of protection of the present application.

Claims

1. A method for locating a three-dimensional object, characterized in that: include: Using a single camera to capture an image of a three-dimensional object, the posture and position of the three-dimensional object are identified, wherein the outer surface of the three-dimensional object is divided into a plurality of regions, and the colors of adjacent regions are different; Displaying a virtual object corresponding to the three-dimensional object; and adjusting the virtual object according to the amount of change in the posture and / or position after detecting that the posture and / or position of the three-dimensional object in the three-dimensional space has changed; Receive input instructions and generate instruction codes; Adjusting the virtual object according to the instruction code, the change in the posture and / or the position; Wherein, the adjusting of the virtual object comprises: rotating the virtual object in the virtual space, the rotation angle and angular velocity of the virtual object corresponding to the amount of change in the posture of the three-dimensional object, or moving the position of the virtual object in the virtual space, the displacement of the virtual object corresponding to the amount of change in the position of the three-dimensional object; Or, the adjusting the virtual object according to the change in the posture and / or position includes: adjusting the size of the virtual object according to the depth of field distance between the three-dimensional object and the camera; Among them, identifying the posture and position of the three-dimensional object includes: performing color block segmentation on the image to decompose the image into multiple regions of different colors; taking the average value of the color of each region and traversing all adjacent color block pairs; using a table lookup method to filter the color block pairs to filter out regions that match a preset model; calculating the orientation data of the matching region and obtaining the position and posture of the three-dimensional object; After calculating the orientation data of the matching area, the method further includes: Determine a candidate solution corresponding to the matching area; Compare the compatibility of the candidate solutions pairwise, and discard any one of the two compatible candidate solutions; Using an edge detection algorithm to identify edge pixels between the color block pairs; The position and posture of the three-dimensional object are optimized using an optimization formula, which is: in It is the position and posture parameters of the three-dimensional object, including position coordinates and attitude angle ; Projection function, used to calculate the position of a three-dimensional object Points on the surface of the three-dimensional object at the time of posture location; is the cost function, calculating the difference between the projection position and the observed position; and is the edge pixel detected in the image, are the coordinates of the edge points in the image, is the tangent angle at the edge point.

2. The method according to claim 1, characterized in that After adjusting the virtual object, it also includes: when the camera captures different combinations of different color surfaces of the three-dimensional object, entering different virtual scenes according to preset instructions, or, when the rotation angular velocity of the virtual object exceeds a first preset threshold, displaying a first virtual scene, or, when the rotation angular velocity of the virtual object is lower than a second preset threshold, displaying a second virtual scene, and the first preset threshold is greater than the second preset threshold.

3. A three-dimensional object positioning device, characterized in that: The device comprises a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, it executes the method for locating a three-dimensional object as described in claim 1 or 2.

Images