Positioning method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a positioning method and device, an electronic device and a storage medium. The method comprises: acquiring multiple images of a target scene in real time; establishing a three-dimensional map based on the target scene according to a preset marker in the multiple images; determining first position information of an optical center of an image acquisition device when each image is acquired according to the three-dimensional map and the multiple images; and determining a relative position relationship between the optical center and a rotation axis of a turntable according to the first position information. According to the positioning method of the embodiment of the present disclosure, the three-dimensional map of the target scene can be obtained through the images taken by the image acquisition device during rotation, and then the position of the optical center in the three-dimensional map is determined to determine the target position information of the rotation axis of the turntable in the three-dimensional map, so that the relative position relationship between the optical center and the rotation axis of the turntable can be obtained to improve the installation accuracy and then improve the image stitching accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to a positioning method and apparatus, electronic device and storage medium. Background Technology

[0002] Image stitching technology is a multidisciplinary processing technique integrating computer vision, image processing, and other fields. It is primarily used to acquire full-view images and plays a vital role in various fields such as minimally invasive medical surgery, geological surveying, aerospace, and security monitoring, with a very broad application prospect. Image acquisition methods can be divided into turntable shooting, airborne platform shooting, and handheld device shooting, among which turntable shooting offers better image quality and stability.

[0003] Image stitching can be divided into image registration and image fusion stages. The accuracy of image registration determines the quality of image stitching. Image registration methods can be categorized into pixel-based, transform domain-based, and feature-based methods. Among these, feature-based registration is the most widely used due to its high accuracy and robustness. Feature-based methods identify matching features between two images, such as feature points, feature regions, and feature edges. Feature points are the most commonly used, consisting of keypoints and descriptors. Well-known point features include SIFT (Scale-invariant feature transform), SURF (Speeded Up Robust Features), and ORB (Oriented Fast and Rotated Brief). However, image stitching based on feature point matching requires the following assumptions: the overlapping area of ​​the images (i.e., the feature points in the two images are on the same plane), or the camera's optical center position does not change, meaning the camera's motion is a pure rotational motion.

[0004] Due to the diversity of environmental scenarios, the overlapping areas of images are often not on the same plane in the world coordinate system. Multi-region image stitching places higher demands on the alignment accuracy of image stitching. Therefore, it is necessary to ensure that the camera movement is a pure rotational motion as much as possible. However, in actual installation, it is difficult to determine the positional relationship between the optical center of the camera and the rotation axis of the turntable, making it difficult to guarantee the installation accuracy of the camera and to achieve a pure rotational motion during rotation. Summary of the Invention

[0005] This disclosure presents a positioning method, apparatus, electronic device, and storage medium.

[0006] According to one aspect of this disclosure, a positioning method is provided, comprising: acquiring multiple frames of images of a target scene in real time, wherein a turntable is provided in the target scene, an image acquisition device is provided on the turntable, the multiple frames of images are acquired by the image acquisition device as it rotates with the turntable, and preset markers are provided at multiple locations in the target scene; establishing a three-dimensional map based on the target scene based on the preset markers captured in the multiple frames of images; determining, based on the three-dimensional map and the multiple frames of images, a first position information of the optical center of the image acquisition device in the three-dimensional map when acquiring each frame of images; and determining the relative positional relationship between the optical center and the rotation axis of the turntable based on the first position information.

[0007] In one possible implementation, establishing a three-dimensional map based on the target scene according to the preset markers captured in the multiple frames of images includes: determining a first positional relationship between the preset markers according to the second positional information of the preset markers in each frame of images; and obtaining the three-dimensional map according to the first positional relationship.

[0008] In one possible implementation, during the rotation, the image acquisition device passes the same position at least twice, and obtains the three-dimensional map based on the first positional relationship, including: determining the third position information of each preset marker in the three-dimensional map based on the first positional relationship; obtaining the three-dimensional relationship diagram based on the third position information; and correcting the three-dimensional relationship diagram based on at least two frames of images acquired by the image acquisition device at the same position and the first positional relationship to obtain the three-dimensional map.

[0009] In one possible implementation, determining the first position information of the optical center of the image acquisition device in the three-dimensional map when acquiring each frame of the image, based on the three-dimensional map and the multiple frames of images, includes: determining a second positional relationship between a fourth positional information of a preset marker in each frame of the image and the position of the optical center of the image acquisition device, based on preset intrinsic parameters of the image acquisition device; and determining the first position information of the optical center in the three-dimensional map based on the position of the preset marker in the three-dimensional map and the second positional relationship.

[0010] In one possible implementation, determining the relative positional relationship between the optical center and the rotation axis of the turntable based on the first position information includes: fitting the first position information of the optical center when acquiring the plurality of images to obtain target position information of the rotation axis; and determining the relative positional relationship between the first position information and the target position information.

[0011] In one possible implementation, the method further includes: adjusting the installation position of the image acquisition device according to the relative positional relationship, so that the optical center of the image acquisition device coincides with the rotation axis.

[0012] In one possible implementation, the method further includes: correcting the image acquired by the image acquisition device according to the relative positional relationship; or determining a third positional relationship between the target surface of the image acquisition device and the rotation axis according to the relative positional information, and correcting the image acquired by the image acquisition device according to the third positional relationship.

[0013] According to one aspect of this disclosure, a positioning device is provided, comprising: an image acquisition module, configured to acquire multiple frames of images of a target scene in real time, wherein a turntable is provided in the target scene, an image acquisition device is provided on the turntable, the multiple frames of images are acquired by the image acquisition device as it rotates with the turntable, and preset markers are provided at multiple locations in the target scene; a map acquisition module, configured to establish a three-dimensional map based on the target scene according to the preset markers captured in the multiple frames of images; an optical center determination module, configured to determine, according to the three-dimensional map and the multiple frames of images, a first position information of the optical center of the image acquisition device in the three-dimensional map when acquiring each frame of images; and a position determination module, configured to determine the relative positional relationship between the optical center and the rotation axis of the turntable according to the first position information.

[0014] In one possible implementation, the map acquisition module is further configured to: determine a first positional relationship between the preset markers based on the second positional information of the preset markers in each frame image; and obtain the three-dimensional map based on the first positional relationship.

[0015] In one possible implementation, during the rotation, the image acquisition device passes the same position at least twice, and the map acquisition module is further configured to: determine the third position information of each of the preset markers in the three-dimensional map according to the first position relationship; obtain the three-dimensional relationship diagram according to the third position information; and correct the three-dimensional relationship diagram according to the at least two frames of images acquired by the image acquisition device at the same position and the first position relationship to obtain the three-dimensional map.

[0016] In one possible implementation, the optical center determination module is further configured to: determine a second positional relationship between the fourth position information of the preset marker in each frame image and the position of the optical center of the image acquisition device according to the preset intrinsic parameters of the image acquisition device; and determine the first position information of the optical center in the three-dimensional map according to the position of the preset marker in the three-dimensional map and the second positional relationship.

[0017] In one possible implementation, the position determination module is further configured to: fit the first position information of the optical center when acquiring the plurality of images to obtain the target position information of the rotation axis; and determine the relative positional relationship between the first position information and the target position information.

[0018] In one possible implementation, the device further includes an adjustment module for adjusting the installation position of the image acquisition device according to the relative positional relationship, such that the optical center of the image acquisition device coincides with the rotation axis.

[0019] In one possible implementation, the device further includes a correction module for correcting the image acquired by the image acquisition device according to the relative positional relationship; or for determining a third positional relationship between the target surface of the image acquisition device and the rotation axis according to the relative positional information, and correcting the image acquired by the image acquisition device according to the third positional relationship.

[0020] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above-described positioning method.

[0021] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described positioning method.

[0022] According to the positioning method of the embodiments of this disclosure, a three-dimensional map can be obtained by acquiring multiple frames of images during the rotation of the image acquisition device. The error of the third position information with the same preset mark is used to correct the three-dimensional map, thereby obtaining an accurate three-dimensional map. Then, the position of the optical center in the three-dimensional map is determined, so as to determine the target position information of the rotation axis of the turntable in the three-dimensional map. This allows the relative positional relationship between the optical center and the rotation axis of the turntable to be obtained, thereby improving the installation accuracy and thus improving the image stitching accuracy.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0024] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0026] Figure 1 A flowchart illustrating a positioning method according to an embodiment of the present disclosure is shown;

[0027] Figure 2 A schematic diagram illustrating the correction process according to an embodiment of the present disclosure is shown;

[0028] Figure 3A and Figure 3B This diagram illustrates an application of the positioning method according to an embodiment of the present disclosure.

[0029] Figure 4 A block diagram of a positioning device according to an embodiment of the present disclosure is shown;

[0030] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0031] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0032] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0033] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0034] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0035] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0036] Figure 1 A flowchart illustrating a positioning method according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the method includes:

[0037] In step S11, multiple frames of images of the target scene are acquired in real time. A turntable is set in the target scene, and an image acquisition device is set on the turntable. The multiple frames of images are acquired by the image acquisition device as the turntable rotates. Preset marks are set at multiple locations in the target scene.

[0038] In step S12, a three-dimensional map based on the target scene is established according to the preset markers captured in the multi-frame images;

[0039] In step S13, based on the three-dimensional map and the multiple frames of images, the first position information of the optical center of the image acquisition device in the three-dimensional map is determined when acquiring each frame of images;

[0040] In step S14, the relative positional relationship between the optical center and the rotation axis is determined based on the first position information.

[0041] According to the positioning method of the embodiments of this disclosure, a three-dimensional map of the target scene can be established by taking images captured by the image acquisition device during rotation, thereby determining the position of the optical center in the three-dimensional map, and determining the target position information of the rotation axis of the turntable in the three-dimensional map. This allows the relative positional relationship between the optical center and the rotation axis of the turntable to be determined, improving installation accuracy and thus improving image stitching accuracy.

[0042] In one possible implementation, to obtain a 3D map of the target scene, multiple preset markers can be set in the target scene. During the movement of the image acquisition device, images of the target scene are acquired. The images may include the captured preset markers. Then, based on the positional relationship between the preset markers in each image, the positional relationship of each preset marker in the 3D map can be determined, thereby establishing a 3D map based on the preset markers.

[0043] In one possible implementation, in step S11, a turntable can be set in the target scene, i.e., a device for supporting and rotating the image acquisition device so that the image acquisition device can acquire images during rotation. Preset marks can be set around the turntable so that the image acquisition device can capture the preset marks during rotation and image capture to determine the positional relationship of each mark in the image, and thus determine the positional relationship of multiple marks in three-dimensional space (e.g., the three-dimensional map to be built).

[0044] In one possible implementation, the image acquisition device can be mounted on the aforementioned turntable, and as the turntable rotates, the image acquisition device can acquire multiple frames of images of the surrounding area. During the rotation, the image acquisition device can pass through the same position at least twice, for example, it can rotate more than one revolution, so as to correct the established 3D map using images taken twice at the same position.

[0045] In one possible implementation, in step S12, a three-dimensional map of the target scene can be established based on multiple frames of images captured by the image acquisition device during rotation. In the example, the preset markers can use Aruco identification codes with a diameter of 35mm. Each Aruco identification code is unique, which facilitates determining the positional relationship between the preset markers in the image, and thus determining the positional relationship of each preset marker in three-dimensional space, thereby establishing a three-dimensional map based on the preset markers.

[0046] In one possible implementation, step S12 may include: determining a first positional relationship between the preset markers based on the second positional information of the preset markers in each frame image; and obtaining the three-dimensional map based on the first positional relationship.

[0047] In the example, a 3D map can be created through methods such as synchronous positioning and mapping. For instance, the positional relationships between preset markers can be determined in images acquired by an image acquisition device. For example, in the first image, the second positional information of preset marker 1, preset marker 2, and preset marker 3 can be determined, and their positional relationships can be established. For example, preset marker 1 is at the far left of the three preset markers, preset marker 2 is 3cm to the right of preset marker 1 (both are at the same height), and preset marker 3 is 3cm to the right of preset marker 2 (both are at the same height). In the second image, the second positional information of preset marker 2, preset marker 3, and preset marker 4 can be determined, and their positional relationships can be established. For example, preset marker 3 is 3cm to the right of preset marker 2 (both are at the same height), and preset marker 4 is 2cm to the right of preset marker 3 (both are at the same height), but the angle between the orientation of preset marker 4 and the orientation of preset marker 3 is 90 degrees. From the above information, it can be seen that preset marker 1, preset marker 2, preset marker 3, and preset marker 4 are at the same height, and the angle between the orientation of preset marker 4 and the other three preset markers is 90 degrees. Furthermore, the distances between the four preset markers can be determined, thus establishing the initial positional relationship of the four preset markers in three-dimensional space (i.e., the three-dimensional map). Using the above method, the positional relationship between all preset markers captured by the image acquisition device during rotation can also be further determined. This disclosure does not limit the number of preset markers or the positional relationship between them.

[0048] In the example, the aforementioned preset markers can be used as points in a 3D map. After determining the positional relationships of all preset markers in the 3D map, the position of each preset marker in the 3D map can be determined, thereby establishing the 3D map. For example, during the 3D map establishment process, a 3D coordinate system can be established based on the first frame image acquired by the image acquisition device (e.g., the captured image can be used for 3D modeling, and the first frame image is the initialization image used during modeling). In the 3D coordinate system, the XY plane can be set as the imaging plane, and the image acquisition device rotates around the Y-axis, with its rotation trajectory within the XZ plane.

[0049] In one possible implementation, errors may occur during the determination of the first positional relationship, such as distance or angle errors. These errors can accumulate as the number of preset markers increases. For example, when inspecting an image acquired by an image acquisition device, it may be determined that preset marker 2 is 3cm to the right of preset marker 1, and preset marker 3 is 3cm to the right of preset marker 2. However, in reality, preset marker 2 is 2.5cm to the right of preset marker 1, and preset marker 3 is 2.5cm to the right of preset marker 2. The distance measurement errors from the two determinations of positional relationship will accumulate; for example, two 0.5cm errors can accumulate to 1cm. This accumulation of errors can result in significant errors in the positional relationships of the preset markers in the 3D map, leading to a larger overall error in the constructed 3D map.

[0050] In one possible implementation, to address the aforementioned problem, the error can be corrected by taking images at least twice from the same location using an image acquisition device. During the rotation of the image acquisition device, if it rotates more than one revolution, it can pass through the same location twice (e.g., twice passing through the starting position of the rotation). For example, the first frame image is acquired at this location, and the nth frame image (where n is a positive integer greater than 1) is also acquired at this location. Furthermore, based on the first, second...n-1th frames and the nth frame image, the positional relationship between preset markers in each image can be determined, thereby determining the position of each preset marker in three-dimensional space. The preset markers in the nth frame image are the same as those in the first frame image. If the positions of the same preset markers are different, it indicates that an error has accumulated in determining the positional relationship. Furthermore, the error can be corrected by the positional error between the same preset markers, i.e., eliminating the accumulated error and making the established three-dimensional map more accurate.

[0051] In one possible implementation, obtaining the three-dimensional map based on the first positional relationship includes: determining third positional information of each of the preset markers in the three-dimensional map based on the first positional relationship; obtaining the three-dimensional relationship diagram based on the third positional information; and correcting the three-dimensional relationship diagram based on at least two frames of images acquired by the image acquisition device at the same location and the first positional relationship to obtain the three-dimensional map.

[0052] Figure 2 A schematic diagram illustrating the correction process according to an embodiment of the present disclosure is shown. Figure 2 As shown, by determining the positional relationship between each preset marker, the position of each preset marker in the 3D map can be determined, that is, the third positional information (i.e., Figure 2 (Position before loop closure detection and correction). For example, the first preset marker in the first frame image can be used as a reference point. By determining the positional relationship between other preset markers and the reference point, the third position information of the other preset markers in the 3D map can be determined. Based on the above third position information, a 3D relationship diagram of each preset marker can be determined, that is, the uncorrected 3D map.

[0053] In one possible implementation, the 3D relationship diagram can be corrected based on at least two images acquired by the image acquisition device at the same location and a first positional relationship to obtain a 3D map. In the example, the preset markers in the two images acquired at the same location are identical, and their third positional information should also be consistent. If there is an error between the third positional information of the preset markers calculated using the first positional relationship (e.g., the third positional information of the preset markers in the nth frame calculated based on the positional relationship between the nth frame image and the preset markers in previous images) and the actual third positional information of the preset markers (e.g., the third positional information of the preset markers directly determined based on the first frame image), then correction can be performed based on this error to eliminate the error, i.e., to obtain... Figure 2 The position after loop closure detection correction. In the example, this error can be eliminated by methods such as BA bundle adjustment or loop closure detection correction; this disclosure does not limit the method for eliminating the error. After eliminating the above error, a corrected three-dimensional relationship diagram, i.e., a three-dimensional map, can be obtained. Based on this three-dimensional map, the position of the rotation axis with higher accuracy can be calculated, i.e., Figure 2 The center of the corrected trajectory is fitted to the circle.

[0054] In this way, the error of the third position information of the same preset mark can be used to correct the 3D map, so as to obtain an accurate 3D map, which is beneficial to improving the accuracy of the position of the rotation axis of the turntable.

[0055] In one possible implementation, in step S13, the first position information of the optical center of the image acquisition device in the three-dimensional map can be determined when acquiring each frame of images. The image acquisition device, such as a camera, has specific optical parameters, such as focal length and target surface parameters, i.e., intrinsic parameters of the image acquisition device. The position of the optical center of the image acquisition device can be determined by using the intrinsic parameters of the image acquisition device, the positions of preset markers in the images captured by the image acquisition device, and the positions of the preset markers in three-dimensional space. Step S13 may include: determining a second positional relationship between the fourth position information of the preset markers in each frame of images and the position of the optical center of the image acquisition device based on the preset intrinsic parameters of the image acquisition device; and determining the first position information of the optical center in the three-dimensional map based on the position of the preset markers in the three-dimensional map and the second positional relationship.

[0056] In the example, the preset marker captured in the image can be used as the projection of the preset marker in the 3D map onto the target surface of the image acquisition device. Based on the above projection relationship and the second positional relationship between the preset marker in the image and the optical center, the first positional information of the optical center in the 3D map can be determined.

[0057] In the example, the second positional relationship between the fourth positional information of the preset marker and the position of the optical center of the image acquisition device can be determined by the intrinsic parameters of the image acquisition device. For example, the target surface of the image acquisition device is perpendicular to the optical axis of the image acquisition device, the optical center is located on the optical axis, and the distance between the optical center and the target surface is determined. Therefore, the target surface can be determined based on the fourth positional information of multiple preset markers, and the second positional relationship between the fourth positional information and the optical center can be determined based on the above positional relationship between the optical center and the target surface.

[0058] In the example, based on the aforementioned projection relationship, the relationship between the position of the preset marker in the 3D map (i.e., the third positional information) and the fourth positional information in the image can also be determined. Furthermore, based on the aforementioned two sets of positional relationships (i.e., the second positional relationship between the optical center and the fourth positional information of the preset marker in the image, and the projection relationship between the fourth positional information of the preset marker in the image and the third positional information of the preset marker in the 3D map), the first positional information of the optical center in the 3D map can be determined. For example, the first positional information of the optical center can be inferred based on the third positional information and the aforementioned two sets of positional relationships. The first positional information of the optical center of the image acquisition device in the 3D map when acquiring each frame of image can also be determined using the above method.

[0059] In one possible implementation, in step S14, the relative positional relationship between the optical center and the rotation axis can be determined by the first position information of the optical centers of multiple cameras at multiple times (i.e., the times when each frame of image is captured).

[0060] In the example, if the camera's optical center coincides with the rotation axis of the turntable, then the first position information at each moment also coincides; that is, the optical center is always in a position coincident with the rotation axis. If the camera's optical center does not coincide with the rotation axis of the turntable, then during the turntable's rotation, the trajectory of each first position information can form a circle, with the position of the rotation axis as the center.

[0061] In one possible implementation, step S14 may include: fitting the first position information of the optical center when acquiring the plurality of images to obtain the target position information of the rotation axis; and determining the relative positional relationship between the first position information and the target position information. In the example, the position of the center of the circle, i.e., the position of the rotation axis, can be obtained by polynomial fitting, least squares fitting, etc., and the relative positional relationship between the first position information and the target position information can be determined, such as a distance relationship, an angular relationship, or other relative positional relationship.

[0062] In the example, the first position information of multiple optical centers is located on the trajectory of the fitted circular trajectory, and the position of the rotation axis is the center of the circular trajectory. Therefore, the relative positional relationship between the position of the optical center on the circular trajectory and the position of the rotation axis at the center of the circle is determined. For example, the distance between the two (i.e., the radius of the circular trajectory) is determined. If the installation position of the image acquisition device is adjustable, the installation position of the image acquisition device can be adjusted based on this distance. That is, the position of the optical center of the image acquisition device is adjusted towards the center of the circle, and the adjustment distance is the radius of the circular trajectory, so that the position of the optical center coincides with the position of the rotation axis.

[0063] In one possible implementation, after determining the position of the rotation axis, if the optical center of the image acquisition device does not coincide with the position of the rotation axis, the installation position of the image acquisition device can be adjusted. The method further includes adjusting the installation position of the image acquisition device according to the relative positional relationship, so that the optical center of the image acquisition device coincides with the rotation axis. That is, by adjusting the position to make the optical center of the image acquisition device coincide with the rotation axis, the image acquisition device can perform pure rotational motion during the rotation of the turntable, which helps improve the accuracy of stitching the captured images.

[0064] In one possible implementation, if the image acquisition device is difficult to adjust its installation position—for example, if the installation position of the image acquisition device is fixed and there is a fixed error between its optical center and the rotation axis of the turntable—then the registration error in the image captured by the image acquisition device can be compensated based on this error to improve the stitching accuracy. The method further includes: correcting the image acquired by the image acquisition device according to the relative positional relationship; or determining a third positional relationship between the target surface of the image acquisition device and the rotation axis based on the relative positional information, and correcting the image acquired by the image acquisition device according to the third positional relationship.

[0065] In the example, the image captured by the image acquisition device can also be projectively transformed directly by the target position information of the rotation axis and the relative position relationship between the optical center (i.e., the positional error between the two) to compensate for the image registration error and improve the stitching accuracy of multiple images captured by the image acquisition device.

[0066] In the example, the angular relationship between the target surface and the optical axis of the image acquisition device is fixed (i.e., perpendicular). Therefore, the third positional relationship (angular relationship) between the target surface and the rotation axis can be determined by the relative positional relationship between the target position information of the rotation axis and the first position information of the optical center (i.e., the positional error between them) and the angular relationship between the target surface and the optical axis. Furthermore, the third positional relationship can be used to perform projective transformation on the images captured by the image acquisition device to compensate for image registration errors and improve the stitching accuracy of the images captured by the image acquisition device. In the example, the target surface is perpendicular to the optical axis and should be parallel to the rotation axis. However, if the third positional relationship (angular relationship) is not parallel, it can be used for compensation and correction to improve the stitching accuracy of the multiple captured images.

[0067] According to the positioning method of the embodiments of this disclosure, a three-dimensional map can be obtained by acquiring multiple frames of images during the rotation of the image acquisition device. The three-dimensional map is then corrected by the error of the third position information with the same preset marker, resulting in an accurate three-dimensional map. The position of the optical center in the three-dimensional map is then determined to ascertain the target position information of the rotation axis of the turntable in the three-dimensional map. This allows for the acquisition of the relative positional relationship between the optical center and the rotation axis of the turntable, enabling correction based on this relative positional relationship, improving installation accuracy, and consequently, improving image stitching accuracy.

[0068] Figure 3A and Figure 3B This diagram illustrates the application of the positioning method according to an embodiment of the present disclosure, such as... Figure 3A As shown, the preset marker can be an Aruco identifier, and each Aruco identifier is different from the others, that is, each Aruco identifier is unique.

[0069] In one possible implementation, such as Figure 3B As shown, preset markers can be set around the turntable, so that the image acquisition device captures the preset markers during the rotation and image capture process. This determines the positional relationship of each marker in the image, and further determines the positional relationship of multiple markers in the 3D map, thereby creating a 3D map. Furthermore, by rotating the image acquisition device more than once, allowing it to pass through the same position at least twice, and by using the positions of the preset markers in the two frames captured at the same position, accumulated errors can be eliminated, resulting in a more accurate 3D map.

[0070] In one possible implementation, the positional relationship between the positional information of the preset marker in each frame of the image and the optical center of the image acquisition device can be determined based on the preset intrinsic parameters of the image acquisition device. Based on this positional relationship and the position of the preset marker in the three-dimensional map, the positional information of the optical center in the three-dimensional map can be obtained.

[0071] In one possible implementation, if the optical center does not coincide with the rotation axis, the trajectory of the optical center is circular. By fitting the position of the optical center, the center of the circle obtained is the position of the rotation axis. The installation position of the image acquisition device can be adjusted so that the optical center coincides with the rotation axis, thereby improving the stitching accuracy of the images captured by the image acquisition device.

[0072] In one possible implementation, the positioning method can be used in fields such as image processing, enabling the optical center of the image acquisition device to coincide with the rotation axis, thereby improving the registration accuracy of images captured by the image acquisition device and thus enhancing the stitching accuracy of the images. This disclosure does not limit the application areas of the positioning method.

[0073] Figure 4 A block diagram of a positioning device according to an embodiment of the present disclosure is shown, such as Figure 4 As shown, the device includes: an image acquisition module 11, used to acquire multiple frames of images of a target scene in real time, wherein a turntable is provided in the target scene, an image acquisition device is provided on the turntable, the multiple frames of images are acquired by the image acquisition device as it rotates with the turntable, and preset markers are provided at multiple locations in the target scene; a map acquisition module 12, used to establish a three-dimensional map based on the preset markers captured in the multiple frames of images; an optical center determination module 13, used to determine, based on the three-dimensional map and the multiple frames of images, the first position information of the optical center of the image acquisition device in the three-dimensional map when acquiring each frame of images; and a position determination module 14, used to determine the relative positional relationship between the optical center and the rotation axis of the turntable based on the first position information.

[0074] In one possible implementation, the map acquisition module is further configured to: determine a first positional relationship between the preset markers based on the second positional information of the preset markers in each frame image; and obtain the three-dimensional map based on the first positional relationship.

[0075] In one possible implementation, during the rotation, the image acquisition device passes the same position at least twice, and the map acquisition module is further configured to: determine the third position information of each of the preset markers in the three-dimensional map according to the first position relationship; obtain the three-dimensional relationship diagram according to the third position information; and correct the three-dimensional relationship diagram according to the at least two frames of images acquired by the image acquisition device at the same position and the first position relationship to obtain the three-dimensional map.

[0076] In one possible implementation, the optical center determination module is further configured to: determine a second positional relationship between the fourth position information of the preset marker in each frame image and the position of the optical center of the image acquisition device according to the preset intrinsic parameters of the image acquisition device; and determine the first position information of the optical center in the three-dimensional map according to the position of the preset marker in the three-dimensional map and the second positional relationship.

[0077] In one possible implementation, the position determination module is further configured to: fit the first position information of the optical center when acquiring the plurality of images to obtain the target position information of the rotation axis; and determine the relative positional relationship between the first position information and the target position information.

[0078] In one possible implementation, the device further includes an adjustment module for adjusting the installation position of the image acquisition device according to the relative positional relationship, such that the optical center of the image acquisition device coincides with the rotation axis.

[0079] In one possible implementation, the device further includes a correction module for correcting the image acquired by the image acquisition device according to the relative positional relationship; or for determining a third positional relationship between the target surface of the image acquisition device and the rotation axis according to the relative positional information, and correcting the image acquired by the image acquisition device according to the third positional relationship.

[0080] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0081] In addition, this disclosure also provides positioning devices, electronic devices, computer-readable storage media, and programs, all of which can be used to implement any of the positioning methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0082] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0083] In some embodiments, the apparatus provided in this disclosure may have functions or include modules that can be used to perform the methods described in the above method embodiments. Specific implementations can be referred to the descriptions in the above method embodiments, and for brevity, will not be repeated here.

[0084] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0085] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured as described above.

[0086] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0087] Figure 5 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0088] Reference Figure 5 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0089] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0090] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0091] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0092] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0093] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0094] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0095] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0096] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0097] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0098] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by a processor 820 of an electronic device 800 to perform the above-described method.

[0099] Figure 6 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 6 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0100] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS XTM Unix™, Linux TM FreeBSD TM Or similar.

[0101] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0102] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0103] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0104] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0105] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0106] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0107] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0108] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0110] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A positioning method, characterized in that, include: The system acquires multiple frames of images of a target scene in real time. A turntable is set in the target scene, and an image acquisition device is set on the turntable. The multiple frames of images are acquired by the image acquisition device as the turntable rotates. Preset marks are set at multiple locations in the target scene. A three-dimensional map based on the target scene is established based on the preset markers captured in the multi-frame images; Based on the three-dimensional map and the multiple frames of images, determine the first position information of the optical center of the image acquisition device in the three-dimensional map when acquiring each frame of images; Based on the first position information, the relative positional relationship between the optical center and the rotation axis of the turntable is determined; Building a 3D map based on the target scene using the preset markers captured in the multiple frames of images includes: determining a first positional relationship between the preset markers based on the second positional information of the preset markers in each frame of images; and obtaining the 3D map based on the first positional relationship. During the rotation, the image acquisition device passes the same position at least twice. Based on the first positional relationship, the three-dimensional map is obtained, including: determining the third position information of each preset marker in the three-dimensional map based on the first positional relationship; obtaining a three-dimensional relationship diagram based on the third position information; and correcting the three-dimensional relationship diagram based on at least two frames of images acquired by the image acquisition device at the same position and the first positional relationship to obtain the three-dimensional map. Based on the 3D map and the multiple image frames, the first position information of the optical center of the image acquisition device in the 3D map is determined when acquiring each image frame, including: Based on the preset intrinsic parameters of the image acquisition device, a second positional relationship is determined between the fourth positional information of the preset marker in each frame image and the position of the optical center of the image acquisition device. Based on the position of the preset marker in the three-dimensional map and the second positional relationship, the first positional information of the optical center in the three-dimensional map is determined; Determining the relative positional relationship between the optical center and the rotation axis of the turntable based on the first position information includes: The target position information of the rotation axis is obtained by fitting the first position information of the optical center when acquiring the multiple images; Determine the relative positional relationship between the first location information and the target location information.

2. The method according to claim 1, characterized in that, The method further includes: Adjust the installation position of the image acquisition device according to the relative positional relationship so that the optical center of the image acquisition device coincides with the rotation axis.

3. The method according to claim 1, characterized in that, The method further includes: The image acquired by the image acquisition device is corrected according to the relative positional relationship; or Based on the relative position information, a third positional relationship between the target surface of the image acquisition device and the rotation axis is determined, and the image acquired by the image acquisition device is corrected based on the third positional relationship.

4. A positioning device, characterized in that, include: An image acquisition module is used to acquire multiple frames of images of a target scene in real time. A turntable is set in the target scene, and an image acquisition device is set on the turntable. The multiple frames of images are acquired by the image acquisition device as the turntable rotates. Preset marks are set at multiple locations in the target scene. The map acquisition module is used to build a three-dimensional map based on the target scene based on the preset markers captured in the multi-frame images; The optical center determination module is used to determine, based on the three-dimensional map and the multiple frames of images, the first position information of the optical center of the image acquisition device in the three-dimensional map when acquiring each frame of images; The position determination module is used to determine the relative positional relationship between the optical center and the rotation axis of the turntable based on the first position information; The map acquisition module is further configured to: determine a first positional relationship between the preset markers based on the second positional information of the preset markers in each frame image; and obtain the three-dimensional map based on the first positional relationship. During the rotation, the image acquisition device passes the same position at least twice, and the map acquisition module is further configured to: determine the third position information of each preset marker in the three-dimensional map according to the first position relationship; obtain a three-dimensional relationship diagram according to the third position information; and correct the three-dimensional relationship diagram according to the at least two frames of images acquired by the image acquisition device at the same position and the first position relationship to obtain the three-dimensional map. The optical center determination module is further used to: determine a second positional relationship between the fourth position information of the preset marker in each frame image and the position of the optical center of the image acquisition device, based on the preset internal parameters of the image acquisition device; Based on the position of the preset marker in the three-dimensional map and the second positional relationship, the first positional information of the optical center in the three-dimensional map is determined; The position determination module is further configured to: fit the first position information of the optical center when acquiring the plurality of images to obtain the target position information of the rotation axis; and determine the relative positional relationship between the first position information and the target position information.

5. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method described in any one of claims 1 to 3.

6. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 3.

Images