Pose information determination method and system for improving camera reconstruction precision, electronic equipment and storage medium

Abstract

The invention belongs to the related technical field of machine vision, and discloses a pose information determination method and system for improving the reconstruction precision of a camera, electronic equipment and a storage medium, and the method comprises the steps: S1, collecting the image information of a first preset number of frames before the current frame of image information, and carrying out the time filtering, judging the frame number with pose information in the image information of the first preset number of frames, if the frame number is greater than or equal to a second preset number, executing S2 and S3 to carry out average pose filtering, and if the frame number is less than the second preset number, executing S4; s2, obtaining average pose information of all pose information; s3, determining final pose information of the current frame according to the average pose information; and S4, determining the final pose information of the current frame according to the final pose information in the previous frame of image information. According to the method, a dual filtering strategy based on time and attitude is used, the reconstruction precision and stability are improved, and stable and high-precision object pose information can be conveniently provided for the robot.

Description

Technical Field

[0001] The present invention belongs to the technical field related to machine vision, and more specifically, relates to a method, system, electronic device and storage medium for determining posture information for improving camera reconstruction accuracy. Background Art

[0002] With the development and maturity of 3D imaging technology, visual cameras are increasingly used in actual production and life related to robots, such as precise positioning in inspection robots, material dismantling and palletizing robots, palletizing and disassembly, etc. Using visual cameras as the visual solution for such scenarios not only has stronger compatibility and flexibility, but also can track and locate the target object at the same time, further improving the accuracy and success rate of the robot's operation.

[0003] The existing method is usually to determine the pose information of an object in the three-dimensional world through a visual camera, that is, to reconstruct the pose information of an object in the three-dimensional world through machine vision. However, in the actual implementation process, it is found that when the results collected by the camera are dropped or due to problems such as exposure adjustment, the output of the pose information will have random jumps. This uncertain pose information is not allowed to appear during the robot operation process. And because the monocular camera obtains the pose information based on a numerical optimization algorithm that solves the PNP problem, it is inevitable that the pose information will be affected to varying degrees due to the limited number of samples, the setting of the initial value, and the iteration step size. That is, because the reconstruction result of the camera has certain errors caused by numerical calculations, the accuracy of the pose information actually transmitted is uncertain. Summary of the invention

[0004] In response to the above defects or improvement needs in the prior art, the present invention provides a method, system, electronic device and storage medium for determining posture information that improves the camera reconstruction accuracy. The method is used to solve the problem that when reconstructing the posture information of three-dimensional objects in the world through machine vision, the posture information output is 0 due to frame loss, etc., and there are errors in the numerical calculation of the posture information, resulting in uncertain accuracy. A dual filtering strategy based on time and posture is used to improve the reconstruction accuracy and stability, so as to provide the robot with stable and high-precision object posture information.

[0005] To achieve the above object, according to a first aspect of the present invention, a method for determining pose information for improving camera reconstruction accuracy is provided, comprising:

[0006] S1, collecting image information of a first preset number of frames before the current frame image information for time filtering, determining the number of frames with posture information in the first preset number of frames of image information, if the number of frames with posture information is greater than or equal to a second preset number, executing S2 and S3 to perform average posture filtering, if the number of frames with posture information is less than the second preset number, executing S4;

[0007] S2, obtaining the average posture information of all posture information;

[0008] S3, determining final pose information of the current frame according to the average pose information for use in three-dimensional reconstruction;

[0009] S4, determining the final posture information of the current frame according to the final posture information in the previous frame image information of the current frame for use in three-dimensional reconstruction.

[0010] According to the method for determining the pose information of the camera with improved reconstruction accuracy provided by the present invention, when the camera is a binocular camera, before S1, the method further includes:

[0011] S00, receiving images transmitted respectively by two cameras in a binocular camera. If the same frame images of the two cameras are both valid images containing the target to be measured, the image information of the acquired frame image contains posture information; if the same frame images of the two cameras contain invalid images that do not contain the target to be measured, the image information of the acquired frame image contains no posture information.

[0012] According to the method for determining the pose information to improve the camera reconstruction accuracy provided by the present invention, before S1, the method further includes:

[0013] S01, when the current frame image information has posture information, performing singular posture filtering on the posture information in the current frame image information to determine whether the posture information in the current frame image information belongs to a singular posture;

[0014] If not, the collected current frame image information contains posture information; if so, the collected current frame image information contains no posture information.

[0015] According to the method for determining the pose information to improve the camera reconstruction accuracy provided by the present invention, whether the pose information in the current frame image information belongs to a singular pose is determined by a threshold method, specifically:

[0016] Obtain the angles of the axis-angle representation of the current frame pose information and the final pose information of the previous frame of the current frame respectively, and calculate the difference between the two;

[0017] When the difference is less than or equal to a preset threshold, it is determined that the current frame pose information belongs to the real pose;

[0018] When the difference is greater than a preset threshold, it is determined that the current frame posture information belongs to a singular posture.

[0019] The method for determining the pose information to improve the camera reconstruction accuracy provided by the present invention also includes:

[0020] In the initial stage, when the current frame image information first satisfies that the number of frames with posture information in the image information of the first preset number of frames before the current frame is greater than or equal to the second preset number, the average posture information of all posture information is output as the final posture information of the current frame.

[0021] According to the method for determining the pose information for improving the camera reconstruction accuracy provided by the present invention, whether the pose information in the current frame image information belongs to a singular pose is determined by a clustering method, specifically:

[0022] Obtain the angles of the axis-angle representation of the current frame pose information and the final pose information of the previous frame of the current frame respectively, and calculate the difference between the two;

[0023] The k-Means clustering method is constructed with the difference as the characteristic distance defined, and the real posture and the singular posture are distinguished by the k-Means clustering method.

[0024] According to the method for determining the pose information for improving the camera reconstruction accuracy provided by the present invention, when determining whether the pose information in the current frame image information belongs to a singular pose by a clustering method, the method further includes:

[0025] All the posture information received within the initial preset time is used as a sample set to train the clustering method. The clustering method can divide all the samples in the sample set into two clusters, and define the cluster with more samples as the true posture;

[0026] After reaching the preset time from the beginning, a clustering method is used to determine whether the posture information in the current frame image information belongs to a singular posture.

[0027] According to a second aspect of the present invention, a system for determining posture information for improving camera reconstruction accuracy is provided, comprising:

[0028] A time filtering module, used for collecting image information of a first preset number of frames before the current frame image information for time filtering, determining the number of frames with posture information in the first preset number of frames of image information, and if the number of frames with posture information is greater than or equal to a second preset number, executing an averaging module and a first output module to perform average posture filtering, and if the number of frames with posture information is less than the second preset number, executing the second output module;

[0029] An average module is used to obtain the average pose information of all pose information;

[0030] A first output module, used to determine final pose information of the current frame according to the average pose information for three-dimensional reconstruction;

[0031] The second output module is used to determine the final posture information of the current frame according to the final posture information in the previous frame image information of the current frame for three-dimensional reconstruction.

[0032] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for determining posture information for improving the camera reconstruction accuracy as described in any one of the above items are implemented.

[0033] According to a fourth aspect of the present invention, there is provided a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for determining pose information for improving camera reconstruction accuracy as described in any one of the above items are implemented.

[0034] In general, compared with the prior art, the above technical solutions conceived by the present invention provide a method, system, electronic device and storage medium for determining pose information for improving camera reconstruction accuracy:

[0035] 1. A method for temporal filtering of posture information is proposed. By tracing back the image information of the first preset number of frames before the current frame, the final output posture information of the current frame is determined according to all the valid posture information in the image information of the first preset number of frames. This can ensure that valid posture information can be transmitted all the time, and invalid posture information will not appear, that is, the posture information will not be transmitted as 0;

[0036] 2. It is proposed to average all valid pose information in the image information of the first preset number of frames, and determine the final pose information of the current frame by averaging the pose information, which can effectively improve the pose information calculation process and the random error problem caused by environmental factors, and improve the accuracy of the pose information;

[0037] 3. When the camera is a binocular camera, image matching of the two cameras can be performed in advance, and the scheme of time filtering and average pose filtering can be applied to the three-dimensional reconstruction of the binocular camera;

[0038] 4. It is proposed to perform singular posture filtering before temporal filtering to remove singular postures and further improve the accuracy of the final posture information. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a method for determining posture information for improving camera reconstruction accuracy provided by the present invention;

[0040] Figure 2 It is a schematic diagram of clustering results of the clustering method provided by the present invention;

[0041] Figure 3 It is a schematic diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] See also Figure 1 , this embodiment 1 provides a method for determining posture information for improving camera reconstruction accuracy, the method comprising:

[0044] S1, collect the image information of the first preset number of frames before the current frame image information for time filtering, determine the number of frames with posture information in the first preset number of frames, if the number of frames with posture information is greater than or equal to the second preset number, execute S2 and S3 to perform average posture filtering, if the number of frames with posture information is less than the second preset number, execute S4; the image information of the first preset number of frames includes the image information of the current frame;

[0045] S2, obtaining average posture information of all posture information; that is, averaging all posture information of the image information of the first preset number of frames to obtain average posture information;

[0046] S3, determining final pose information of the current frame according to the average pose information for use in three-dimensional reconstruction;

[0047] S4, determining the final posture information of the current frame according to the final posture information in the previous frame image information of the current frame for use in three-dimensional reconstruction.

[0048] The technical problems mainly addressed by this embodiment can be listed as follows: in the process of existing machine vision to reconstruct the posture information of the object to be measured, there may be frame loss, which will cause the transmitted posture information to be 0 in a certain frame; secondly, the posture information is calculated based on numerical methods, which will cause certain random errors in the posture information, which may cause the posture information of the object transmitted in a certain frame to be incorrect; and there are unavoidable environmental factors in the camera reconstruction process, such as random errors caused by excessive light, which will result in a certain degree of error in the final posture information, which will cause accidents during the operation of the robot.

[0049] In response to the first problem that "camera detection may result in frame loss, resulting in the output of posture information being 0", this embodiment proposes a time-based filtering strategy. For example, under the conditions of a 1080P, 30Fps camera, it is stipulated that if posture information can be detected in 6 or more of the 10 consecutive frames before the current frame of the camera, including the current frame, all the posture information in the 10 frames are considered to be the valid posture information group of the current frame; all the posture information in the 10 frames can be stored, and then the subsequent posture information calculation can be performed. After continuous testing, as long as there is a reference code to be detected in the visible space of the camera, it can be guaranteed that the camera will always transmit a valid posture, and there will be no invalid posture, that is, there will be no posture information output of 0. This embodiment calls this method time-based filtering, and further calculates the stored multi-frame posture information to perform posture filtering.

[0050] In response to the second problem of "random errors caused by numerical calculation of pose information and environmental factors during camera reconstruction", this embodiment proposes a filtering strategy based on pose. Specifically, since the camera obtains pose information based on a numerical optimization algorithm that solves the PNP problem, it is inevitable that the pose information will be affected to varying degrees due to the limited number of samples, the setting of initial values, and the influence of iterative step size. Therefore, this embodiment proposes a method of averaging each group of pose information of the valid pose stored in the above-mentioned time filtering process, and determining the output pose information of the current frame based on the average pose information. Since robot operations are often performed on static objects, pose information during motion will not be used. The average pose can return valid pose information within 1 / 3s under static conditions. After testing, the deviation between the actual detected valid pose transmitted back by the robot under quasi-static conditions and the average pose after filtering multiple frames of poses is very small, and the average pose filtering is effective.

[0051] Furthermore, the calculation method of the average pose information is as follows: first, the pose information is divided into position information and attitude information. The average of the position information can be obtained by arithmetic averaging. The average of the attitude is obtained by obtaining the quaternion of the rotation matrix and averaging all the quaternions. That is, the average pose information is realized by the following process:

[0052] The posture information is It is expressed as a homogeneous coordinate transformation from the object coordinate system {O} to the robot coordinate system {R}, which is expanded as follows:

[0053]

[0054] Where R is the posture information, t is the position information, and the average position information is calculated as follows:

[0055]

[0056] Get the average position information vector of the arithmetic mean t i is the position information in the i-th pose information, f s is the total number of frames with pose information.

[0057] The rotation matrix is ​​further converted into a quaternion. The rotation matrix can be specifically expressed as:

[0058]

[0059] The quaternion q(q1,q2,q3,q4) can be obtained from the rotation matrix;

[0060] in:

[0061]

[0062]

[0063] The multiple posture information in the multiple pose information has multiple sets of quaternions. For the multiple sets of quaternions q i Find the average use:

[0064]

[0065] Among them, due to The constraints of the quaternion need to be satisfied, so the four-dimensional vector is constrained to a three-dimensional sphere in a four-dimensional space, namely the hypersphere S 3 On, further:

[0066]

[0067] in:

[0068]

[0069] Agreement:

[0070]

[0071] Since the sum of squares of quaternions is 1, we have:

[0072]

[0073] Finally, we can get:

[0074] Tr[A(q)B T ] = q T Kq;

[0075] The problem turns into:

[0076]

[0077] Since q i is the i-th group of quaternions, n is the number of quaternion groups, w i Represents the weight of the i-th quaternion; M is a symmetric matrix, which is similarly diagonalized. From the spectral decomposition, it can be seen that the eigenvector corresponding to the maximum eigenvalue is the average quaternion vector.

[0078] Depend on Get the average rotation matrix So the average homogeneous coordinate transformation matrix, that is, the average pose information, is obtained as follows:

[0079]

[0080] In some specific embodiments, S3 specifically includes: using the calculated average pose information as the final pose information transmitted by the current frame. The second preset number may be 1 / 2-4 / 5 of the first preset number.

[0081] Furthermore, when the camera is a binocular camera, before S1, the following steps are also included:

[0082] S00, receiving images transmitted respectively by two cameras in the binocular camera. If the same frame images of the two cameras are both valid images containing the target to be measured, the image information of the acquired frame image has posture information; if there is an invalid image that does not contain the target to be measured in the same frame image of the two cameras, the image information of the acquired frame image has no posture information.

[0083] This embodiment aims at the frame loss problem in the binocular camera reconstruction process. Since the left and right cameras need to detect the object to be measured at the same time during the binocular camera reconstruction process, it is necessary to first match the left and right cameras to detect the object to be measured at the same time before the above-mentioned time-based filtering. The detected target to be measured is a valid image. Similarly, when the conditions are met, that is, the same frame image of the two cameras are valid images, the two groups of images of the left and right cameras of the binocular camera are stored, and a pose information is returned when the subsequent pose calculation is output for this frame. If the left and right cameras of the binocular camera do not meet the conditions, that is, the image of one camera is an invalid image or the images of both cameras are invalid images, the pose information output of this frame is zero. An invalid image means that the target to be measured is not detected in the image. Similarly, after testing, the binocular camera reconstruction can transmit valid pose information through continuous testing under the above-mentioned filtering conditions.

[0084] Furthermore, before S1, it also includes:

[0085] S01, when there is posture information in the current frame image information, singular posture filtering is performed on the posture information in the current frame image information to determine whether the posture information in the current frame image information belongs to a singular posture; if not, the collected current frame image information has posture information; if so, there is no posture information in the collected current frame image information. That is, if it is determined that the posture information in the current frame image information belongs to a singular posture, the posture information in the current frame image information is set to zero, so that there is no posture information in the collected current frame image information. Singular posture filtering is performed when there is posture information in the current frame image information, and no singular posture filtering is performed when the posture information in the current frame image information is zero, that is, there is no posture information. Through singular posture filtering, posture information belonging to singular postures can be removed, which is beneficial to improving singular posture errors caused by environmental factors such as overexposure, and is beneficial to improving the accuracy of the final posture information, so as to improve the accuracy of machine vision three-dimensional reconstruction.

[0086] Whether the posture information in the current frame image information belongs to a singular posture can be determined by a threshold method or a clustering method. In some specific embodiments, whether the posture information in the current frame image information belongs to a singular posture can be determined by a threshold method, specifically:

[0087] Obtain the angles of the axis-angle representation of the current frame pose information and the final pose information of the previous frame of the current frame respectively, and calculate the difference between the two;

[0088] When the difference is less than or equal to a preset threshold, it is determined that the current frame pose information belongs to the real pose;

[0089] When the difference is greater than a preset threshold, it is determined that the current frame posture information belongs to a singular posture.

[0090] The final pose information of the previous frame of the current frame is the pose information determined by the final output of the previous frame of the current frame. The angle of the axis-angle representation of the pose information of the current frame and the angle of the axis-angle representation of the final pose information of the previous frame of the current frame are compared and subtracted. When the difference exceeds the threshold, it is considered that the pose error is large, and the pose is discarded and the pose information is output as zero; when the difference is within the threshold range, it means that the pose is reasonable, and the pose information after singular pose filtering is used as valid pose information for the next step of time filtering.

[0091] Furthermore, the method for determining the pose information based on hybrid filtering to improve the camera reconstruction accuracy also includes:

[0092] In the initial stage, when the current frame image information first satisfies that the number of frames with posture information in the image information of the first preset number of frames before the current frame is greater than or equal to the second preset number, the average posture information of all posture information is output as the final posture information of the current frame.

[0093] That is, starting from receiving the camera image, starting from the first frame of image information, determine whether there is posture information in each frame of image information, starting from the first preset number of frames of image information, determine whether the number of frames with posture information in the image information of the previous first preset number of frames is greater than or equal to the second preset number, when the current frame image information first meets the requirement that the number of frames with posture information in the image information of the first preset number of frames before the current frame is greater than or equal to the second preset number, output the average posture information of all posture information in the image information of the first preset number of frames before the current frame as the final posture information of the current frame; before the requirement that the number of frames with posture information in the image information of the first preset number of frames before the current frame is greater than or equal to the second preset number is met for the first time, do not output the final posture information.

[0094] After the final pose information is output for the first time, it is judged whether the number of frames with pose information in the image information of the first preset number of frames before the current frame image information is greater than or equal to the second preset number. If the number of frames with pose information is greater than or equal to the second preset number, S2 and S3 are executed to perform average pose filtering. If the number of frames with pose information is less than the second preset number, S4 is executed. That is, the final pose information is not output for the first few frames when the algorithm is started. After the final pose information is output for the first time, a final pose information is output for each frame, and it will be corrected to a smaller error through continuous calculation later.

[0095] In some other specific embodiments, whether the posture information in the current frame image information belongs to a singular posture is determined by a clustering method, specifically:

[0096] Obtain the angles of the axis-angle representation of the current frame pose information and the final pose information of the previous frame of the current frame respectively, and calculate the difference between the two;

[0097] The k-Means clustering method is constructed with the difference as the characteristic distance defined, and the real posture and the singular posture are distinguished by the k-Means clustering method.

[0098] Furthermore, when the clustering method is used to determine whether the posture information in the current frame image information belongs to a singular posture, it also includes:

[0099] All the posture information received within the initial preset time is used as a sample set to train the clustering method. The clustering method can divide all the samples in the sample set into two clusters, and define the cluster with more samples as the true posture;

[0100] After reaching the preset time from the beginning, a clustering method is used to determine whether the posture information in the current frame image information belongs to a singular posture.

[0101] In order to further improve the effectiveness of posture filtering, according to existing studies, the singular posture problem in the machine vision reconstruction process is related to the camera arrangement homogeneous coordinate transformation matrix. Therefore, this embodiment proposes a k-Means clustering method to distinguish between real postures and singular postures by using the feature distance defined by the difference in the above-mentioned axis-angle representation.

[0102] Since the camera pose is the factor that determines the distribution of singular poses, when the camera pose is determined, the distribution of singular poses on the definition domain {SO3} is also determined, that is, the real pose and singular pose will be clustered into two categories on {SO3}. In addition, since the pose conforms to the hypersphere constraint, the k-Means clustering method can be used to cluster the pose information of the image into two clusters, one of which is the real pose and the other is the singular pose. It can be seen from the actual reconstruction effect that the one with more sample points should be the real pose. After actual testing, the silhouette coefficient of the cluster is 0.8. The test effect is shown in the attached figure. Figure 2 When the current frame image information contains posture information, the posture information of the current frame can be clustered and compared to distinguish whether it is a real posture or a strange posture.

[0103] In some specific embodiments, a method for determining posture information based on hybrid filtering to improve camera reconstruction accuracy is specifically as follows, taking application to a binocular camera as an example: starting from receiving camera images, that is, starting from the first frame image, first receiving the current frame images returned by the two cameras of the binocular camera respectively to perform image matching of the two cameras, and when the current frame images of the two cameras are both valid images, the image information of the current frame outputted through posture information calculation includes a posture information; when there is an invalid image in the current frame images of the two cameras, the posture information in the output current frame image information is zero.

[0104] Then, when the current frame image information contains posture information, singular posture filtering is performed. The threshold method or clustering method can be used to determine whether the current frame posture information is a singular posture. If it is not a singular posture, that is, it is a real posture, then the posture information of the current frame is used as valid posture information, so that the collected current frame graphic information contains posture information; if it is a singular posture, the posture information of the current frame is considered invalid and the output is zero.

[0105] Starting from receiving the first frame of image, determine whether there is posture information in each frame of image information collected. When the current frame image information first meets the condition that the number of frames with posture information in the first preset number of frames before the current frame is greater than or equal to the second preset number, output the average posture information of all posture information as the final posture information of the current frame, that is, output the final posture information for the first time. Then, determine whether the number of frames with posture information in the first preset number of frames before the current frame image information is greater than or equal to the second preset number. If the number of frames with posture information is greater than or equal to the second preset number, execute S2 and S3 to perform average posture filtering. If the number of frames with posture information is less than the second preset number, execute S4.

[0106] Temporal filtering is proposed to address the problems of frame loss, exposure adjustment, left and right camera mismatch, etc., and of course there is also the consideration of reducing the error on average, that is, this embodiment considers only calculating valid and reasonable data, so as to ensure the stability and effectiveness of the output.

[0107] Posture-based filtering is proposed for situations such as singular postures and excessive posture errors. The main processing is to first delete these non-compliant posture results, and then take the average to further reduce the error. This embodiment proposes the removal of unreasonable postures, namely singular postures, and provides a clearer binocular camera reconstruction scenario and more practical problems. This embodiment takes into account the problems in the binocular camera reconstruction process as much as possible (including whether the camera successfully acquires the image and whether the reconstruction is accurate), and improves the output posture accuracy and stability (stability means that the output posture of each frame is basically correct). In summary, a hybrid filtering method based on time filtering and position filtering is implemented, which effectively improves the accuracy and stability of the posture information of objects in the three-dimensional world reconstructed based on hybrid machine vision.

[0108] Finally, based on the Ubuntu 20.04 environment, the test code was developed using cpp using the OpenCV 4.2.0 and Eigen 3.3 open source libraries. After one hour of continuous testing, the stereo camera at 30FPS theoretically returned 107,991 frames of pose information. Due to exposure adjustment, the relationship between the binocular camera's viewing angle, etc., 9,896 frames of pose information were tested under the condition of constantly moving objects. The pose information during the moving object process was discarded, and 8,971 pose information under static conditions was obtained. The maximum deviation of the pose information returned at each static position was within 0.4 mm, and the attitude deviation was within 0.1 degrees.

[0109] The second embodiment provides a system for determining a pose information for improving camera reconstruction accuracy. The system is used to execute the method for determining a pose information for improving camera reconstruction accuracy described in any of the above embodiments, and can be understood by reference to the above method. The system includes:

[0110] A time filtering module, used for collecting image information of a first preset number of frames before the current frame image information for time filtering, determining the number of frames with posture information in the first preset number of frames of image information, and if the number of frames with posture information is greater than or equal to a second preset number, executing an averaging module and a first output module to perform average posture filtering, and if the number of frames with posture information is less than the second preset number, executing the second output module;

[0111] An average module is used to obtain the average pose information of all pose information;

[0112] A first output module, used to determine final pose information of the current frame according to the average pose information for three-dimensional reconstruction;

[0113] The second output module is used to determine the final posture information of the current frame according to the final posture information in the previous frame image information of the current frame for three-dimensional reconstruction.

[0114] The third embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned methods for determining posture information for improving camera reconstruction accuracy are implemented.

[0115] The fourth embodiment of the present invention provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for determining posture information for improving the camera reconstruction accuracy as described in any one of the above items are implemented.

[0116] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The processor may call the logic instructions in the memory to execute a method for determining the pose information to improve the camera reconstruction accuracy.

[0117] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method for determining posture information for improving the camera reconstruction accuracy provided by the above-mentioned methods.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned methods for determining posture information for improving camera reconstruction accuracy.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0122] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining pose information for improving camera reconstruction accuracy, characterized in that: include: S1, collecting image information of a first preset number of frames before the current frame image information for time filtering, determining the number of frames with posture information in the first preset number of frames of image information, if the number of frames with posture information is greater than or equal to a second preset number, executing S2 and S3 to perform average posture filtering, if the number of frames with posture information is less than the second preset number, executing S4; S2, obtaining the average posture information of all posture information; S3, determining final pose information of the current frame according to the average pose information for use in three-dimensional reconstruction; S4, determining the final posture information of the current frame according to the final posture information in the previous frame image information of the current frame for use in three-dimensional reconstruction.

2. The method for determining the pose information for improving the camera reconstruction accuracy according to claim 1, characterized in that: When the camera is a binocular camera, the following are also included before S1: S00, receiving images transmitted respectively by two cameras in a binocular camera. If the same frame images of the two cameras are both valid images containing the target to be measured, the image information of the acquired frame image contains posture information; if the same frame images of the two cameras contain invalid images that do not contain the target to be measured, the image information of the acquired frame image contains no posture information.

3. The method for determining the pose information for improving the camera reconstruction accuracy as claimed in claim 1, characterized in that: Before S1, it also included: S01, when the current frame image information has posture information, performing singular posture filtering on the posture information in the current frame image information to determine whether the posture information in the current frame image information belongs to a singular posture; If not, the collected current frame image information contains posture information; if so, the collected current frame image information contains no posture information.

4. The method for determining the pose information for improving the camera reconstruction accuracy as claimed in claim 3, characterized in that: Whether the posture information in the current frame image information belongs to a singular posture is determined by a threshold method, specifically: Obtain the angles of the axis-angle representation of the current frame pose information and the final pose information of the previous frame of the current frame respectively, and calculate the difference between the two; When the difference is less than or equal to a preset threshold, it is determined that the current frame pose information belongs to the real pose; When the difference is greater than a preset threshold, it is determined that the current frame posture information belongs to a singular posture.

5. The method for determining the pose information for improving the camera reconstruction accuracy according to any one of claims 1 to 4, characterized in that: Also includes: In the initial stage, when the current frame image information first satisfies that the number of frames with posture information in the image information of the first preset number of frames before the current frame is greater than or equal to the second preset number, the average posture information of all posture information is output as the final posture information of the current frame.

6. The method for determining the pose information for improving the camera reconstruction accuracy as claimed in claim 3, characterized in that: Whether the posture information in the current frame image information belongs to a singular posture is determined by a clustering method, specifically: Obtain the angles of the axis-angle representation of the current frame pose information and the final pose information of the previous frame of the current frame respectively, and calculate the difference between the two; The k-Means clustering method is constructed with the difference as the characteristic distance defined, and the real posture and the singular posture are distinguished by the k-Means clustering method.

7. The method for determining the pose information for improving the camera reconstruction accuracy as claimed in claim 6, characterized in that: When the clustering method is used to determine whether the posture information in the current frame image information belongs to a singular posture, it also includes: All the posture information received within the initial preset time is used as a sample set to train the clustering method. The clustering method can divide all the samples in the sample set into two clusters, and define the cluster with more samples as the true posture; After reaching the preset time from the beginning, a clustering method is used to determine whether the posture information in the current frame image information belongs to a singular posture.

8. A posture information determination system for improving camera reconstruction accuracy, characterized in that: include: A time filtering module, used for collecting image information of a first preset number of frames before the current frame image information for time filtering, determining the number of frames with posture information in the first preset number of frames of image information, and if the number of frames with posture information is greater than or equal to a second preset number, executing an averaging module and a first output module to perform average posture filtering, and if the number of frames with posture information is less than the second preset number, executing the second output module; An average module is used to obtain the average pose information of all pose information; A first output module, used to determine final pose information of the current frame according to the average pose information for three-dimensional reconstruction; The second output module is used to determine the final posture information of the current frame according to the final posture information in the previous frame image information of the current frame for three-dimensional reconstruction.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for determining posture information for improving camera reconstruction accuracy as described in any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for determining posture information for improving camera reconstruction accuracy as described in any one of claims 1 to 7 are implemented.

Images