Arrow optimization modeling method and device

Abstract

The invention relates to an arrow optimization modeling method and device, and the method comprises the steps: reconstructing an arrow model according to a constraint relation between objects, carrying out large-scale nonlinear optimization, and adjusting a pose to eliminate an accumulative error in a system, and improving the modeling precision of an arrow under an SLAM system; the object comprises a characteristic line and / or an arrow; the constraint relation comprises plane constraint between the characteristic lines and the arrows, parallel vertical constraint between the characteristic lines or coplanar constraint between the arrows, and the device is used for realizing the method. Compared with the prior art, the method has the advantages of higher mapping accuracy and the like.

Description

Technical Field

[0001] The present invention relates to the field of visual processing, and in particular to an arrow optimization modeling method and device. Background Art

[0002] The reconstruction of planar objects in visual SLAM often ignores their edge information. The generation of edge 3D feature lines is still calculated by triangulating the left and right eyes during initialization or by matching the two frames of images before and after the left eye during tracking. Compared with the overall data of the map, the information used is insufficient. At present, conventional SLAM algorithms need to load their CAD models in advance or reconstruct them through deep learning point cloud segmentation for the reconstruction of arrow objects. The process is tortuous and complicated. Considering that the arrow has complete and clear edge features, the three-dimensional model of the arrow can be successfully generated by extracting the edge line features. However, due to the influence of the cumulative error of the SLAM system, it is inaccurate to construct the arrow model only by simple triangulation of straight lines. Summary of the invention

[0003] The purpose of the present invention is to provide an arrow optimization modeling method and device in order to overcome the defects of the above-mentioned prior art, further perform large-scale nonlinear optimization by establishing constraint relationships between feature lines and arrows, between feature lines, and between arrow objects, adjust the posture to eliminate the cumulative error in the system, and improve the modeling accuracy of the arrow under the SLAM system.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] According to a first aspect of the present invention, there is provided an arrow optimization modeling method, which reconstructs an arrow model based on a constraint relationship between objects; the objects include feature lines and / or arrows; the constraint relationship includes a plane constraint between feature lines and arrows, a parallel vertical constraint between feature lines, or a coplanar constraint between arrows; wherein,

[0006] Reconstructing the arrow model according to the plane constraint between the feature line and the arrow includes: extracting candidate feature lines of the arrow and assigning semantic information to the candidate feature lines; converting the feature lines into 3D according to the semantic information to obtain 3D feature lines, and obtaining the Plücker coordinates of the 3D feature lines in the arrow coordinate system; performing line-surface constraint optimization, converting the Plücker coordinates of the 3D feature lines from the arrow coordinate system to the world coordinate system, and then converting the Plücker coordinates in the world coordinate system to the Plücker coordinates of the camera coordinate system, so that the 3D feature line and the arrow are located in the same plane, and connecting the 3D feature lines to form a closed figure, which is the reconstructed arrow; the semantic information includes a first attribute and a second attribute; the first attribute is the arrow ID, and the second attribute is the feature line number;

[0007] Reconstructing the arrow model according to the parallel vertical constraint between the feature lines includes: numbering all arrows in the regional scene, extracting the feature lines and using the arrow numbers as the third attribute; performing principal component analysis on the feature lines with the same third attribute based on the PCA algorithm to obtain the corresponding main and secondary directions of the arrows, and establishing the relationship between the direction vectors of all feature lines of the arrow and the main and secondary directions; updating the feature line Plücker coordinates according to the relationship; and reconstructing the arrow model according to the number;

[0008] Reconstructing the arrow model based on the coplanar constraint between the arrows includes: constructing a posture rotation matrix, wherein the posture rotation matrix is ​​a 3×3 matrix and the elements in the third row are [0, 0, 1] in sequence; obtaining the three-dimensional model of the first arrow in the regional map as the initial model, and identifying the second arrow based on the initial model; using the posture rotation matrix to restore the posture of the second arrow to achieve arrow modeling; the first arrow is all arrows of different specifications that appear for the first time in the recognition process, and the second arrow is an arrow with the same specification as the first arrow that appears repeatedly in the recognition process, and the first arrow is coplanar with the second arrow.

[0009] As a preferred technical solution, the method for obtaining the first attribute is: when multiple candidate feature lines are coplanar with the same arrow and can form a closed figure, the ID of the arrow is used as the first attribute of the multiple candidate feature lines;

[0010] The second attribute is obtained by numbering the feature lines to obtain the second attribute of the feature lines, wherein feature lines with the same first attribute have different numbers.

[0011] As a preferred technical solution, obtaining the Plücker coordinates of the characteristic line specifically includes:

[0012] The candidate feature lines are screened according to the first attribute, and the candidate feature lines with the same arrow ID are selected as feature lines;

[0013] Add the arrow and the binary edge of the feature line whose first attribute is the corresponding arrow ID to obtain the 3D feature line;

[0014] The 3D feature line is transformed from the world coordinate system to the visual coordinate system to obtain the 3D Plücker coordinates in the visual coordinate system;

[0015] According to the second attribute, the Plücker coordinates of a plurality of characteristic lines having the same first attribute are obtained.

[0016] As a preferred technical solution, the conversion of the Plücker coordinates is specifically: converting the Plücker coordinates from the arrow coordinate system to the camera coordinate system, and the conversion is based on the formula Carry out; among them L M is the coordinate in the camera coordinate system, is m in the Plücker coordinate systemM The transpose of a vector on an axis, is d in the Plücker coordinate system M The transpose of a vector on an axis.

[0017] As a preferred technical solution, the arrow is reconstructed according to the parallel vertical constraint between the feature lines, specifically including:

[0018] Add binary edges of feature lines with the same arrow ID to obtain 3D feature lines, represent the 3D feature lines using Plücker coordinates, and sample the 3D feature lines to obtain a three-dimensional space sample point set;

[0019] Construct a sample matrix according to the sample point set to calculate the column mean, obtain a standardized matrix based on the column mean, process the standardized matrix to obtain a covariance matrix, obtain the eigenvectors and eigenvalues ​​of the covariance matrix, and sort the eigenvalues;

[0020] The eigenvector direction corresponding to the largest eigenvalue is selected as the main direction, and the eigenvector direction corresponding to the second largest eigenvalue is selected as the secondary direction; the main direction is perpendicular to the secondary direction;

[0021] Construct a separate feature line set for all arrows based on the arrow ID attribute Determine the direction of all feature lines in each set. If their angle direction is consistent with the main direction, add them to the main direction feature line set of the arrow. If the angle direction is consistent with the secondary direction, it is added to the secondary direction feature line set of the arrow. In which i represents the i-th arrow and k represents k characteristic lines;

[0022] Update the set based on the parallel relationship between the main and minor directions and the feature lines and the perpendicular relationship between the main and minor directions. and The Plücker coordinates of the characteristic lines in the two sets are obtained, and the new Plücker coordinates of all characteristic lines in the two sets are obtained.

[0023] Specifically, the method for obtaining the new Plücker coordinates is:

[0024] Pair Collection and The characteristic lines in the set are numbered, and the characteristic lines in the same set are numbered differently;

[0025] Keep Collection The Z-axis coordinates of the two end points of all feature lines in C j,1 and C j,2 , and execute according to the feature line number,

[0026] Project all feature lines onto the xoy plane of the camera coordinate system to obtain feature line projections, and obtain the plane coordinates of the two end points of each feature line projection (a j,1,b j,1 ) and (a j,2 ,b j,2 ),

[0027] Based on the two end point plane coordinates, the Plücker coordinates of each corresponding characteristic line are updated as (a j,1 -a j,2 , b j,1 -b j,2 , C j,1 -C i,2 , p 1 , p 2 , p 3 ),in,

[0028]

[0029] a j,1 represents the x-axis coordinate of the starting point of feature line j, b j,1 The y-axis coordinate of the starting point of feature line j, C j,1 represents the Z-axis coordinate of the starting point of feature line j, a j,2 represents the x-axis coordinate of the end point of feature line j, b j,2 The y-axis coordinate of the end point of feature line j, C j,2 represents the Z-axis coordinate of the end point of feature line j, p 1 、p 2 and p 3 represents the directional component of the new Plücker coordinates of the characteristic line j;

[0030] Get the collection by the feature line number New Plücker coordinates of the remaining characteristic lines in;

[0031] Press Update Collection Method for obtaining the Plücker coordinates of all characteristic lines in the update set The Plücker coordinates of all feature lines in .

[0032] As a preferred technical solution, the method for constructing the posture rotation matrix is:

[0033] Construct the initial pose rotation matrix based on the relative pose γ between any two arrows selected f,g =(γ f ) -1 γ g , perform multiple fittings on the posture rotation matrices of the two arrows to obtain candidate posture rotation matrices, and select the first element and the second element of each candidate posture rotation matrix; where γ f,g represents the relative position of arrows f and g, γ f represents the position of arrow f, γ g Indicates the position of arrow g;

[0034] The error value of the corresponding candidate pose rotation matrix is ​​calculated based on the first element and the second element, and the error values ​​are sorted to screen out the minimum error value, and the candidate pose rotation matrix corresponding to the minimum error value is the pose rotation matrix.

[0035] As a preferred technical solution, the method for restoring the second arrow according to the posture rotation matrix is:

[0036] Acquire known information of a first arrow, the information including a position and a primary and secondary direction of the first arrow;

[0037] The first arrow plane is taken as the xoy plane, the main vector is set as the x-axis, the secondary vector is set as the y-axis, the result vector of the cross multiplication of the main vector and the secondary vector is taken as the z-axis, and the center point of the first arrow plane is taken as the origin;

[0038] Get the corresponding posture rotation matrix of the second arrow, and translate on the xoy plane or rotate around the Z axis based on the posture rotation matrix.

[0039] According to a second aspect of the present invention, there is provided an arrow optimization modeling device, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the above-mentioned arrow optimization modeling method when executing the program.

[0040] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described above when executed by a processor.

[0041] Compared with the prior art, the present invention optimizes the arrow object and the edge feature lines of the arrow, and proposes three optimization schemes, which further perform large-scale nonlinear optimization based on the constraint relationship between the feature line and the arrow, between the feature lines, and between the arrow objects, adjust the posture to eliminate the cumulative error in the system, and improve the accuracy of arrow reconstruction under the SLAM system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the method of the present invention;

[0043] Figure 2 This is the arrow reconstruction effect diagram in Example 1 of the present invention;

[0044] Figure 3 This is a schematic diagram of the main and secondary directions in Example 2 of the present invention;

[0045] Figure 4 This is an arrow reconstruction effect diagram without adding constraints in Example 2 of the present invention;

[0046] Figure 5 This is the arrow reconstruction effect diagram in Example 2 of the present invention;

[0047] Figure 6 This is the arrow reconstruction effect diagram in Example 3 of the present invention; DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The present invention reconstructs the arrow model based on the constraint relationship of the feature line and / or arrow, wherein the constraint relationship refers to the plane constraint between the feature line and the arrow, the parallel vertical constraint between the feature lines, or the coplanar constraint between the arrows. The method category and the flowchart of the corresponding method are as follows: Figure 1 shown.

[0051] Embodiment 1:

[0052] This embodiment provides an arrow optimization modeling method, which reconstructs the arrow model based on the plane constraint between the feature line and the arrow. The detailed steps are as follows:

[0053] S11, extracting candidate arrow feature lines, and assigning semantic information to the candidate feature lines, wherein the semantic information includes a first attribute and a second attribute, the first attribute is the arrow ID, and the second attribute is the feature line number;

[0054] S111, obtaining a first attribute: when multiple candidate feature lines are coplanar with the same arrow and can be surrounded by a closed figure, the ID of the arrow is used as the first attribute of the multiple candidate feature lines;

[0055] S112, obtaining a second attribute: numbering the feature lines to obtain a second attribute of the feature lines, wherein feature lines with the same first attribute have different numbers.

[0056] S12, converting the feature line into 3D according to the semantic information to obtain a 3D feature line, and obtaining the Plücker coordinates of the 3D feature line in the arrow coordinate system. The specific method for obtaining the Plücker coordinates is:

[0057] S121, screening candidate feature lines according to the first attribute, and selecting candidate feature lines with the same arrow ID as feature lines;

[0058] S122, adding an arrow and a binary edge of a feature line whose first attribute is a corresponding arrow ID to obtain a 3D feature line;

[0059] S123, performing relative pose transformation from the world coordinate system to the visual coordinate system on the 3D feature line to obtain the 3D Plücker coordinates in the visual coordinate system;

[0060] S124. Repeat steps S121 to S123 according to the second attribute to obtain the Plücker coordinates of multiple characteristic lines with the same first attribute.

[0061] S13, perform line-surface constraint optimization, convert the Plücker coordinates of the 3D feature line from the arrow coordinate system to the world coordinate system, and then convert the Plücker coordinates in the world coordinate system according to the formula is the Plücker coordinate of the camera coordinate system, so that the 3D feature line and the arrow are in the same plane, where L M is the coordinate in the camera coordinate system, is m in the Plücker coordinate system M The transpose of a vector on an axis, is d in the Plücker coordinate system M The transposition of the vector on the axis connects the 3D feature lines to form a closed figure, which is the reconstructed arrow.

[0062] The above method is used to reconstruct one of the arrows in the regional map under the SLAM system. The reconstruction effect is as follows: Figure 2 shown.

[0063] Embodiment 2:

[0064] This embodiment provides an arrow optimization modeling method, which reconstructs an arrow model according to the parallel vertical constraints between feature lines and specifically includes the following steps:

[0065] S21. Number all arrows in the regional scene, extract feature lines and use the arrow number as the third attribute.

[0066] S22, based on the PCA algorithm, principal component analysis is performed on the feature lines with the same third attribute to obtain the corresponding arrow major and minor directions, and the relationship between the direction vectors of all feature lines of the arrow and the major and minor directions is established. The specific steps include:

[0067] S221, adding binary edges of feature lines with the same arrow ID to obtain 3D feature lines, and representing the 3D feature lines using Plücker coordinates, sampling the 3D feature lines to obtain a three-dimensional space sample point set;

[0068] S222, constructing a sample matrix according to the sample point set to calculate the column mean, obtaining a standardized matrix based on the column mean, processing the standardized matrix to obtain a covariance matrix, obtaining the eigenvectors and eigenvalues ​​of the covariance matrix, and sorting the eigenvalues;

[0069] S223, select the eigenvector direction corresponding to the largest eigenvalue as the main direction, and the eigenvector direction corresponding to the second largest eigenvalue as the secondary direction. The main direction is perpendicular to the secondary direction. The effect is as follows: Figure 3 As shown;

[0070] S224. Build a separate feature line set for all arrows based on the arrow ID attribute Determine the direction of all feature lines in each set. If their angle direction is consistent with the main direction, add them to the main direction feature line set of the arrow. If the angle direction is consistent with the secondary direction, it is added to the secondary direction feature line set of the arrow. In which i represents the i-th arrow and k represents k characteristic lines;

[0071] S225, pair set and The characteristic lines in the set are numbered, and the characteristic lines in the same set are numbered differently;

[0072] S226, Reserved Set The Z-axis coordinates of the two end points of all feature lines in C j,1 and C j,2 , project all feature lines onto the xoy plane of the camera coordinate system to obtain feature line projections, and obtain the plane coordinates of the two end points of each feature line projection (a j,1 ,b j,1 ) and (a j,2 ,b j,2), based on the plane coordinates of the two end points, the Plücker coordinates of each corresponding feature line are updated as (a j,1 -a j,2 , b j,1 -b j,2 , C j,1 -C i,2 , p 1 , p 2 , p 3 ),in,

[0073]

[0074] a j,1 represents the x-axis coordinate of the starting point of feature line j, b j,1 The y-axis coordinate of the starting point of feature line j, C j,1 represents the Z-axis coordinate of the starting point of feature line j, a j,2 represents the x-axis coordinate of the end point of feature line j, b j,2 The y-axis coordinate of the end point of feature line j, C j,2 represents the Z-axis coordinate of the end point of feature line j, p 1 、p 2 and p 3 represents the directional component of the new Plücker coordinates of the characteristic line j;

[0075] S227, repeat step S226 according to the characteristic line number to obtain a set New Plücker coordinates of the remaining characteristic lines in;

[0076] S228, press update set Method for obtaining the Plücker coordinates of all characteristic lines in the update set The Plücker coordinates of all feature lines in .

[0077] S23, updating the Plücker coordinates of the characteristic lines according to the relationship; updating the Plücker coordinates of the characteristic lines according to the numbers, and connecting the characteristic lines after the Plücker coordinates are updated to form a closed figure, wherein the closed figure is the reconstructed arrow model;

[0078] The modeling method without adding constraints and the above method are used to reconstruct one of the arrows in the regional map under the SLAM system. The effect of the modeling method without adding constraints is as follows: Figure 4 As shown, Figure 5 This is the effect diagram of reconstructing the arrow model based on the parallel vertical constraints between feature lines; Figure 4 and Figure 5 It can be seen that adding the constraint effect between the feature lines can enhance the coplanar characteristics of the feature lines of the same arrow. Therefore, after using this method, the accuracy of reconstructing the arrow in the SLAM system is better.

[0079] Embodiment 3:

[0080] This embodiment provides an arrow optimization modeling method, which reconstructs an arrow model based on the coplanar constraint between arrows, and the specific steps include:

[0081] S31, constructing a posture rotation matrix, wherein the posture rotation matrix is ​​a 3×3 matrix and the elements in the third row are [0, 0, 1] in sequence, and the specific steps include:

[0082] S311, construct the initial posture rotation matrix, based on the relative posture γ between two randomly selected arrows f,g =(γ f ) -1 γ g , perform multiple fittings on the posture rotation matrices of the two arrows to obtain candidate posture rotation matrices, and select the first element and the second element of each candidate posture rotation matrix; where γ f,g represents the relative position of arrows f and g, γ f represents the position of arrow f, γ g Indicates the position of arrow g;

[0083] S312. Calculate the error value of the corresponding candidate posture rotation matrix based on the first element and the second element, sort the error values, and select the minimum error value, whose corresponding candidate posture rotation matrix is ​​the posture rotation matrix.

[0084] S32, obtaining a three-dimensional model of a first arrow in the regional map as an initial model, and identifying a second arrow based on the initial model, wherein the first arrow is all arrows of different specifications that appear for the first time in the identification process, and the second arrow is an arrow with the same specification as the first arrow that appears repeatedly in the identification process, and the first arrow is coplanar with the second arrow;

[0085] S33, using the posture rotation matrix to restore the second arrow posture, and realize arrow modeling, specifically:

[0086] S331, obtaining known information of a first arrow, the information including a position and a primary and secondary direction of the first arrow;

[0087] S332, taking the first arrow plane as the xoy plane, setting the main vector as the x-axis, setting the secondary vector as the y-axis, and taking the result vector of the cross multiplication of the main vector and the secondary vector as the z-axis, and taking the center point of the first arrow plane as the origin;

[0088] S333. Obtain the corresponding posture rotation matrix of the second arrow, and translate on the xoy plane or rotate around the Z axis based on the posture rotation matrix.

[0089] In this embodiment, the positions of the first straight arrow and the first turn arrow are obtained, and the second arrow in the area map is reconstructed using the above method. The final effect is as follows: Figure 6 shown.

[0090] The above is an introduction to a method embodiment. The following is a further explanation of the solution of the present invention through an apparatus embodiment.

[0091] Embodiment 4:

[0092] The present embodiment also provides an arrow optimization modeling device, including a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0093] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0094] The processing unit performs the various methods and processes described above, such as methods S11 to S13, S21 to S23 and / or S31 to 33. For example, in some embodiments, methods S11 to S13, S21 to S23 and / or S31 to 33 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of the methods S11 to S13, S21 to S23 and / or S31 to 33 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to perform methods S11 to S13, S21 to S23 and / or S31 to 33 by any other appropriate means (e.g., by means of firmware).

[0095] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0096] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0098] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0099] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for arrow optimization modeling, It is characterized in that The method reconstructs an arrow model based on a constraint relationship between objects; the objects include feature lines and / or arrows; the constraint relationship includes a plane constraint between feature lines and arrows, a parallel vertical constraint between feature lines, or a coplanar constraint between arrows; wherein, Reconstructing the arrow model according to the plane constraint between the feature line and the arrow includes: extracting candidate feature lines of the arrow and assigning semantic information to the candidate feature lines; converting the feature lines into 3D according to the semantic information to obtain 3D feature lines, and obtaining the Plücker coordinates of the 3D feature lines in the arrow coordinate system; performing line-surface constraint optimization, converting the Plücker coordinates of the 3D feature lines from the arrow coordinate system to the world coordinate system, and then converting the Plücker coordinates in the world coordinate system to the Plücker coordinates of the camera coordinate system, so that the 3D feature line and the arrow are located in the same plane, and connecting the 3D feature lines to form a closed figure, which is the reconstructed arrow; the semantic information includes a first attribute and a second attribute; the first attribute is the arrow ID, and the second attribute is the feature line number; Reconstructing the arrow model according to the parallel vertical constraint between the feature lines includes: numbering all arrows in the regional scene, extracting feature lines and using the arrow numbers as the third attribute; performing principal component analysis on feature lines with the same third attribute based on the PCA algorithm to obtain the corresponding main and secondary directions of the arrows, and establishing the relationship between the direction vectors of all feature lines of the arrows and the main and secondary directions; updating the feature line Plücker coordinates according to the relationship; updating the feature line Plücker coordinates according to the numbers, and connecting the feature lines after the Plücker coordinates are updated to form a closed figure, which is the reconstructed arrow model; Reconstructing the arrow model based on the coplanar constraint between the arrows includes: constructing a posture rotation matrix, wherein the posture rotation matrix is ​​a 3×3 matrix and the elements in the third row are [0, 0, 1] in sequence; obtaining the three-dimensional model of the first arrow in the regional map as the initial model, and identifying the second arrow based on the initial model; using the posture rotation matrix to restore the posture of the second arrow to achieve arrow modeling; the first arrow is all arrows of different specifications that appear for the first time in the recognition process, and the second arrow is an arrow with the same specification as the first arrow that appears repeatedly in the recognition process, and the first arrow is coplanar with the second arrow.

2. The arrow optimization modeling method according to claim 1, It is characterized in that The method for obtaining the first attribute is: when multiple candidate feature lines are coplanar with the same arrow and can form a closed figure, the ID of the arrow is used as the first attribute of the multiple candidate feature lines; The second attribute is obtained by numbering the feature lines to obtain the second attribute of the feature lines, wherein feature lines with the same first attribute have different numbers.

3. The arrow optimization modeling method according to claim 1, It is characterized in that The specific steps of obtaining the Plücker coordinates of the characteristic line include: The candidate feature lines are screened according to the first attribute, and the candidate feature lines with the same arrow ID are selected as feature lines; Add the arrow and the binary edge of the feature line whose first attribute is the corresponding arrow ID to obtain the 3D feature line; The 3D feature line is transformed from the world coordinate system to the visual coordinate system to obtain the 3D Plücker coordinates in the visual coordinate system; According to the second attribute, the Plücker coordinates of a plurality of characteristic lines having the same first attribute are obtained.

4. The arrow optimization modeling method according to claim 1, It is characterized in that The specific transformation of Plücker coordinates is as follows: the Plücker coordinates are transformed from the arrow coordinate system to the camera coordinate system. The transformation is based on the formula Carry out; among them L M is the coordinate in the camera coordinate system, is m in the Plücker coordinate system M The transpose of a vector on an axis, is d in the Plücker coordinate system M The transpose of a vector on an axis.

5. The arrow optimization modeling method according to claim 1, It is characterized in that Reconstruct arrows based on parallel and perpendicular constraints between feature lines, including: Add binary edges of feature lines with the same arrow ID to obtain 3D feature lines, represent the 3D feature lines using Plücker coordinates, and sample the 3D feature lines to obtain a three-dimensional space sample point set; Construct a sample matrix according to the sample point set to calculate the column mean, obtain a standardized matrix based on the column mean, process the standardized matrix to obtain a covariance matrix, obtain the eigenvectors and eigenvalues ​​of the covariance matrix, and sort the eigenvalues; The eigenvector direction corresponding to the largest eigenvalue is selected as the main direction, and the eigenvector direction corresponding to the second largest eigenvalue is selected as the secondary direction; the main direction is perpendicular to the secondary direction; Construct a separate feature line set for all arrows based on the arrow ID attribute Determine the direction of all feature lines in each set. If their angle direction is consistent with the main direction, add them to the main direction feature line set of the arrow. If the angle direction is consistent with the secondary direction, it is added to the secondary direction feature line set of the arrow. In which i represents the i-th arrow and k represents k characteristic lines; Update the set based on the parallel relationship between the main and minor directions and the feature lines and the perpendicular relationship between the main and minor directions. and The Plücker coordinates of the characteristic lines in the two sets are obtained, and the new Plücker coordinates of all characteristic lines in the two sets are obtained.

6. The arrow optimization modeling method according to claim 5, It is characterized in that The method to obtain the new Plücker coordinates is: Pair Collection and The characteristic lines in the set are numbered, and the characteristic lines in the same set are numbered differently; Keep Collection The Z-axis coordinates of the two end points of all feature lines in j,1 and c j,2 , and execute according to the feature line number, Project all feature lines onto the xoy plane of the camera coordinate system to obtain feature line projections, and obtain the plane coordinates of the two end points of each feature line projection (a j,1 ,b j,1 ) and (a j,2 ,b j,2 ), Based on the two end point plane coordinates, the Plücker coordinates of each corresponding characteristic line are updated as (a j,1 -a j,2 , b j,1 -b j,2 , C j,1 -C i,2 , p 1 , p 2 , p 3 ),in, a j,1 represents the x-axis coordinate of the starting point of feature line j, b j,1 The y-axis coordinate of the starting point of feature line j, C j,1 represents the Z-axis coordinate of the starting point of feature line j, a j,2 represents the x-axis coordinate of the end point of feature line j, b j,2 The y-axis coordinate of the end point of feature line j, C j,2 represents the Z-axis coordinate of the end point of feature line j, p 1 、p 2 and p 3 represents the directional component of the new Plücker coordinates of the characteristic line j; Get the collection by the feature line number New Plücker coordinates of the remaining characteristic lines in; Press Update Collection Method for obtaining the Plücker coordinates of all characteristic lines in the update set The Plücker coordinates of all feature lines in .

7. The arrow optimization modeling method according to claim 1, It is characterized in that The method for constructing the posture rotation matrix is: Construct the initial pose rotation matrix based on the relative pose γ between any two arrows selected f,g =(γ f ) -1 γ g , perform multiple fittings on the posture rotation matrices of the two arrows to obtain candidate posture rotation matrices, and select the first element and the second element of each candidate posture rotation matrix; where γ f,g represents the relative position of arrows f and g, γ f represents the position of arrow f, γ g Indicates the position of arrow g; The error value of the corresponding candidate pose rotation matrix is ​​calculated based on the first element and the second element, and the error values ​​are sorted to screen out the minimum error value, and the candidate pose rotation matrix corresponding to the minimum error value is the pose rotation matrix.

8. The arrow optimization modeling method according to claim 1, It is characterized in that The method for restoring the second arrow according to the posture rotation matrix is: Acquire known information of a first arrow, the information including a position, a main direction, and a secondary direction of the first arrow; The first arrow plane is taken as the xoy plane, the main direction vector is set as the x-axis, the secondary direction vector is set as the y-axis, the result vector of the main direction vector cross-multiplied by the secondary direction vector is taken as the z-axis, and the center point of the first arrow plane is taken as the origin; Get the corresponding posture rotation matrix of the second arrow, and translate on the xoy plane or rotate around the Z axis based on the posture rotation matrix.

9. An arrow optimization modeling device, It is characterized in that The method comprises a memory, a processor, and a program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images