Large-scale space axis measurement method and system based on computer vision, terminal and storage medium

Abstract

The invention relates to the technical field of computer vision measurement, and particularly provides a large-scale space axis measurement method and system based on computer vision, a terminal and a storage medium, and the method comprises the steps: constructing a space axis measurement environment which comprises a feature plate and a to-be-measured object containing a to-be-measured axis, a dot matrix is arranged on the characteristic plate; the method comprises the following steps: in a space axis measurement environment, acquiring images of a plurality of feature plates and a to-be-measured object in the same picture at different positions and angles by using image acquisition equipment to obtain a to-be-measured axis key point image set; and fitting the three-dimensional coordinates of the key points of the axis to be measured to obtain an axis equation, calculating the distance from the three-dimensional coordinates of the current position of the image acquisition equipment to the axis equation, and when the distance is shortest, the position of the image acquisition equipment is the position of the axis. The method does not need to design and install a clamp for the to-be-measured object, and is suitable for objects in various shapes.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision measurement technology, and specifically relates to a large-scale spatial axis measurement method, system, terminal and storage medium based on computer vision. Background Art

[0002] The spatial axis of an object refers to the spatial straight line connecting the key points on the surface of the object in a three-dimensional coordinate system (such as the central symmetry axis, the center line of a hole, and the normal of a plate). It is a reference line used to locate and describe the direction and position relationship of the object. Large-scale spatial axis measurement refers to measuring the three-dimensional spatial coordinates of the key points on the surface of a large-scale object, fitting the linear equation of the spatial axis based on the coordinates of the points, and finding the position of the axis in the real physical space. In real industrial production and military applications, it is often necessary to accurately measure and locate the spatial axis of large-scale objects. For example, in the assembly process of large-scale mechanical equipment such as aircraft and ships, accurate axis positioning can ensure the accurate installation and safe operation of parts. In the military field, the consistency measurement of the optoelectronic axis and the firepower axis of missiles, radars and other weapons and equipment is directly related to the combat accuracy and reliability of the system. In the construction of large-scale projects such as bridges and high-rise buildings, accurate measurement of the axis is also required to ensure structural safety and stability.

[0003] However, existing spatial axis measurement technologies often require fixtures designed based on the shape of the object being measured, with auxiliary equipment such as lasers and targets installed on the fixtures, or measurement equipment such as plumb bobs and theodolites. This not only results in high measurement equipment costs, but also a time-consuming and labor-intensive process, making operation difficult and susceptible to environmental influences on measurement accuracy. This is particularly problematic in uneven or dynamic environments, such as marine vessels and floating platforms. These technical limitations result in low measurement efficiency, high costs, and poor environmental adaptability, making them unable to meet the increasingly stringent requirements for high-precision and high-efficiency measurement in industrial production and defense and military applications. Summary of the Invention

[0004] In view of the problem in the prior art that a fixture needs to be designed according to the shape of the object to be measured, auxiliary equipment such as lasers and targets are installed on the fixture, or measuring equipment such as plumb bobs and theodolites are used, which has high requirements for the measurement environment and complicated operation, the present invention provides a large-scale spatial axis measurement method, system, terminal and storage medium based on computer vision. There is no need to design and install a fixture for the object to be measured, it is suitable for objects of various shapes and large sizes, is simple to operate and has low requirements for the measurement environment, and is suitable for measurement on uneven, rocking, floating and sinking platforms.

[0005] In a first aspect, the present invention provides a large-scale spatial axis measurement method based on computer vision, comprising: Constructing a spatial axis measurement environment, the spatial axis measurement environment comprising a feature plate and an object to be measured including the axis to be measured, wherein the feature plate is provided with a dot matrix; In a spatial axis measurement environment, an image acquisition device is used to capture several images of the feature plate and the object to be measured simultaneously in the same frame at multiple different positions and angles to obtain an image set of key points of the axis to be measured; Based on the preset feature plate detection algorithm, each dot in the feature plate of the key point image set of the axis to be measured is identified and numbered, and the two-dimensional coordinates of each dot are extracted. Based on the two-dimensional coordinates of each dot, the intrinsic parameter matrix, rotation matrix and translation vector of the image acquisition device are calculated through the rear intersection algorithm to obtain the image acquisition device parameters; The key points of the axis to be measured of the object to be measured are selected from the key point image of the axis to be measured, and the two-dimensional coordinates of the key points of the axis to be measured are obtained. Based on the two-dimensional coordinates of the key points of the axis to be measured, the intrinsic parameter matrix, the rotation matrix and the translation vector, the three-dimensional coordinates of the key points of the axis to be measured are obtained by the forward intersection algorithm; The three-dimensional coordinates of the key points of the axis to be measured are fitted to obtain the axis equation. The three-dimensional coordinates of the current position of the image acquisition device are obtained based on the image acquisition device parameters. The distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation is calculated. When the distance is shortest, the position of the image acquisition device is the position of the axis.

[0006] Furthermore, based on the preset feature plate detection algorithm, each dot in the feature plate of the key point image set of the axis to be measured is identified and numbered, and the two-dimensional coordinates of each dot are extracted, including: Binarize the image of the key points of the axis to be measured to obtain a binary image, detect the dots in the binary image, screen out the dots, and use the center coordinates of the dots as the coordinates of the dots; Perform topological relationship analysis on the dot coordinates to select the dot coordinates that match the dot arrangement of the feature plate. Number each dot according to the dot arrangement, and make the numbers of the same dots in each binary image consistent to obtain the two-dimensional coordinates of the dots in the binary image. .

[0007] Furthermore, based on the two-dimensional coordinates of each dot, the intrinsic parameter matrix, rotation matrix, and translation vector of the image acquisition device are calculated using the resection algorithm to obtain the image acquisition device parameters, including: Establish the plane where the feature plate is located as plane, all the points The coordinates are set to 0; Based on the dot array arrangement of the feature plate, the three-dimensional coordinates of each dot are obtained ; The three-dimensional coordinates of the point , based on the three-dimensional coordinates of the points By formula Perform iterative calculation to minimize the reprojection error, where n represents the total number of dots and i represents the i-th dot; Using the formula , calculate the intrinsic parameter matrix K, rotation matrix R and translation vector t of the image acquisition device, and record the intrinsic parameter matrix K, rotation matrix R and translation vector t of the image acquisition device as image acquisition device parameters.

[0008] Furthermore, the key points of the axis to be measured of the object to be measured are selected from the key point image of the axis to be measured to obtain the two-dimensional coordinates of the key points of the axis to be measured. Based on the two-dimensional coordinates of the key points of the axis to be measured, the intrinsic parameter matrix, the rotation matrix and the translation vector, the three-dimensional coordinates of the key points of the axis to be measured are obtained by the forward intersection algorithm, including: In the key point image set of the axis to be measured, obtain the two-dimensional coordinates of the key point of the axis to be measured in each key point image of the axis to be measured ; Set the three-dimensional coordinates of the key points of the axis to be measured to , combined with the intrinsic parameter matrix K, rotation matrix R and translation vector t, construct the linear projection equation corresponding to each key point image of the axis to be measured:

[0009] in, is the scale factor of the depth of field, m is the number of key point images of the axis to be measured, and the projection equation is expanded and the depth scale factor is eliminated to obtain two linear equations:

[0010] in, 、 and The rotation matrix OK, represents the displacement in the x-axis direction, represents the displacement on the y-axis, represents the displacement in the z-axis; According to each key point image of the axis to be measured, two linear equations are obtained. The key point images of the axis to be measured are obtained equations, and all equations are formed into a coefficient matrix , solve The least squares solution of the axis to be measured is used to obtain the three-dimensional coordinates of the key points .

[0011] Further, the three-dimensional coordinates of the key points of the axis to be measured are fitted to obtain an axis equation, the three-dimensional coordinates of the current position of the image acquisition device are obtained based on the parameters of the image acquisition device, and the distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation is calculated; when the distance is the shortest, the position of the image acquisition device is the position of the axis, comprising: calculating the three-dimensional coordinates of the image acquisition device: wherein R is a rotation matrix, and t is a translation vector, is the transpose matrix of the rotation matrix R; based on the three-dimensional coordinates of the image acquisition device, the distance between the current position of the image acquisition device and the axis is calculated by using the point-line distance equation; when the distance is 0, the position of the camera is the position of the axis; Further, it further comprises: According to the geometric relationship between the current three-dimensional space position of the image acquisition device and the fitted space axis, the spatial offset information of the image acquisition device relative to the space axis is calculated, and the motion guide information is generated based on the offset information to assist the image acquisition device to move along the preset path to the space axis direction until its position coincides with the space axis, thereby realizing the spatial automatic alignment or manual assisted alignment of the image acquisition device; wherein the spatial offset information comprises the shortest distance between the image acquisition device and the space axis and the offset direction, the shortest distance is obtained based on the spatial geometric distance calculation formula of point to straight line, and the offset direction is represented by the direction vector corresponding to the nearest point from the current position of the image acquisition device to the space axis; The motion guide information comprises control parameters for describing the moving direction, path length and angle adjustment amount of the image acquisition device, for driving the external execution mechanism to realize the spatial position adjustment of the image acquisition device.

[0012] In a second aspect, the present application provides a large-scale space axis measurement system based on computer vision, comprising: An environment construction module is configured to construct a space axis measurement environment, wherein the space axis measurement environment comprises a feature plate and an object to be measured containing an axis to be measured, and the feature plate is provided with a circular dot array; An image acquisition module is configured to acquire a plurality of images in which the feature plate and the object to be measured exist in the same frame at different positions and angles in the space axis measurement environment by using an image acquisition device, to obtain an image set of key points of the axis to be measured; A parameter calculation module is configured to identify and number each circular dot in the feature plate of the image set of key points of the axis to be measured based on a preset feature plate detection algorithm, to extract the two-dimensional coordinates of each circular dot, and to obtain the parameters of the image acquisition device by calculating the intrinsic matrix, the rotation matrix and the translation vector of the image acquisition device based on the two-dimensional coordinates of each circular dot through the resection algorithm. The coordinate calculation module is configured to select the axis key points of the object to be measured in the image of the axis key points of the object to be measured, obtain two-dimensional coordinates of the axis key points of the object to be measured, and obtain three-dimensional coordinates of the axis key points of the object to be measured based on the two-dimensional coordinates of the axis key points of the object to be measured, an intrinsic matrix, a rotation matrix and a translation vector through a forward intersection algorithm.

[0013] The axis calculation module is configured to fit an axis equation based on the three-dimensional coordinates of the axis key points of the object to be measured, obtain three-dimensional coordinates of a current position of the image acquisition device based on parameters of the image acquisition device, and calculate a distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation, so that the position of the image acquisition device is the position of the axis when the distance is the shortest.

[0014] Further, the parameter calculation module comprises: The circle point coordinate calculation unit is configured to perform binaryzation processing on the image of the axis key points of the object to be measured to obtain a binary image, detect circle points in the binary image, and screen out the circle points, so as to take the center coordinates of the circle points as the circle point coordinates. The circle point numbering unit is configured to analyze the topological relationship of the circle point coordinates, screen out the circle point coordinates conforming to the arrangement mode of the circle points of the feature plate, mark a number for each circle point according to the arrangement mode of the circle points, and make the numbers of the same circle points consistent in each binary image, so as to obtain two-dimensional coordinates of the circle points in the binary image.

[0015] In a third aspect, a terminal is provided, comprising: A processor and a memory, wherein The memory is configured to store a computer program, The processor is configured to call and run the computer program from the memory, so that the terminal executes the method of the terminal described above.

[0016] In a fourth aspect, a computer storage medium is provided, and the computer readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the method described in the above aspects.

[0017] The beneficial effects of the present application are that the computer vision-based large-scale space axis measurement method, system, terminal and storage medium provided by the present application can realize the simultaneous capture of the feature plate and the object key points in the image by constructing a space axis measurement environment containing a feature plate and an object to be measured, and can realize efficient and high-precision measurement of the space axis by combining the image capturing of the image acquisition device and the computer vision algorithm processing. First, the feature plate provided with a circular dot array is used, and the two-dimensional coordinates of each circular dot in the image are identified by a preset feature plate detection algorithm, and then the internal parameter matrix, rotation matrix and translation vector of the image acquisition device are accurately solved based on the resection algorithm, so that the corresponding relationship between the image and the three-dimensional space can be quickly established, the cumbersome operation of arranging the measurement environment and leveling the device in the traditional method is omitted, the measurement efficiency is greatly improved, and the measurement time can be shortened to within ten minutes. Secondly, the axis key point image coordinates of the object to be measured are selected by manual assistance, and the three-dimensional coordinates of the key points are calculated by combining the forward intersection algorithm, which effectively avoids the axis deviation caused by automatic recognition error, ensures the accuracy of key point positioning and the stability of axis fitting, and the angle measurement accuracy can be within 0.15°. Further, the current space position of the image acquisition device is calculated using the known camera parameters, and the positional relationship between the image acquisition device and the axis is determined by combining the point-line distance formula, which not only realizes the accurate determination of the position of the space axis, but also dynamically guides the image acquisition device to move to the position of the axis or any angle, any distance space point, significantly improving the application flexibility of the system. This method does not require the installation of fixtures and the adjustment of the platform state, and is suitable for complex environments such as uneven, floating and swinging, reducing the dependence on the ability of the operator and the measurement site, and having good scene adaptability. At the same time, only an ordinary camera and a computer processing device are needed to complete the measurement task, avoiding the use of traditional theodolites, gyroscopes and total stations, etc. high-cost devices, reducing the system cost, improving the universality and reusability of the device, and having significant technical advantages and wide engineering application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0019] Figure 1 is a schematic flowchart of the method of one embodiment of the present application.

[0020] Figure 2 is a schematic diagram of image acquisition of one embodiment of the present application.

[0021] Figure 31 is a schematic diagram of using a point-line distance equation to calculate the distance between the current camera position and the axis to be measured according to an embodiment of the present invention.

[0022] Figure 4 FIG. 4 is a schematic block diagram of a system according to an embodiment of the present invention.

[0023] Figure 5 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0026] The large-scale spatial axis measurement method based on computer vision provided in an embodiment of the present invention is executed by a computer device. Accordingly, the large-scale spatial axis measurement system based on computer vision runs in the computer device.

[0027] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject can be a large-scale spatial axis measurement system based on computer vision. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0028] To facilitate understanding of the present invention, the large-scale spatial axis measurement method based on computer vision provided by the present invention is further described below based on the principle of the large-scale spatial axis measurement method based on computer vision provided by the present invention.

[0029] Specifically, such as Figure 1 As shown, the large-scale spatial axis measurement method based on computer vision includes: S1. Construct a spatial axis measurement environment, wherein the spatial axis measurement environment includes a feature plate and an object to be measured including an axis to be measured, wherein a dot matrix is ​​provided on the feature plate.

[0030] Specifically, a 3D reconstruction feature plate is placed near the object to be measured, keeping it fixed in place. When a camera is used to capture an image of the object, the feature plate and key points appear simultaneously in the camera image. The camera is then connected to a data processing computer, where the images captured by the camera can be transmitted to the computer in real time.

[0031] S2. In a spatial axis measurement environment, an image acquisition device is used to capture several images of the feature plate and the object to be measured simultaneously in the same screen at multiple positions and angles to obtain an image set of key points of the axis to be measured.

[0032] Specifically, the image acquisition device can be a camera, and the camera can be an ordinary micro single camera, and the image captured by the camera can be transmitted to the computer in real time. The feature plate adopts a dot checkerboard grid and is placed near the axis to be measured. The size, number and arrangement of the dots are known, and it is used to establish a three-dimensional coordinate system and to locate the key points of the camera and the axis to be measured. Several key points are selected on the spatial axis to be measured, and the key points are used to fit the spatial axis. The data processing computer is installed with a spatial axis measurement program for interacting with the camera, obtaining the images captured by the camera in real time to reconstruct the spatial axis, and calculating the current spatial position of the camera and the distance between the camera and the axis in real time.

[0033] S3. Based on the preset feature plate detection algorithm, each dot in the feature plate of the key point image set of the measured axis is identified and numbered, the two-dimensional coordinates of each dot are extracted, and the internal parameter matrix, rotation matrix and translation vector of the image acquisition device are calculated based on the two-dimensional coordinates of each dot through the rear intersection algorithm to obtain the image acquisition device parameters.

[0034] The key point image of the axis to be measured is binarized to obtain a binary image. The dots in the binary image are detected and screened out. The center coordinates of the dots are used as the dot coordinates. The topological relationship of the dot coordinates is analyzed to screen out the dot coordinates that meet the dot arrangement of the feature plate. Each dot is numbered according to the dot arrangement, and the numbers of the same dots in each binary image are made consistent to obtain the two-dimensional coordinates of the dots in the binary image. .

[0035] Establish the plane where the feature plate is located as plane, all the points The coordinates are set to 0. Based on the dot array arrangement of the feature plate, the three-dimensional coordinates of each dot are obtained. .

[0036] The three-dimensional coordinates of the point , based on the three-dimensional coordinates of the points By formula Perform iterative calculations to minimize the reprojection error, where n represents the total number of dots and i represents the i-th dot.

[0037] Using the formula , calculate the intrinsic parameter matrix K, rotation matrix R and translation vector t of the image acquisition device, and record the intrinsic parameter matrix K, rotation matrix R and translation vector t of the image acquisition device as image acquisition device parameters.

[0038] S4. Select the key points of the axis to be measured of the object to be measured in the key point image of the axis to be measured, obtain the two-dimensional coordinates of the key points of the axis to be measured, and obtain the three-dimensional coordinates of the key points of the axis to be measured through the forward intersection algorithm based on the two-dimensional coordinates of the key points of the axis to be measured, the internal parameter matrix, the rotation matrix and the translation vector.

[0039] In the key point image set of the axis to be measured, obtain the two-dimensional coordinates of the key point of the axis to be measured in each key point image of the axis to be measured . Set the three-dimensional coordinates of the key points of the axis to be measured to , combined with the intrinsic parameter matrix K, rotation matrix R and translation vector t, construct the linear projection equation corresponding to each key point image of the axis to be measured:

[0040] in, is the scale factor of the depth of field, m is the number of key point images of the axis to be measured, and the projection equation is expanded and the depth scale factor is eliminated to obtain two linear equations:

[0041] in, 、 and The rotation matrix OK, represents the displacement in the x-axis direction, represents the displacement on the y-axis, Represents the displacement on the z-axis.

[0042] According to each key point image of the axis to be measured, two linear equations are obtained. The key point images of the axis to be measured are obtained equations, and all equations are formed into a coefficient matrix , solve The least squares solution of the axis to be measured is used to obtain the three-dimensional coordinates of the key points .

[0043] S5, fitting the three-dimensional coordinates of the axis key points to obtain an axis equation, obtaining the three-dimensional coordinates of the current position of the image acquisition device based on the image acquisition device parameters, and calculating the distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation, when the distance is the shortest, the position of the image acquisition device is the position of the axis.

[0044] The three-dimensional coordinates of the image acquisition device are calculated as follows: wherein R is a rotation matrix and t is a translation vector. Based on the three-dimensional coordinates of the image acquisition device, the distance between the current position of the image acquisition device and the axis is calculated using the point-line distance equation, and when the distance is 0, the position of the camera is the position of the axis.

[0045] The direction vector of the camera lens orientation, i.e. the direction of the lens optical axis, can be calculated using the following formula:

[0046] After calculating the camera lens orientation, the angle between the lens optical axis and the axis can be calculated using the straight line angle equation, and when the angle is 0, it means that the lens optical axis is parallel to the axis.

[0047] Specifically, from Figure 2 It can be seen that the feature board has a circular dot chessboard, and the circular dots are uniformly sized and arranged at the same interval in the plane. The feature board is placed near the axis to be measured. The camera uses a common micro single camera, which can clearly capture the feature board and the axis key points when measuring large-scale spatial axes at a long distance. The data processing computer is the processing center of the camera image, which can complete the interaction with the camera, real-time acquisition of the camera image, manual selection of the axis key points, calculation of the three-dimensional reconstruction of the spatial axis and the positioning of the camera, and indication of the camera moving to the axis position.

[0048] In Figure 3 , after calculating the three-dimensional coordinates of the camera, the distance between the current position of the camera and the spatial axis can be calculated using the point-line distance equation, and when the distance is 0, the position of the camera is the position of the spatial axis; connecting the camera and a certain axis key point into a straight line, the angle between the current position of the camera and the spatial axis can be calculated using the straight line angle equation, so that the position of the axis can be found, and the position with an arbitrary angle to the spatial axis can also be found. After calculating the straight line equation of the lens optical axis, the angle between the lens optical axis and the spatial axis can be calculated using the straight line angle equation, and when the angle is 0, it means that the lens optical axis is parallel to the spatial axis.

[0049] In one embodiment, it further includes: According to a geometric relationship between a current three-dimensional space position of the image acquisition device and the fitted space axis, spatial offset information of the image acquisition device relative to the space axis is calculated, and motion guidance information is generated based on the offset information to assist the image acquisition device to move along a preset path in a direction of the space axis until a position of the image acquisition device coincides with the space axis, so that spatial automatic alignment or manual assisted alignment of the image acquisition device is realized. The spatial offset information includes a shortest distance between the image acquisition device and the space axis and an offset direction, the shortest distance is obtained based on a point-to-line spatial geometric distance calculation formula, and the offset direction is represented by a direction vector corresponding to a direction from a current position of the image acquisition device to a nearest point on the space axis. The motion guidance information includes control parameters for describing a moving direction, a moving path length and an angle adjustment amount of the image acquisition device, and is used to drive an external execution mechanism to realize spatial position adjustment of the image acquisition device.

[0050] In some embodiments, the computer vision-based large-scale space axis measurement system can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the computer vision-based large-scale space axis measurement system can be stored in the memory of the computer device and executed by at least one processor to perform (see Figure 1 Description) the functions of the computer vision-based large-scale space axis measurement.

[0051] In this embodiment, the computer vision-based large-scale space axis measurement system can be divided into a plurality of functional modules according to the functions performed by the system, as shown in Figure 4 The functional modules of the system 200 can include an environment construction module 210, an image acquisition module 220, a parameter calculation module 230, a coordinate calculation module 240 and an axis calculation module 250. The module referred to in the present application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0052] The environment construction module is used to construct a space axis measurement environment, the space axis measurement environment includes a feature plate and a to-be-measured object containing a to-be-measured axis, and the feature plate is provided with a circular dot array; The image acquisition module is used to acquire a plurality of images in which the feature plate and the to-be-measured object exist in the same frame at different positions and angles in the space axis measurement environment by using the image acquisition device, to obtain a to-be-measured axis key point image set; The parameter calculation module is configured to identify and number each circle point in the feature plate of the image set of the axis key point of the object to be measured based on a preset feature plate detection algorithm, extract two-dimensional coordinates of each circle point, and calculate an intrinsic matrix, a rotation matrix and a translation vector of the image acquisition device based on the two-dimensional coordinates of each circle point by a resection algorithm to obtain the parameters of the image acquisition device. The coordinate calculation module is configured to select the axis key point of the object to be measured in the image of the axis key point of the object to be measured, obtain two-dimensional coordinates of the axis key point of the object to be measured, and obtain three-dimensional coordinates of the axis key point of the object to be measured based on the two-dimensional coordinates of the axis key point of the object to be measured, the intrinsic matrix, the rotation matrix and the translation vector by a forward intersection algorithm.

[0053] The axis calculation module is configured to fit an axis equation based on the three-dimensional coordinates of the axis key point of the object to be measured, obtain three-dimensional coordinates of a current position of the image acquisition device based on the parameters of the image acquisition device, calculate a distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation, and determine that the position of the image acquisition device is the position of the axis when the distance is the shortest.

[0054] Optionally, as an embodiment of the present application, the parameter calculation module comprises: The circle point coordinate calculation unit is configured to perform binaryzation processing on the image of the axis key point of the object to be measured to obtain a binary image, detect circle points in the binary image, and select the circle points to take the center coordinates of the circle points as the circle point coordinates. The circle point numbering unit is configured to analyze the topological relationship of the circle point coordinates, select the circle point coordinates in line with the arrangement mode of the circle points of the feature plate, mark a number for each circle point according to the arrangement mode of the circle points, and make the numbers of the same circle points consistent in each binary image to obtain the two-dimensional coordinates of the circle points in the binary image.

[0055] Figure 5 A structure schematic diagram of a terminal 300 is provided for an embodiment of the present application, and the terminal 300 can be used to execute the computer vision-based large-scale space axis measurement method provided by the embodiment of the present application.

[0056] The terminal 300 can include a processor 310, a memory 320 and a communication unit 330. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation on the present application, which can be a bus structure or a star structure, and can include more or fewer components than shown in the figure, or combine certain components, or different component arrangements.

[0057] The memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 can execute part or all of the steps in the following method embodiments.

[0058] The processor 310 is the control center of the storage terminal, connects various parts of the entire electronic terminal through various interfaces and lines, executes the software programs and / or modules stored in the memory 320, and calls the data stored in the memory, so as to execute various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, can be composed of a single packaged IC, or can be composed of a plurality of packaged ICs with the same function or different functions. For example, the processor 310 can only include a central processing unit (CPU). In the embodiments of the present application, the CPU can be a single operation core or can include multiple operation cores.

[0059] The communication unit 330 is used to establish a communication channel, so that the storage terminal can communicate with other terminals. The communication unit 330 receives user data sent by other terminals or sends user data to other terminals.

[0060] The present application also provides a computer storage medium, wherein the computer storage medium can store a program, and the program can include part or all of the steps in the embodiments provided by the present application when executed. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0061] Therefore, the application realizes the simultaneous capture of the feature plate and the key points of the object in the image by constructing a space axis measurement environment containing the feature plate and the object to be measured, combining the image acquisition device to shoot the image and the computer vision algorithm processing, and can realize the efficient and high-precision measurement of the space axis. First, the feature plate provided with a dot matrix of circles is used, and the two-dimensional coordinates of each circle point in the image are identified through a preset feature plate detection algorithm, and then the internal parameter matrix, rotation matrix and translation vector of the image acquisition device are accurately solved based on the resection algorithm, so that the corresponding relationship between the image and the three-dimensional space can be quickly established, the cumbersome operation of arranging the measurement environment and leveling the device in the traditional method is omitted, the measurement efficiency is greatly improved, and the measurement time can be shortened to within ten minutes. Secondly, the image coordinates of the key points of the axis of the object to be measured are selected by manual assistance, and the three-dimensional coordinates of the key points are calculated by combining the forward intersection algorithm, which effectively avoids the axis deviation caused by automatic recognition error, ensures the accuracy of key point positioning and the stability of axis fitting, and the angle precision can be within 0.15°. Further, the current space position of the image acquisition device is calculated by using the known camera parameters, and the positional relationship between the image acquisition device and the axis is determined by combining the point-line distance formula, which can not only realize the accurate determination of the position of the space axis, but also dynamically guide the image acquisition device to move to the position of the axis or any angle, any distance space point, significantly improving the application flexibility of the system. The method does not need to install a fixture and adjust the platform state, and is suitable for complex environments such as uneven, floating and swinging, reduces the dependence on the ability of the operator and the measurement site, and has good scene adaptability. At the same time, only an ordinary camera and a computer processing device are needed to complete the measurement task, avoiding the use of traditional theodolites, gyroscopes and total stations and other high-cost devices, reducing the system cost, improving the universality and reusability of the device, and having significant technical advantages and wide engineering application prospect. The technical effects that can be achieved by the embodiment can be referred to the description in the foregoing, which will not be repeated here.

[0062] Those skilled in the art can clearly understand that the technology in the embodiment of the application can be realized by means of software and necessary general hardware platform. Based on such understanding, the technical solutions in the embodiments of the application can be embodied in the form of a software product, which is stored in a storage medium such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage media that can store program codes, including a plurality of instructions for causing a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the method described in the embodiments of the application.

[0063] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0064] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, and can be electrical, mechanical or other forms.

[0065] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0066] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0067] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and such modifications or substitutions shall be within the scope of the present invention. Any person skilled in the art who is familiar with the present invention may easily conceive of changes or substitutions within the technical scope disclosed in the present invention, and such changes or substitutions shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A large-scale spatial axis measurement method based on computer vision, characterized in that: include: Constructing a spatial axis measurement environment, the spatial axis measurement environment comprising a feature plate and an object to be measured including the axis to be measured, wherein the feature plate is provided with a dot matrix; In a spatial axis measurement environment, an image acquisition device is used to capture several images of the feature plate and the object to be measured simultaneously in the same frame at multiple different positions and angles to obtain an image set of key points of the axis to be measured; Based on the preset feature plate detection algorithm, each dot in the feature plate of the key point image set of the axis to be measured is identified and numbered, and the two-dimensional coordinates of each dot are extracted. Based on the two-dimensional coordinates of each dot, the intrinsic parameter matrix, rotation matrix and translation vector of the image acquisition device are calculated through the rear intersection algorithm to obtain the image acquisition device parameters; The key points of the axis to be measured of the object to be measured are selected from the key point image of the axis to be measured, and the two-dimensional coordinates of the key points of the axis to be measured are obtained. Based on the two-dimensional coordinates of the key points of the axis to be measured, the intrinsic parameter matrix, the rotation matrix and the translation vector, the three-dimensional coordinates of the key points of the axis to be measured are obtained by the forward intersection algorithm; The three-dimensional coordinates of the key points of the axis to be measured are fitted to obtain the axis equation. The three-dimensional coordinates of the current position of the image acquisition device are obtained based on the image acquisition device parameters. The distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation is calculated. When the distance is shortest, the position of the image acquisition device is the position of the axis.

2. The method according to claim 1, characterized in that Based on the preset feature plate detection algorithm, each dot in the feature plate of the key point image set of the axis to be measured is identified and numbered, and the two-dimensional coordinates of each dot are extracted, including: Binarize the image of the key points of the axis to be measured to obtain a binary image, detect the dots in the binary image, screen out the dots, and use the center coordinates of the dots as the coordinates of the dots; Perform topological relationship analysis on the dot coordinates to select the dot coordinates that match the dot arrangement of the feature plate. Number each dot according to the dot arrangement, and make the numbers of the same dots in each binary image consistent to obtain the two-dimensional coordinates of the dots in the binary image. .

3. The method according to claim 1, characterized in that Based on the two-dimensional coordinates of each point, the intrinsic parameter matrix, rotation matrix, and translation vector of the image acquisition device are calculated using the rear intersection algorithm to obtain the image acquisition device parameters, including: Establish the plane where the feature plate is located as plane, all the points The coordinates are set to 0; Based on the dot array arrangement of the feature plate, the three-dimensional coordinates of each dot are obtained ; The three-dimensional coordinates of the point Perform two-dimensional projection to obtain two-dimensional projection coordinates , based on the three-dimensional coordinates of the points and two-dimensional projection coordinates , get two-dimensional projection at different angles, through the formula Perform iterative calculation to minimize the reprojection error, where n represents the total number of dots and i represents the i-th dot; Using the formula , calculate the intrinsic parameter matrix K, rotation matrix R and translation vector t of the image acquisition device, and record the intrinsic parameter matrix K, rotation matrix R and translation vector t of the image acquisition device as image acquisition device parameters.

4. The method according to claim 1, wherein The key points of the axis to be measured of the object to be measured are selected from the key point image of the axis to be measured, and the two-dimensional coordinates of the key points of the axis to be measured are obtained. Based on the two-dimensional coordinates of the key points of the axis to be measured, the intrinsic parameter matrix, the rotation matrix and the translation vector, the three-dimensional coordinates of the key points of the axis to be measured are obtained by the forward intersection algorithm, including: In the key point image set of the axis to be measured, obtain the two-dimensional coordinates of the key point of the axis to be measured in each key point image of the axis to be measured ; Set the three-dimensional coordinates of the key points of the axis to be measured to , combined with the intrinsic parameter matrix K, rotation matrix R and translation vector t, construct the linear projection equation corresponding to each key point image of the axis to be measured: in, is the scale factor of the depth of field, m is the number of key point images of the axis to be measured, and the projection equation is expanded and the depth scale factor is eliminated to obtain two linear equations: in, 、 and The rotation matrix OK, represents the displacement in the x-axis direction, represents the displacement on the y-axis, represents the displacement in the z-axis; According to each key point image of the axis to be measured, two linear equations are obtained. The key point images of the axis to be measured are obtained equations, and all equations are formed into a coefficient matrix , solve The least squares solution of the axis to be measured is used to obtain the three-dimensional coordinates of the key points .

5. The method according to claim 1, wherein The axis equation is obtained by fitting the three-dimensional coordinates of the key points of the axis to be measured. The three-dimensional coordinates of the current position of the image acquisition device are obtained based on the parameters of the image acquisition device. The distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation is calculated. When the distance is the shortest, the position of the image acquisition device is the position of the axis, including: Calculate the three-dimensional coordinates of the image acquisition device: , where R is the rotation matrix, t is the translation vector, is the transposed matrix of the rotation matrix R; Based on the three-dimensional coordinates of the image acquisition device, the distance between the current image acquisition device position and the axis is calculated using the point-line distance equation. When the distance is 0, the camera position is the axis position.

6. The method according to claim 1, characterized in that Also includes: Calculating spatial offset information of the image acquisition device relative to the fitted spatial axis based on a geometric relationship between the current three-dimensional spatial position of the image acquisition device and the fitted spatial axis, and generating motion guidance information based on the offset information to assist the image acquisition device in moving along a preset path toward the spatial axis until its position coincides with the spatial axis, thereby achieving automatic spatial alignment or manually assisted alignment of the image acquisition device; The spatial offset information includes the shortest distance between the image acquisition device and the spatial axis and the offset direction. The shortest distance is obtained based on a spatial geometric distance calculation formula from a point to a line. The offset direction is represented by a direction vector corresponding to the closest point on the spatial axis from the current position of the image acquisition device. The motion guidance information includes control parameters for describing the direction in which the image acquisition device should move, the length of the moving path, and the amount of angle adjustment, and is used to drive an external actuator to adjust the spatial position of the image acquisition device.

7. A large-scale spatial axis measurement system based on computer vision, characterized in that: include: An environment construction module is used to construct a spatial axis measurement environment, wherein the spatial axis measurement environment includes a feature plate and an object to be measured including the axis to be measured, wherein a dot matrix is ​​provided on the feature plate; An image acquisition module is used to acquire, in a spatial axis measurement environment, a plurality of images of the feature plate and the object to be measured being simultaneously present in the same frame at multiple positions and angles using an image acquisition device, thereby obtaining an image set of key points of the axis to be measured; A parameter calculation module is used to identify and number each dot in the feature plate of the key point image set of the axis to be measured based on a preset feature plate detection algorithm, extract the two-dimensional coordinates of each dot, and calculate the intrinsic parameter matrix, rotation matrix and translation vector of the image acquisition device based on the two-dimensional coordinates of each dot through the back intersection algorithm to obtain the image acquisition device parameters; The coordinate calculation module is used to select the key points of the axis to be measured of the object to be measured in the key point image of the axis to be measured, obtain the two-dimensional coordinates of the key points of the axis to be measured, and obtain the three-dimensional coordinates of the key points of the axis to be measured through the forward intersection algorithm based on the two-dimensional coordinates of the key points of the axis to be measured, the internal parameter matrix, the rotation matrix and the translation vector. The axis calculation module is used to fit the three-dimensional coordinates of the key points of the axis to be measured to obtain the axis equation, obtain the three-dimensional coordinates of the current position of the image acquisition device based on the image acquisition device parameters, and calculate the distance from the three-dimensional coordinates of the current position of the image acquisition device to the axis equation. When the distance is shortest, the position of the image acquisition device is the position of the axis.

8. The system according to claim 7, characterized in that The parameter calculation module includes: The dot coordinate calculation unit is used to perform binarization processing on the key point image of the axis to be measured to obtain a binary image, detect the dots in the binary image, screen out the dots, and use the center coordinates of the dots as the dot coordinates; The dot numbering unit is used to perform topological relationship analysis on the dot coordinates, screen out the dot coordinates that conform to the dot arrangement of the feature plate, mark and number each dot according to the arrangement of the dots, and make the numbers of the same dots in each binary image consistent, so as to obtain the two-dimensional coordinates of the dots in the binary image.

9. A terminal, characterized in that: include: A memory for storing a large-scale spatial axis measurement program based on computer vision; A processor is used to implement the steps of the large-scale space axis measurement method based on computer vision as described in any one of claims 1 to 6 when executing the large-scale space axis measurement program based on computer vision.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a large-scale spatial axis measurement program based on computer vision, and when the large-scale spatial axis measurement program based on computer vision is executed by the processor, the steps of the large-scale spatial axis measurement method based on computer vision as described in any one of claims 1-6 are implemented.

Images