Space container pose measuring device, method, equipment, medium and product

Abstract

The invention discloses a space container pose measurement device, method and equipment, a medium and a product, and relates to the field of three-dimensional pose measurement, the device comprises an image shooting device, a structured light generation device and a data processing terminal, the structured light generation device is used for emitting multi-line and single-line structured light to a space container to be measured, and the data processing terminal is used for processing the multi-line and single-line structured light; the multi-line structured light is perpendicular to the single-line structured light; the image shooting device is used for shooting an image of the space container to be detected irradiated by the multi-line and single-line structured light, and the data processing terminal is used for obtaining coordinate values of intersection points in a coordinate system of the image shooting device according to coordinate values of the intersection points in a pixel coordinate system on the image; obtaining a plane equation corresponding to each light bar based on the coordinate value of each intersection point under the coordinate system of the image shooting device; and obtaining the pose of the space container to be measured according to the plane equation corresponding to each light bar and the coordinate value of each point in the pixel coordinate system. According to the invention, the pose of the space container can be measured in real time, so that the space container can be accurately captured.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional posture measurement, and in particular to a space container posture measurement device, method, equipment, medium and product. Background Art

[0002] In the on-orbit space scenario, when capturing a space container, it is necessary to obtain the position and posture information of the space container in advance. The accurate perception of the position and posture of the space container often directly determines whether the space container can be accurately captured. In the current on-orbit space scenario, the three-dimensional measurement systems for cooperative targets are mainly monocular three-dimensional position and posture measurement systems and binocular / multi-eye three-dimensional measurement systems. In order to improve the measurement accuracy of the system, these two measurement systems usually need to arrange a certain number of targets on the measured target. The number and layout of targets seriously affect the measurement accuracy of the system. Due to the limited surface space of the space container, it is difficult to arrange dense targets, resulting in the inability to use monocular or multi-eye three-dimensional measurement systems to measure the position and posture of the space container in real time, resulting in the inability to accurately capture the space container. Summary of the invention

[0003] The purpose of this application is to provide a space container posture measurement device, method, equipment, medium and product, which can measure the posture of the space container in real time to achieve accurate capture of the space container.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a space container posture measurement device, comprising:

[0006] An image capturing device, a structured light generating device and a data processing terminal, wherein the structured light generating device is used to emit multi-line structured light and single-line structured light to the space container to be tested, wherein the multi-line structured light and the single-line structured light are perpendicular to each other; the image capturing device is used to capture the image of the space container to be tested irradiated by the multi-line structured light and the single-line structured light to obtain the image to be processed, and the data processing terminal is used to obtain the coordinate value of each intersection point in the image to be processed in the pixel coordinate system; each intersection point in the image to be processed is obtained according to the intersection point of each light strip in the multi-line structured light strip set in the image to be processed with the single-line structured light strip; the single-line structured light strip includes the light strip formed by the single-line structured light irradiating the space container to be tested; the multi-line structured light strip set includes the light strips formed by the multi-line structured light irradiating the space container to be tested; according to the coordinate value of each intersection point in the pixel coordinate system The coordinate value of each intersection point in the image capture device coordinate system is obtained based on the coordinate value of each intersection point in the image capture device coordinate system and the plane equation of the light plane of each line structured light in the multi-line structured light in the image capture device coordinate system, and the plane equation corresponding to each light strip in the multi-line structured light strip set is obtained; the coordinate value of each point on each light strip in the multi-line structured light strip set in the image capture device coordinate system is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameters of the image capture device and the plane equation corresponding to each light strip in the multi-line structured light strip set; the position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the image capture device coordinate system.

[0007] Optionally, the structured light generating device includes: a multi-line laser and a single-line laser; the multi-line laser is used to emit multi-line structured light to the space container to be tested, and the single-line laser is used to emit single-line structured light to the space container to be tested.

[0008] In a second aspect, the present application provides a method for measuring the position and posture of a space container, which is applied to the above-mentioned device for measuring the position and posture of a space container. The method for measuring the position and posture of a space container comprises:

[0009] Obtaining coordinate values ​​of each intersection point in the image to be processed in a pixel coordinate system; each intersection point in the image to be processed is obtained according to the intersection of each light strip in the multi-line structured light strip set and the single-line structured light strip in the image to be processed; the single-line structured light strip includes a light strip formed by irradiating a single-line structured light on the space container to be tested; the multi-line structured light strip set includes each light strip formed by irradiating a multi-line structured light on the space container to be tested;

[0010] Obtaining the coordinate values ​​of each intersection point in the image capture device coordinate system according to the coordinate values ​​of each intersection point in the pixel coordinate system, the internal parameters of the image capture device, and the plane equation of the light plane of the single-line structured light in the image capture device coordinate system;

[0011] Based on the coordinate values ​​of each intersection point in the image capture device coordinate system and the plane equation of each line structured light plane in the multi-line structured light in the image capture device coordinate system, the plane equation corresponding to each light strip in the multi-line structured light strip set is obtained;

[0012] According to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameter of the image capture device and the plane equation corresponding to each light strip in the multi-line structured light strip set, the coordinate value of each point on each light strip in the multi-line structured light strip set in the image capture device coordinate system is obtained;

[0013] The position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image shooting device.

[0014] Optionally, before obtaining the coordinate values ​​of each intersection point in the image to be processed in the pixel coordinate system, the following steps are further included:

[0015] The image capture device, the single-line structured light plane and the line structured light planes in the multi-line structured light are calibrated to obtain the internal parameters of the image capture device, the plane equation of the single-line structured light plane in the image capture device coordinate system and the plane equation of the line structured light planes in the multi-line structured light in the image capture device coordinate system.

[0016] Optionally, the process of determining each intersection point in the image to be processed specifically includes:

[0017] Binarize the image to be processed to obtain a binary image;

[0018] Performing a contour extraction operation on the binary image to obtain a contour with the largest area, and deleting contours other than the contour with the largest area from the binary image to obtain a first contour image;

[0019] Performing an expansion operation on the first contour image to obtain an expansion image;

[0020] Performing Hough line detection on the expansion image to obtain a single-line structured light strip, and obtaining coordinate values ​​of all points in the single-line structured light strip in a pixel coordinate system;

[0021] Modifying the grayscale value of the single-line structured light strip in the first contour image according to a preset grayscale value to obtain a second contour image;

[0022] Performing dilation and erosion operations on the second contour map in sequence to obtain a third contour map;

[0023] Performing Hough line detection on the third contour image to obtain each light strip in the multi-line structured light strip set, and obtaining the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system to obtain a point coordinate value set corresponding to each light strip in the multi-line structured light strip set;

[0024] For any one light strip in the multi-line structured light strip set, determine that all points corresponding to coordinate values ​​in the intersection of the point coordinate value set corresponding to the light strip and the single-line coordinate value set constitute an intersection point set corresponding to the light strip;

[0025] Determine the center point of the intersection point set corresponding to the light strip as the intersection point corresponding to the light strip;

[0026] The intersection points corresponding to all the light strips in the multi-line structured light strip set are determined as the intersection points in the image to be processed.

[0027] Optionally, obtaining the plane equation corresponding to each light strip in the multi-line structured light strip set based on the coordinate values ​​of each intersection point in the image capture device coordinate system and the plane equation of each line structured light plane in the multi-line structured light in the image capture device coordinate system specifically includes:

[0028] For any intersection point, the distance between the intersection point and the light planes of each line structured light in the multi-line structured light is calculated, and the plane equation of the line structured light plane with the smallest distance in the coordinate system of the image capture device is determined to be the plane equation corresponding to the light strip where the intersection point is located in the multi-line structured light strip set.

[0029] Optionally, the position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device, specifically including:

[0030] Fitting the surface equation of the space container to be measured according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device;

[0031] Fitting the axis equation of the space container to be tested according to the surface equation of the space container to be tested;

[0032] The position and posture of the space container to be measured is obtained according to the axis equation of the space container to be measured.

[0033] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described methods for measuring the position and posture of a space container.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for measuring the position and posture of a space container.

[0035] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for measuring the position and posture of a space container.

[0036] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0037] The present application provides a space container posture measurement device, method, equipment, medium and product. A structured light generating device is set to emit multi-line structured light and single-line structured light to the space container to be measured. An image shooting device shoots the image of the space container to be measured illuminated by the multi-line structured light and the single-line structured light. A data processing terminal processes the shot image to obtain the posture of the space container to be measured. There is no need to set a target on the space container, and the posture of the space container can be measured in real time to achieve accurate capture of the space container. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of the structure of a space container posture measurement device provided in one embodiment of the present application;

[0040] Figure 2 A schematic diagram of a structured light pattern provided in one embodiment of the present application;

[0041] Figure 3 A schematic diagram of the collected image information and image changes during the processing flow provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of a checkerboard used for image capture device calibration and structured light plane calibration provided in an embodiment of the present application;

[0043] Figure 5 A framework diagram of a method for measuring the position and posture of a space container provided in one embodiment of the present application;

[0044] Figure 6 A specific flow chart of a method for measuring the position and posture of a space container provided in one embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0048] The space container posture measurement device provided in the embodiment of the present application is specifically a space container posture measurement device based on the combination of monocular vision and multi-line structured light, such as Figure 1 As shown, the device comprises:

[0049] An image capturing device 1, a structured light generating device and a data processing terminal, wherein the structured light generating device is used to emit multi-line structured light and single-line structured light to the space container to be tested (referring to a certain container in space), wherein the multi-line structured light and the single-line structured light are perpendicular to each other to form a structured light pattern such as Figure 2 As shown; the image capturing device 1 is used to capture the image of the space container to be tested irradiated by multi-line structured light and single-line structured light to obtain the image to be processed, and the data processing terminal is used to obtain the coordinate value of each intersection point in the image to be processed in the pixel coordinate system; each intersection point in the image to be processed is obtained according to the intersection point of each light strip in the multi-line structured light strip set and the single-line structured light strip in the image to be processed; the single-line structured light strip includes the light strip formed by the single-line structured light irradiating the space container to be tested; the multi-line structured light strip set includes the light strips formed by the multi-line structured light irradiating the space container to be tested; according to the coordinate value of each intersection point in the pixel coordinate system, the internal parameter of the image capturing device 1 and the plane equation of the light plane of the single-line structured light in the coordinate system of the image capturing device 1 , obtain the coordinate value of each intersection point in the coordinate system of the image capture device 1; based on the coordinate value of each intersection point in the coordinate system of the image capture device 1 and the plane equation of the light plane of each line structured light in the multi-line structured light in the coordinate system of the image capture device 1, obtain the plane equation corresponding to each light strip in the multi-line structured light strip set; according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameter of the image capture device 1 and the plane equation corresponding to each light strip in the multi-line structured light strip set, obtain the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device 1; according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device 1, obtain the position and posture of the space container to be measured.

[0050] In practical applications, the structured light generating device includes: a multi-line laser 2 and a single-line laser 3; the multi-line laser 2 is used to emit multi-line structured light (11 vertical laser lines, which are parallel to each other) to the space container to be tested, and the single-line laser 3 is used to emit single-line structured light (1 horizontal laser line, which is perpendicular to the vertical laser line) to the space container to be tested.

[0051] In actual applications, the overall size of the structured light pattern is proportional to the projection distance. The size of the pattern projected on the wall 1.5m away from the structured light generator is 840*860mm. Since the field of view of the image capture device 1 is limited, at a distance of 1.5, the theoretical field of view of the image capture device 1 is only 768*768mm. Therefore, when the structured light pattern is used for measurement, only the 7 central laser lines are used as vertical laser lines.

[0052] In practical applications, the space container posture measurement device also includes: a fixed platform for fixing the image shooting device 1 and the structured light generating device.

[0053] In practical applications, the image capture device 1 uses a global exposure CMOS grayscale industrial camera. The camera model is ME2P-2621-15U3M / C. An industrial lens is added to the camera. The industrial lens model is Kowa-LM12JC10M.

[0054] In practical applications, the multi-line laser 2 uses a DOE industrial laser. The model of the multi-line laser 2 is EL65L200IG16-DOE.

[0055] In practical applications, the single-line laser 3 uses a diode mirror laser. The model of the single-line laser 3 is M-16A648A (10-150) -GLX.

[0056] In actual application, the fixed platform is made using 3D printing technology.

[0057] The space container posture measurement device provided in the present application does not need to set a target on the space container. It only needs to project a structured light pattern to the space container to be measured and calculate the posture information of the space container to be measured in combination with the structured light plane constraint. The measurement accuracy will not be affected by the target and the posture of the space container can be measured in real time to achieve accurate capture of the space container.

[0058] Compared with the structured light projection equipment in the commonly used structured light three-dimensional posture measurement system, the two lasers used in the space container posture measurement device provided in this application have smaller power, volume and weight, the cost of the entire system is lower, and it can better meet the needs of on-orbit space scenarios.

[0059] The present application also provides a space container posture measurement method applied to the above-mentioned space container posture measurement device, Figure 3 Taking the single-line structured light as an example, the specific measurement principle of the three-dimensional posture measurement method of a space container provided in the embodiment of the present application is as follows: Figure 3 As shown, point P is on the straight line O c P' is on the plane formed by the structured light generator and the structured light on the surface of the object. The relative positions of the image capture device 1 and the structured light generator remain unchanged. In the coordinate system of the image capture device 1, the equation of the light plane corresponding to the single-line structured light remains unchanged. Suppose the plane equation of the single-line structured light plane in the coordinate system of the image capture device 1 is: c x c +b c y c +c c z c +d c = 0, the image coordinate value of point P' in the image capture device 1 coordinate system is (x, y, 1), and the coordinate value of point P is (x c ,y c ,z c ), then the linear equation of the straight line OcP in the coordinate system of the image capturing device 1 is: Combining the above two formulas, we can get: Therefore, when the structured light plane equation and the image coordinate value of point P' are known, the coordinate value of any point on the structured light in the coordinate system of the image capture device 1 can be obtained by this formula. By simultaneously projecting multiple beams of parallel structured light (multi-line structured light) to the space container to be measured, multiple sets of coordinate values ​​of the surface points of the space container can be obtained. On this basis, these coordinate value information can be used to fit the overall posture state of the space container. Among them, the structured light plane equation in the coordinate system of the image capture device 1 needs to be obtained by calibrating the structured light plane, and the calculation of the image coordinate value of point P' requires knowing the intrinsic parameter matrix of the image capture device 1, which requires the calibration of the image capture device 1.

[0060] The method generally proceeds as follows Figure 5 The method includes: collecting images, preprocessing, extracting intersection coordinates, determining the light plane corresponding to the intersection, extracting point coordinates and fitting axes, and calculating posture. Figure 6 Specifically shown include:

[0061] Step 201: Obtain the coordinate value of each intersection point in the image to be processed in the pixel coordinate system; each intersection point in the image to be processed is obtained according to the intersection point of each light strip in the multi-line structured light strip set in the image to be processed and the single-line structured light strip; the single-line structured light strip includes the light strip formed by the single-line structured light irradiating the space container to be tested; the multi-line structured light strip set includes the light strips formed by the multi-line structured light irradiating the space container to be tested.

[0062] Step 202 : Obtain the coordinate values ​​of each intersection point in the image capture device 1 coordinate system according to the coordinate values ​​of each intersection point in the pixel coordinate system, the internal parameters of the image capture device 1 , and the plane equation of the single-line structured light plane in the image capture device 1 coordinate system.

[0063] Step 203: Based on the coordinate values ​​of each intersection point in the coordinate system of the image capture device 1 and the plane equation of the light plane of each line structured light in the multi-line structured light in the coordinate system of the image capture device 1, the plane equation corresponding to each light strip in the multi-line structured light strip set is obtained, and a corresponding relationship between each light strip in the multi-line structured light strip set in the image and the plane equation of the light plane of each line structured light in the multi-line structured light in the coordinate system of the image capture device 1 is established.

[0064] Step 204: Obtain the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device 1 according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameters of the image capture device 1 and the plane equation corresponding to each light strip in the multi-line structured light strip set.

[0065] Step 205: Obtain the position and posture of the space container to be measured according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device 1; the shape of the space container is roughly cylindrical.

[0066] In another exemplary embodiment of the present application, before obtaining the coordinate value of each intersection point in the image to be processed in the pixel coordinate system, the method further includes:

[0067] The image capturing device 1, the single-line structured light plane and the line structured light planes in the multi-line structured light are calibrated to obtain the internal parameters of the image capturing device 1, the plane equation of the single-line structured light plane in the coordinate system of the image capturing device 1 and the plane equation of the line structured light planes in the multi-line structured light in the coordinate system of the image capturing device 1.

[0068] In another exemplary embodiment of the present application, multi-line structured light is used for posture measurement of the target to be measured, and single-line structured light is used to distinguish the structured light plane equation corresponding to the multi-line structured light projected onto the target to be measured during measurement. Since the size of the target to be measured is limited, only a few lines of structured light in the multi-line structured light can be projected onto the target to be measured during measurement. Since there is no obvious characteristic difference between the multi-line structured light, it is impossible to clearly know which lines of structured light are projected on the surface of the target to be measured, and it is impossible to determine the structured light plane equation corresponding to these light strips. Therefore, the present application introduces a horizontal single-line structured light for the identification of vertical multi-line structured light. In the image, there is a clear difference between the horizontal single-line structured light and the vertical multi-line structured light, which can be easily distinguished, and it is also possible to determine the equation of the structured light plane corresponding to a certain light strip in the set of multi-line structured light strips.

[0069] Taking the image capturing device 1 as a camera, the single-line structured light as a horizontal laser line, and the multi-line structured light as a plurality of vertical laser lines as an example, the calibration process is introduced:

[0070] Step S101: Take the pictures required for calibration: Place the checkerboard calibration plate vertically on the ground about 1.2 meters in front of the camera to ensure that the entire checkerboard pattern can be fully observed in the camera field of view. Figure 4 As shown. Fix the camera, single-line laser 3 and multi-line laser 2 on the fixed platform with screws, keep the camera fixed, and adjust the positions of the two lasers to ensure that: on the one hand, the light strip corresponding to the horizontal laser line is located in the center of the camera field of view and parallel to the upper and lower boundaries of the camera field of view; on the other hand, the 7-9 light strips in the center of the vertical multiple laser lines can be projected onto the checkerboard calibration board and are perpendicular to the light strips corresponding to the horizontal laser lines. Change the position of the checkerboard calibration board, and take pictures with only the light strips corresponding to the vertical line lasers, only the light strips corresponding to the horizontal line lasers, and no light strips. These three pictures are regarded as a group of pictures. To ensure the accuracy of camera calibration and light plane calibration, at least ten groups of pictures are required. When changing the position of the checkerboard calibration board, it is necessary to ensure that the complete checkerboard pattern is always in the camera field of view and the light strips corresponding to the 7 vertical laser lines in the center can be projected onto the calibration board.

[0071] Step S102: extracting the coordinate values ​​of the corner points in the image in the pixel coordinate system through Harris corner points.

[0072] Step S103: Use Zhang Zhengyou calibration method to complete camera calibration.

[0073] Step S104: Fit the plane equation of the checkerboard plane in each group of pictures in the camera coordinate system. When using Zhang Zhengyou calibration method to solve the camera's intrinsic parameter matrix, the extrinsic parameter matrix corresponding to different checkerboard pictures will also be solved. For a group of pictures, assume that its extrinsic parameter matrix is: Where R is the third-order rotation matrix and T is the translation vector.

[0074] In Zhang Zhengyou's calibration method, the origin of the world coordinate system is set at the corner point of the upper left corner of the chessboard, and the Z axis is perpendicular to the entire chessboard plane, and the X axis and Y axis coincide with the two sides of the chessboard. Therefore, the coordinate value of any corner point on the chessboard in the world coordinate system is known. Let the coordinate value of a corner point of the chessboard in the world coordinate system be (U, V, W, 1) (W = 0), and let the coordinate value of the corner point in the camera coordinate system be (X c ,Y c ,Z c ,1), according to the definition of the external parameter matrix: Therefore, under the condition of known extrinsic parameter matrix, the coordinate value of each checkerboard corner point in the camera coordinate system can be calculated. For each set of pictures, the plane equation of the checkerboard plane in the camera coordinate system can be fitted through the coordinate values ​​of these corner points.

[0075] Step S105: extract the coordinate value of a point on each light strip in each picture in the pixel coordinate system, and calculate the corresponding camera coordinate value by combining the chessboard plane equation where the point is located and the camera intrinsic parameter matrix.

[0076] Assume that the coordinate value of a point on the light strip in the pixel coordinate system is (u, v, 1), and the corresponding camera coordinate value is (X c ,Y c ,Z c ,1), the camera internal parameter matrix is:

[0077] According to the definition of the camera intrinsic parameter matrix, we have:

[0078] Also assume that the plane equation of the chessboard plane where the point is located in the camera coordinate system is: Ax+By+Cz+D=0.

[0079] Combining the above equations, we can get the system of equations: Therefore, under the conditions that the coordinate value of the point on the light strip in the pixel coordinate system is known, the camera intrinsic parameter matrix is ​​known, and the chessboard plane equation is known, the camera coordinate value of the point can be solved.

[0080] Among them, the operation of extracting the coordinate values ​​of the points on each light strip in each picture in the pixel coordinate system requires the use of a specific laser center point extraction method. In the present application, the following method is adopted to implement this operation. The specific steps are: using the Steger algorithm to extract the laser center point of the collected image containing light strips. If the image contains only one light strip corresponding to a horizontal laser line, the extracted point is the point of the light strip. If the image contains multiple light strips corresponding to vertical laser lines, subsequent steps are required to further classify the extracted points according to the light strips corresponding to the vertical laser lines to which they belong.

[0081] The classification steps include:

[0082] The image processed by the laser center point extraction method is subjected to a morphological dilation operation, the purpose of which is to connect the light strips that are broken due to the absorption of energy by the black area of ​​the calibration plate.

[0083] By extracting the contours of the expanded light strips, the same number of contours as the number of laser lines can be obtained, and then the contours are numbered so that each contour has an independent identity feature.

[0084] Calculate the distance from the extracted point to each contour edge to determine whether the point is inside the current contour. If it is determined to be inside, the point is also numbered, and the number is the same as the corresponding contour number. Repeat this process until all points are numbered, and the classification of the extracted points is completed.

[0085] Step S106: according to the coordinate values ​​of the points on each light strip in the camera coordinate system, the light plane equation in the camera coordinate system is fitted using the least square method.

[0086] In another exemplary embodiment of the present application, before obtaining the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device 1 according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameter of the image capture device 1, and the plane equation corresponding to each light strip in the multi-line structured light strip set, the method further includes:

[0087] The image to be processed is processed, the coordinate value of each point on each light strip in the multi-line structured light strip set in the image in the pixel coordinate system is extracted, and the points are classified according to the coordinate value of each point in the pixel coordinate system.

[0088] In another exemplary embodiment of the present application, obtaining the coordinate value of each intersection point in the image to be processed in the pixel coordinate system specifically includes:

[0089] The image to be processed is binarized to obtain a binary image.

[0090] The binary image is subjected to contour extraction to obtain the contour with the largest area, and the contours other than the contour with the largest area are deleted from the binary image to obtain a first contour image. Specifically, the target points in the binary image are deleted to obtain the first contour image; the target points are all points other than the points on the contour with the largest area.

[0091] A dilation operation is performed on the first contour image to obtain a dilated image.

[0092] Perform Hough line detection on the expansion image to obtain a single-line structured light strip, and obtain the coordinate values ​​of all points in the single-line structured light strip in the pixel coordinate system. Specifically, perform Hough line detection on the expansion image, and screen the detected lines according to the slope, and screen out the horizontal lines to obtain the single-line structured light strip.

[0093] The second contour image is obtained by modifying the grayscale value of the single-line structured light strip in the first contour image according to a preset grayscale value. Specifically, the obtained horizontal straight line is drawn into the original contour image with a grayscale value of 0, thereby obtaining a contour image without the single-line structured light strip.

[0094] The second contour image is sequentially expanded and eroded to obtain a third contour image, and each light strip in the multi-line structured light strip set that is disconnected due to the removal of the single-line structured light strip is repaired.

[0095] Hough line detection (multi-laser line extraction operation) is performed on the third contour image to obtain each light strip in the multi-line structured light strip set, and the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system is obtained to obtain a set of point coordinate values ​​corresponding to each light strip in the multi-line structured light strip set.

[0096] For any light strip in the multi-line structured light strip set, determine the intersection point set corresponding to the light strip and all points corresponding to the coordinate values ​​in the intersection of the point coordinate value set corresponding to the light strip and the single-line coordinate value set. Because the light strip has a width, the intersection point set corresponding to the light strip includes multiple intersection points.

[0097] The center point of the intersection point set corresponding to the light bar is determined as the intersection point corresponding to the light bar.

[0098] The intersection points corresponding to all the light strips in the multi-line structured light strip set are determined as the intersection points in the image to be processed.

[0099] In another exemplary embodiment of the present application, the plane equation corresponding to each light strip in the multi-line structured light strip set is obtained based on the coordinate values ​​of each intersection point in the coordinate system of the image capture device 1 and the plane equation of each line structured light plane in the multi-line structured light in the coordinate system of the image capture device 1, specifically including:

[0100] For any intersection point, the distance between the intersection point and the light planes of each line structured light in the multi-line structured light is calculated, and the plane equation of the line structured light plane with the smallest distance in the coordinate system of the image capture device 1 is determined to be the plane equation corresponding to the light strip where the intersection point is located in the multi-line structured light strip set.

[0101] In another exemplary embodiment of the present application, the position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device 1, specifically including:

[0102] The surface equation of the space container to be measured is fitted according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image shooting device 1.

[0103] The axis equation of the space container to be measured is fitted according to the surface equation of the space container to be measured.

[0104] The position and posture of the space container to be measured is obtained according to the axis equation of the space container to be measured.

[0105] The space container posture measurement method provided by this application has the following effects:

[0106] 1. The space container posture measurement method provided in this application is less affected by the measurement scene than the posture measurement method in the space on-orbit scene of the related technology. On the other hand, only one picture is needed to obtain the posture information of the measured space container through simple mathematical operations. Compared with the related technology structured light three-dimensional posture measurement method, it has faster measurement speed and lower measurement cost, and can realize real-time measurement of the posture information of the space container.

[0107] 2. At present, the structured light visual posture measurement method is mainly divided into two types: coded structured light and non-coded structured light. Among them, the coded structured light uses a structured light generator to generate coded light patterns, captures the shape of the light pattern projection in the image capture device, and calculates the posture of the object by calculating the shape and position of the light pattern. Coded structured light can be divided into time-coded structured light and space-coded structured light according to the different ways and patterns of projecting structured light. Time-coded structured light refers to projecting a series of simple patterns onto the surface of the object to be measured in chronological order. The encoding of a given pixel is the change of the position in time with the brightness of the projected pattern. This coding scheme has high measurement accuracy in three-dimensional measurement and is easy to implement in engineering. However, this scheme requires continuous shooting of multiple images for each measurement, and the measurement speed is slow, and it is impossible to reconstruct dynamic objects in real time. The space container posture measurement method provided in this application only needs to capture one image to realize the posture measurement of the space container. The measurement speed is fast and the measured target can be measured in real time.

[0108] 3. Spatially coded structured light refers to structured light that is mathematically coded and projected into the measured space, and the light spot does not have repeatability within a certain range. Therefore, the coded value of a certain pixel point can be obtained through the information provided by the points around the point. This coding method only projects and collects one image during the measurement process, and the amount of calculation is significantly reduced compared to the time domain coding method, and can be used for real-time three-dimensional scene measurement. However, the resolution of spatial domain coding is not high, and information about adjacent points in space will be lost during the decoding stage, and the final calculated three-dimensional point of the scene is less accurate than that of time domain coding. Moreover, coded structured light requires a projector to project the coded pattern, and the overall power and weight of the measurement system will be relatively large, making it difficult to apply to space scenes that have strict requirements on power and weight. The space container posture measurement method provided in this application uses a small laser as a structured light generating device, which has the advantages of low power and light weight compared to a projector, and can better meet the needs of space scenes.

[0109] 4. The most common non-coded structured light pose measurement method is to use single-line structured light to scan the object to obtain the target three-dimensional information. This method has high measurement accuracy, but it takes a certain amount of time to scan the object to be measured, and the real-time performance is poor. This application combines single-line structured light and multi-line structured light. There is no need to scan the object to be measured, only image processing is required, which can solve the problem of poor real-time performance.

[0110] 5. In a related technology, the laser speckle projector is equivalent to a virtual image shooting device, and an innovative pseudo-binocular model is established, so that binocular stereo matching can be applied in a monocular system to generate a disparity map and perform three-dimensional shape measurement of the object. However, the stereo matching in this method takes a long time and the accuracy of this method is low. The space container posture measurement method provided in this application mainly uses laser triangulation for measurement, which has a lower calculation amount, a shorter measurement time, and a higher measurement accuracy than stereo matching.

[0111] 6. Another related technology uses a simple, highly robust, non-coded grid structured light template, and proposes a three-dimensional morphology measurement method based on non-coded grid structured light. In addition, the binocular and monocular measurement models are studied separately. A new grid structured light feature point extraction algorithm is proposed for the binocular measurement model, and a binocular stereo matching method is used for three-dimensional measurement; a grid light plane calibration method based on coplanar constraints is proposed for the monocular measurement model. However, the grid structured light used in this method is composed of lasers of different colors, and the measurement needs to rely on the feature information provided by the color of the laser line, which cannot be applied to the three-dimensional measurement of grayscale image shooting devices. The space container posture measurement method provided in this application does not need to rely on the color of the laser to provide feature information, and grayscale images can be used for three-dimensional measurement.

[0112] 7. In other related technologies, a grid structured light visual posture measurement system is designed, which obtains the posture information of the measured target by obtaining the point cloud data of the surface of the measured target and aligning it with the original known point cloud data. This method requires obtaining accurate dense point cloud information of the measured target in advance, and point cloud alignment takes a lot of time. The space container posture measurement method provided in this application does not require the alignment of the acquired point cloud information, and the measurement takes less time.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of a method for measuring the posture of a space container. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for measuring the posture of a space container is implemented.

[0114] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0115] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0116] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0119] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A space container posture measurement device, characterized in that: The space container posture measurement device comprises: An image capturing device, a structured light generating device and a data processing terminal, wherein the structured light generating device is used to emit multi-line structured light and single-line structured light to the space container to be tested, wherein the multi-line structured light and the single-line structured light are perpendicular to each other; the image capturing device is used to capture the image of the space container to be tested irradiated by the multi-line structured light and the single-line structured light to obtain the image to be processed, and the data processing terminal is used to obtain the coordinate value of each intersection point in the image to be processed in the pixel coordinate system; each intersection point in the image to be processed is obtained according to the intersection point of each light strip in the multi-line structured light strip set in the image to be processed with the single-line structured light strip; the single-line structured light strip includes the light strip formed by the single-line structured light irradiating the space container to be tested; the multi-line structured light strip set includes the light strips formed by the multi-line structured light irradiating the space container to be tested; according to the coordinate value of each intersection point in the pixel coordinate system The coordinate value of each intersection point in the image capture device coordinate system is obtained based on the coordinate value of each intersection point in the image capture device coordinate system and the plane equation of the light plane of each line structured light in the multi-line structured light in the image capture device coordinate system, and the plane equation corresponding to each light strip in the multi-line structured light strip set is obtained; the coordinate value of each point on each light strip in the multi-line structured light strip set in the image capture device coordinate system is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameters of the image capture device and the plane equation corresponding to each light strip in the multi-line structured light strip set; the position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the image capture device coordinate system.

2. The space container position and posture measurement device according to claim 1, characterized in that: The structured light generating device comprises: a multi-line laser and a single-line laser; the multi-line laser is used to emit multi-line structured light to the space container to be tested, and the single-line laser is used to emit single-line structured light to the space container to be tested.

3. A method for measuring the position and posture of a space container, characterized in that: The space container posture measurement device applied to any one of claims 1-2, wherein the space container posture measurement method comprises: Obtaining coordinate values ​​of each intersection point in the image to be processed in a pixel coordinate system; each intersection point in the image to be processed is obtained according to the intersection of each light strip in the multi-line structured light strip set and the single-line structured light strip in the image to be processed; the single-line structured light strip includes a light strip formed by irradiating a single-line structured light on the space container to be tested; the multi-line structured light strip set includes each light strip formed by irradiating a multi-line structured light on the space container to be tested; Obtaining the coordinate values ​​of each intersection point in the image capture device coordinate system according to the coordinate values ​​of each intersection point in the pixel coordinate system, the internal parameters of the image capture device, and the plane equation of the light plane of the single-line structured light in the image capture device coordinate system; Based on the coordinate values ​​of each intersection point in the image capture device coordinate system and the plane equation of each line structured light plane in the multi-line structured light in the image capture device coordinate system, the plane equation corresponding to each light strip in the multi-line structured light strip set is obtained; According to the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system, the internal parameter of the image capture device and the plane equation corresponding to each light strip in the multi-line structured light strip set, the coordinate value of each point on each light strip in the multi-line structured light strip set in the image capture device coordinate system is obtained; The position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image shooting device.

4. The method for measuring the position and posture of a space container according to claim 3, characterized in that: Before obtaining the coordinate values ​​of each intersection point in the image to be processed in the pixel coordinate system, the following steps are also included: The image capture device, the single-line structured light plane and the line structured light planes in the multi-line structured light are calibrated to obtain the internal parameters of the image capture device, the plane equation of the single-line structured light plane in the image capture device coordinate system and the plane equation of the line structured light planes in the multi-line structured light in the image capture device coordinate system.

5. The method for measuring the position and posture of a space container according to claim 3, characterized in that: The process of determining each intersection point in the image to be processed specifically includes: Binarize the image to be processed to obtain a binary image; Performing a contour extraction operation on the binary image to obtain a contour with the largest area, and deleting contours other than the contour with the largest area from the binary image to obtain a first contour image; Performing an expansion operation on the first contour image to obtain an expansion image; Performing Hough line detection on the expansion image to obtain a single-line structured light strip, and obtaining coordinate values ​​of all points in the single-line structured light strip in a pixel coordinate system to obtain a single-line coordinate value set; Modifying the grayscale value of the single-line structured light strip in the first contour image according to a preset grayscale value to obtain a second contour image; Performing dilation and erosion operations on the second contour map in sequence to obtain a third contour map; Performing Hough line detection on the third contour image to obtain each light strip in the multi-line structured light strip set, and obtaining the coordinate value of each point on each light strip in the multi-line structured light strip set in the pixel coordinate system to obtain a point coordinate value set corresponding to each light strip in the multi-line structured light strip set; For any one light strip in the multi-line structured light strip set, determine that all points corresponding to coordinate values ​​in the intersection of the point coordinate value set corresponding to the light strip and the single-line coordinate value set constitute an intersection point set corresponding to the light strip; Determine the center point of the intersection point set corresponding to the light strip as the intersection point corresponding to the light strip; The intersection points corresponding to all the light strips in the multi-line structured light strip set are determined as the intersection points in the image to be processed.

6. The method for measuring the position and posture of a space container according to claim 3, characterized in that: The obtaining of the plane equation corresponding to each light strip in the multi-line structured light strip set based on the coordinate values ​​of each intersection point in the image capture device coordinate system and the plane equation of each light plane of the multi-line structured light in the image capture device coordinate system specifically includes: For any intersection point, the distance between the intersection point and the light planes of each line structured light in the multi-line structured light is calculated, and the plane equation of the line structured light plane with the smallest distance in the coordinate system of the image capture device is determined to be the plane equation corresponding to the light strip where the intersection point is located in the multi-line structured light strip set.

7. The method for measuring the position and posture of a space container according to claim 3, characterized in that: The position and posture of the space container to be measured is obtained according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image shooting device, specifically including: Fitting the surface equation of the space container to be measured according to the coordinate value of each point on each light strip in the multi-line structured light strip set in the coordinate system of the image capture device; Fitting the axis equation of the space container to be tested according to the surface equation of the space container to be tested; The position and posture of the space container to be measured is obtained according to the axis equation of the space container to be measured.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the space container posture measurement method described in any one of claims 3 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for measuring the position and posture of a space container described in any one of claims 3 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for measuring the position and posture of a space container described in any one of claims 3 to 7 is implemented.

Images