Cable bending radius detection method and device

Abstract

The present application relates to a cable bending radius detection method and device. The method comprises: obtaining first image information collected in a real cable space, the first image information comprising a positioning mark image and image information of a real cable, the positioning mark image being an image preset at a preset position in the real cable space; determining a camera projection matrix according to the coordinate information of the positioning mark image; performing reverse projection processing on the real cable according to the camera projection matrix to obtain a curve corresponding to the real cable in a cable bending plane; determining the bending radius of the real cable according to the curve in the cable bending plane. The use of this method can improve the accuracy of cable bending radius detection.

Description

Technical Field

[0001] The present application relates to the technical field of cable visual detection, and in particular to a cable bending radius detection method and device. Background Art

[0002] Cables are important components for transmitting current and signals and ensuring the connection and operation of equipment. Taking spacecraft as an example, the laying of cables in various spacecrafts has process requirements for curvature, which is generally required to be no less than the corresponding minimum bending radius. Therefore, it is necessary to test the bending radius of the cable to check whether the laying of the cable meets the process requirements.

[0003] In the related technology, a color camera is used in conjunction with a depth camera to obtain the depth information of the cable through the depth camera, and a three-dimensional model of the cable is reconstructed by aligning the depth image with the plane image. Finally, the cable bending radius is detected on the three-dimensional model of the cable.

[0004] However, the registration of depth images and plane images is relatively complex, and the sampling points of depth images are generally sparse. Especially for slender objects like cables, the depth image can provide fewer effective information feature points. Therefore, it is difficult to reconstruct the three-dimensional model of the cable and the reconstructed model has low accuracy, which leads to low detection accuracy of the cable bending radius. Summary of the invention

[0005] Based on this, it is necessary to provide a cable bending radius detection method and device that can improve the detection accuracy of the cable bending radius in order to solve the above technical problems.

[0006] A cable bending radius detection method, comprising:

[0007] Acquire first image information collected in a real cable space, the first image information including a positioning mark image and image information of a real cable, the positioning mark image being an image preset at a preset position in the real cable space;

[0008] Determining the projection matrix of the camera according to the coordinate information of the positioning mark image;

[0009] Performing a back-projection process on the real cable according to the projection matrix of the camera to obtain a curve corresponding to the real cable in the cable bending plane;

[0010] The bending radius of the real cable is determined according to the curve in the cable bending plane.

[0011] In one embodiment, determining the projection matrix of the camera according to the coordinate information of the positioning mark image includes:

[0012] Determine the coordinate information of the feature point in the world coordinate system according to the coordinate information of the feature point in the positioning mark image in the marking plane coordinate system and the preset position corresponding to the positioning mark image;

[0013] A projection matrix of a camera is determined according to the coordinate information of the feature point in the first image information and the coordinate information of the feature point in the world coordinate system.

[0014] In one embodiment, performing a back-projection process on the real cable according to the projection matrix of the camera to obtain a curve corresponding to the real cable in the cable bending plane includes:

[0015] Performing cable backbone extraction on the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information;

[0016] Establishing a cable bending plane according to coordinate information of a virtual cable corresponding to the real cable in the virtual cable space in a world coordinate system, wherein the virtual cable space is a three-dimensional model of the real cable space, and a unified world coordinate system is established between the real cable space and the virtual cable space;

[0017] According to the projection matrix of the camera, the cable backbone curve is subjected to a back-projection process to obtain a curve corresponding to the cable backbone curve in the cable bending plane.

[0018] In one embodiment, the bending area of ​​the virtual cable in the virtual cable space is contained in a mask.

[0019] The extracting the cable backbone of the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information includes:

[0020] Determine the position and posture information of the camera in the world coordinate system;

[0021] According to the position information, controlling the virtual camera to collect second image information in the virtual cable space, wherein the second image information includes the virtual cable;

[0022] generating a mask image according to the mask corresponding to the bending area of ​​the virtual cable in the second image information;

[0023] determining a real cable area image from the first image information according to the mask image;

[0024] A cable backbone is extracted from the real cable region image to obtain a cable backbone curve corresponding to the real cable.

[0025] In one embodiment, performing a back-projection process on the cable backbone curve according to the projection matrix of the camera to obtain a curve corresponding to the cable backbone curve in the cable bending plane includes:

[0026] For any pixel point in the cable backbone curve, determine the coordinate information of the pixel point in the world coordinate system according to the coordinate information corresponding to the pixel point in the first image information and the projection matrix;

[0027] Converting the coordinate information of each pixel point in the world coordinate system into two-dimensional coordinate information corresponding to each pixel point in the cable bending plane;

[0028] According to the two-dimensional coordinate information corresponding to each of the pixel points, a curve corresponding to the cable backbone curve in the cable bending plane is obtained.

[0029] In one embodiment, determining the bending radius of the real cable according to the curve in the cable bending plane includes:

[0030] The curve is processed by using the least square method to obtain the bending radius of the real cable.

[0031] A cable bending radius detection device, the device comprising:

[0032] An acquisition module, used to acquire first image information collected in a real cable space, wherein the first image information includes a positioning mark image and image information of a real cable, wherein the positioning mark image is an image preset at a preset position in the real cable space;

[0033] A first determination module, used to determine the projection matrix of the camera according to the coordinate information of the positioning mark image;

[0034] A processing module, configured to perform a reverse projection process on the real cable according to the projection matrix of the camera, so as to obtain a curve corresponding to the real cable in a cable bending plane;

[0035] The second determination module is used to determine the bending radius of the real cable according to the curve in the cable bending plane.

[0036] In one embodiment, the first determining module is further used to:

[0037] Determine the coordinate information of the feature point in the world coordinate system according to the coordinate information of the feature point in the positioning mark image in the marking plane coordinate system and the preset position corresponding to the positioning mark image;

[0038] A projection matrix of a camera is determined according to the coordinate information of the feature point in the first image information and the coordinate information of the feature point in the world coordinate system.

[0039] In one embodiment, the processing module is further used for:

[0040] Performing cable backbone extraction on the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information;

[0041] Establishing a cable bending plane according to coordinate information of a virtual cable corresponding to the real cable in the virtual cable space in a world coordinate system, wherein the virtual cable space is a three-dimensional model of the real cable space, and a unified world coordinate system is established between the real cable space and the virtual cable space;

[0042] According to the projection matrix of the camera, the cable backbone curve is subjected to a back-projection process to obtain a curve corresponding to the cable backbone curve in the cable bending plane.

[0043] In one embodiment, the bending area of ​​the virtual cable in the virtual cable space is included in a mask, and the processing module is further used to:

[0044] Determine the position and posture information of the camera in the world coordinate system;

[0045] According to the position information, controlling the virtual camera to collect second image information in the virtual cable space, wherein the second image information includes the virtual cable;

[0046] generating a mask image according to the mask corresponding to the bending area of ​​the virtual cable in the second image information;

[0047] determining a real cable area image from the first image information according to the mask image;

[0048] A cable backbone is extracted from the real cable region image to obtain a cable backbone curve corresponding to the real cable.

[0049] In one embodiment, the processing module is further used for:

[0050] For any pixel point in the cable backbone curve, determine the coordinate information of the pixel point in the world coordinate system according to the coordinate information corresponding to the pixel point in the first image information and the projection matrix;

[0051] Converting the coordinate information of each pixel point in the world coordinate system into two-dimensional coordinate information corresponding to each pixel point in the cable bending plane;

[0052] According to the two-dimensional coordinate information corresponding to each of the pixel points, a curve corresponding to the cable backbone curve in the cable bending plane is obtained.

[0053] In one embodiment, the second determining module is further used to:

[0054] The curve is processed by using the least square method to obtain the bending radius of the real cable.

[0055] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0056] Acquire first image information collected in a real cable space, the first image information including a positioning mark image and image information of a real cable, the positioning mark image being an image preset at a preset position in the real cable space;

[0057] Determining the projection matrix of the camera according to the coordinate information of the positioning mark image;

[0058] Performing a back-projection process on the real cable according to the projection matrix of the camera to obtain a curve corresponding to the real cable in the cable bending plane;

[0059] The bending radius of the real cable is determined according to the curve in the cable bending plane.

[0060] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0061] Acquire first image information collected in a real cable space, the first image information including a positioning mark image and image information of a real cable, the positioning mark image being an image preset at a preset position in the real cable space;

[0062] Determining the projection matrix of the camera according to the coordinate information of the positioning mark image;

[0063] Performing a back-projection process on the real cable according to the projection matrix of the camera to obtain a curve corresponding to the real cable in the cable bending plane;

[0064] The bending radius of the real cable is determined according to the curve in the cable bending plane.

[0065] The above-mentioned cable bending radius detection method and device obtain the first image information collected in the real cable space, and the first image information includes the positioning mark image and the real cable. The positioning mark image is an image preset at a preset position in the real cable space, and the projection matrix of the camera can be determined according to the coordinate information of the positioning mark image. Furthermore, the real cable can be reversely projected according to the projection matrix of the camera to obtain the corresponding curve of the real cable in the cable bending plane, and the bending radius of the real cable can be determined according to the curve in the cable bending plane. The cable bending radius detection method and device provided by the embodiment of the present disclosure can realize the detection of the cable bending radius through the projection matrix of the camera and the collected plane image, without the need for complex equipment such as a depth camera, thereby reducing the calculation complexity and improving the detection efficiency and detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of a flow chart of a cable bending radius detection method in one embodiment;

[0067] Figure 2 is an example schematic diagram of a real cable space in one embodiment;

[0068] Figure 3 A schematic diagram of a flow chart of a cable bending radius detection method in one embodiment;

[0069] Figure 4 An example schematic diagram of a positioning mark image in one embodiment;

[0070] Figure 5 is an exemplary schematic diagram of a coordinate system in one embodiment;

[0071] Figure 6 is a schematic diagram of an example of a cable bending plane in one embodiment;

[0072] Figure 7 A schematic diagram of a flow chart of a cable bending radius detection method in one embodiment;

[0073] Figure 8 A schematic diagram of a flow chart of a cable bending radius detection method in one embodiment;

[0074] Fig. 9 An exemplary schematic diagram of a cable bending radius detection method in an embodiment;

[0075] Figure 10a to Figure 10b An exemplary schematic diagram of a cable bending radius detection method in an embodiment;

[0076] Fig.11 An exemplary schematic diagram of a cable bending radius detection method in an embodiment;

[0077] Fig.12 A schematic diagram of a flow chart of a cable bending radius detection method in one embodiment;

[0078] Figure 13a to Figure 13d An exemplary schematic diagram of a cable bending radius detection method in an embodiment;

[0079] Fig.14 An exemplary schematic diagram of a cable bending radius detection method in an embodiment;

[0080] Fig.15 is a structural block diagram of a cable bending radius detection device in one embodiment;

[0081] Fig.16 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0083] In one embodiment, Figure 1 As shown, a cable bending radius detection method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0084] Step 102: Acquire first image information collected in the real cable space, the first image information including a positioning mark image and image information of the real cable, the positioning mark image being an image preset at a preset position in the real cable space.

[0085] For example, a plurality of real cables are laid in the real cable space, and positioning mark images can be posted at preset positions in the real cable space. The positioning mark images can include: ARToolKit marks, QR codes, pre-processed plane images, etc., wherein the number of positioning mark images can be one or more, that is, multiple positioning mark images can be posted at multiple preset positions. Figure 2 As shown, it includes: a camera 202 , a fixed mark 204 , a cable 206 , a buckle 208 , and a cabin panel 210 .

[0086] The first image information in the real cable space can be collected by a camera (or any image acquisition device, a camera is taken as an example in the embodiment of the present disclosure), and the first image information may include image information of the real cable and a positioning mark image, wherein the image information of the real cable may include a cable with a bending radius to be detected.

[0087] Step 104, determining the projection matrix of the camera according to the coordinate information of the positioning mark image.

[0088] For example, a preset position in the real cable space (such as a cabin corner) can be used as the coordinate origin, and three mutually perpendicular characteristic directions can be selected as the world coordinate system x, y, and z directions to establish a world coordinate system. The coordinate information of the positioning mark image in the world coordinate system can be extracted, and then the projection matrix of the camera can be determined based on the coordinate information.

[0089] In one embodiment, referring to Figure 3 In step 104, the projection matrix of the camera is determined according to the coordinate information of the positioning mark image, which may specifically include:

[0090] Step 302, determining the coordinate information of the feature point in the world coordinate system according to the coordinate information of the feature point in the positioning mark image in the marking plane coordinate system and the preset position corresponding to the positioning mark image;

[0091] Step 304: Determine the projection matrix of the camera according to the coordinate information of the feature points in the first image information and the coordinate information of the feature points in the world coordinate system.

[0092] For example, the feature points of the positioning mark image can be extracted, and the coordinate information 1 of the feature points of the positioning mark image in the mark plane coordinate system corresponding to the positioning mark image can be determined. After the coordinate information 2 of the preset position corresponding to the positioning mark image in the world coordinate system is determined, the coordinate information 1 is rotated and translated through the coordinate information 2 to obtain the coordinate information of each feature point in the world coordinate system. The coordinate information of the feature point in the first image information (that is, the coordinate information in the camera coordinate system) is determined, and the projection matrix of the camera can be determined based on the coordinate information of the feature point in the first image information and the coordinate information in the world coordinate system.

[0093] by Figure 4 Take the ARToolkit plane positioning marker image as an example, where the world coordinate system (origin is O w , the x-axis, y-axis, and z-axis are marked as X w , Y w and Z w ), mark the plane coordinate system (origin is O T , the x-axis, y-axis, and z-axis are marked as X T , YT and Z T ), camera coordinate system (origin is O C , the x-axis, y-axis, and z-axis are marked as X C , Y C and Z C ) can refer to Figure 5 As shown. First, the plane positioning mark image is preprocessed to extract several feature points x i , for example, the corner points of the black and white areas are extracted as feature points. For each corner point x i , according to the camera projection principle: In the formula, is the corner point x i The homogeneous coordinates of the corresponding three-dimensional point in the world coordinate system, is the corner point x i The homogeneous coordinates corresponding to the two-dimensional coordinates in the first image information, K is the camera internal parameter matrix, [R|t] is the camera external parameter matrix, and K[R|t] is called the camera projection matrix. Expanding the above formula, we can get:

[0094]

[0095] Since each feature point can generate two constraints, the camera projection matrix K[R|t] can be solved by obtaining more than 6 feature points.

[0096] Among them, the camera intrinsic parameter matrix K can be obtained by calibrating the camera in advance. After obtaining the camera's projection matrix K[R|t], the camera's projection matrix K[R|t] can be decomposed by the camera intrinsic parameter matrix K to obtain the rotation matrix R and translation t, thereby obtaining the extrinsic parameter matrix between the camera coordinate system and the world coordinate system, that is, the pose information [R|t].

[0097] In a possible implementation, a virtual cable space consistent with the real cable space can be pre-established, and the same preset position and the same x, y, and z directions can be selected in the virtual cable space to construct a world coordinate system, so as to achieve alignment between the virtual and real spaces and form a unified world coordinate system for the virtual and real spaces. At the same time, a feature point and a feature plane can be selected in the virtual cable space (the feature plane is also an actual plane in the real cable space), and two mutually perpendicular directions can be selected in the feature plane to determine the posting posture of the positioning mark image. The positioning mark image is set at the feature point corresponding to the feature plane in the virtual cable space in the posting posture. The positioning mark image is a virtual image that is completely consistent with the positioning mark image posted in the real cable space.

[0098] On this basis, the same positioning mark image is posted in the same posting posture on the same feature plane and the same feature point in the real cable space.

[0099] The image information of the cable including the positioning mark image and the bending radius to be detected can be collected in the virtual cable space at the same viewing angle, and the feature points of the positioning mark image can be extracted through collection, and the coordinate information of the feature points in the world coordinate system of the virtual cable space can be determined. Since the virtual cable space is a reconstructed 3D model, the coordinate information of the feature points in the world coordinate system can be directly determined from the virtual cable space.

[0100] In the disclosed embodiment, there is no specific limitation on the method for determining the coordinate information of the feature point in the world coordinate system. The coordinate information can be determined from the real cable space or from the virtual cable space.

[0101] It should be noted that the virtual cable space is consistent with the real cable space, including cables, clips, cabins, equipment, etc. And consistent with the real cable space, the clip model and equipment model in the virtual cable space should be installed in a fixed position, and the cable model should be set strictly according to the specified route to meet the corresponding minimum bending radius requirements.

[0102] Step 106 , performing a reverse projection process on the real cable according to the projection matrix of the camera, and obtaining a curve corresponding to the real cable in the cable bending plane.

[0103] After obtaining the camera projection matrix, the real cable can be back-projected according to the camera projection matrix to back-project the real cable onto the cable bending plane to obtain the corresponding curve of the real cable in the cable bending plane. The cable bending plane can be the plane where the real cable is located in the actual center. For example, the cable bending plane can refer to Figure 6 shown.

[0104] Step 108: Determine the bending radius of the actual cable according to the curve in the cable bending plane.

[0105] In the embodiment of the present disclosure, after the curve of the real cable in the cable bending plane is obtained, the bending radius of the real cable can be determined by the curve.

[0106] In one embodiment, in the above step 108, determining the bending radius of the real cable according to the curve in the cable bending plane may specifically include:

[0107] The least square method is used to process the curve to obtain the bending radius of the actual cable.

[0108] In the embodiment of the present disclosure, the least square method can be used to fit the curve to obtain the bending radius of the real cable. In fact, the method of curve processing in the embodiment of the present disclosure is not specifically limited, and any method that can calculate the bending radius of the curve is applicable to the embodiment of the present disclosure.

[0109] The above-mentioned cable bending radius detection method obtains the first image information collected in the real cable space, and the first image information includes the positioning mark image and the real cable. The positioning mark image is an image preset at a preset position in the real cable space, and the projection matrix of the camera can be determined according to the coordinate information of the positioning mark image. Furthermore, the real cable can be reverse-projected according to the projection matrix of the camera to obtain the corresponding curve of the real cable in the cable bending plane, and the bending radius of the real cable can be determined according to the curve in the cable bending plane. The cable bending radius detection method provided by the embodiment of the present disclosure can realize the detection of the cable bending radius through the projection matrix of the camera and the collected plane image, without the need for complex equipment such as a depth camera, thereby reducing the computational complexity and improving the detection efficiency and detection accuracy.

[0110] In one embodiment, Figure 7 As shown, in step 106, the real cable is back-projected according to the projection matrix of the camera to obtain the curve corresponding to the real cable in the cable bending plane, which may specifically include:

[0111] Step 702, extracting the cable backbone of the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information;

[0112] Step 704, establishing a cable bending plane according to coordinate information of a virtual cable corresponding to a real cable in the virtual cable space in the world coordinate system, wherein the virtual cable space is a three-dimensional model of the real cable space, and a unified world coordinate system is established between the real cable space and the virtual cable space;

[0113] Step 706: perform a reverse projection process on the cable backbone curve according to the projection matrix of the camera to obtain a curve corresponding to the cable backbone curve in the cable bending plane.

[0114] In the disclosed embodiment, after collecting the first image information in the real cable space, the cable backbone curve corresponding to the real cable can be extracted from the first image information, that is, the content in the first image information irrelevant to the real cable to be detected is eliminated.

[0115] The process of establishing the virtual cable space can refer to the aforementioned embodiment, which will not be described in detail in the embodiments of the present disclosure. The virtual cable space and the real cable space deploy clips at the same position, and the cables are laid based on the deployed clips, that is, there are virtual cables in the virtual cable space corresponding to the real cables in the real cable space. The coordinate information of the two endpoints P1 and P2 of the virtual cable and the middle point P3 of the virtual cable in the world coordinate system can be read from the virtual cable space, and the plane where the points P1, P2, and P3 are located is taken as the cable bending plane S. It can be approximately considered that the bending of the virtual cable occurs in the cable bending plane S, where the cable bending plane S can be referred to. Figure 6 shown.

[0116] In actual installation, the position accuracy of the buckle is relatively high, and the position of the buckle defines the starting and ending points of the cable bending section. Therefore, the cable bending plane where the real cable is located is basically coincident with the cable bending plane where the virtual cable is located. Therefore, the cable bending plane S can be used as the cable bending plane where the real cable is located.

[0117] After obtaining the cable bending plane and the cable backbone curve, the cable backbone curve can be back-projected according to the projection matrix of the camera based on spatial back-projection processing to obtain a curve corresponding to the cable backbone curve in the cable bending plane.

[0118] In one embodiment, referring to Figure 8 As shown, in step 706, the cable backbone curve is back-projected according to the projection matrix of the camera to obtain a curve corresponding to the cable backbone curve in the cable bending plane, which may specifically include:

[0119] Step 802, for any pixel point in the cable backbone curve, determine the coordinate information of the pixel point in the world coordinate system according to the coordinate information corresponding to the pixel point in the first image information and the projection matrix;

[0120] Step 804, converting the coordinate information of each pixel point in the world coordinate system into the two-dimensional coordinate information corresponding to each pixel point in the cable bending plane;

[0121] Step 806: Obtain a curve corresponding to the cable backbone curve in the cable bending plane according to the two-dimensional coordinate information corresponding to each pixel point.

[0122] In the embodiments of the present disclosure, for example, reference can be made to Fig. 9 As shown, for any pixel point on the cable backbone curve, its corresponding coordinate information in the first image information can be determined as m=(u,v) T (That is, in Fig. 9 The coordinate information in the camera imaging plane shown in the figure) is obtained according to the camera projection matrix and the coordinate information m = (u, v)T You can get its coordinate information in the world coordinate system M = (X, Y, Z) T .

[0123] According to the camera imaging principle, the first image information and the coordinates of the world coordinate system satisfy the projection relationship: P is the camera projection matrix K[R|t]. is the homogeneous coordinate corresponding to the two-dimensional coordinate of the pixel point m in the first image information, is the homogeneous coordinate of the three-dimensional point corresponding to the pixel point M in the world coordinate system. Expanding the projection equation yields:

[0124]

[0125] According to the meaning of homogeneous coordinates, we have:

[0126]

[0127] The above formula is organized into a system of equations about X, Y, and Z. Since the pixel point M is a pixel point on the cable backbone curve, and the above description has assumed that the real cable and the virtual cable are both on the same cable bending plane S, the parameter expression of the cable bending plane S can be obtained based on the coordinates of the endpoints P1 and P2 of the virtual cable and the coordinates of the midpoint P3, which is recorded as AX+BY+CZ=1. The three formulas are combined to obtain the parameter expression of the cable bending plane S, refer to the following formula (I).

[0128]

[0129] By solving the equation group, the two-dimensional coordinates of the corresponding cable backbone curve point M on the cable bending plane S can be obtained according to the pixel coordinates of any point on the cable backbone curve. Similarly, by performing a reverse projection operation on all pixel points on the cable backbone curve, the two-dimensional coordinate point set of the pixel points constituting the cable backbone curve on the cable bending plane S can be obtained. After performing smoothing processing such as refining, deburring, and debifurcating the two-dimensional coordinate information corresponding to each pixel point in the two-dimensional coordinate point set, the curve corresponding to the cable backbone curve in the cable bending plane can be obtained ( Fig. 9 The cable space backbone curve shown in ).

[0130] For example, the pixel points of the cable backbone curve can be assembled to obtain a spatial point set U world , the three-dimensional coordinates of each pixel point in the spatial point set are converted into a point set U of two-dimensional coordinates in a new coordinate system on the cable bending plane S s , in order to simplify the calculation process of the bending radius. We can take the point set U world The first point O and the last point Q in the equation are given by O as the origin. is the positive direction of the x-axis, the normal vector of plane S is the positive direction of the z-axis, then for the point set U world For any point N except point O, its coordinates in the new plane coordinate system are (x N ,y N ), space point set U world like Fig.10a As shown, the new point set U s like Fig.10b shown.

[0131] Among them, x N ,y N For details, please refer to the following formula (2).

[0132]

[0133] In one embodiment, referring to Fig.11 As shown in the figure, the bending area of ​​the virtual cable in the virtual cable space is included in the mask. Fig.12 As shown, in the above step 704, the cable backbone is extracted from the first image information to obtain the cable backbone curve corresponding to the real cable in the first image information, which may specifically include:

[0134] Step 1202, determining the position and posture information of the camera in the world coordinate system;

[0135] Step 1204, controlling the virtual camera to collect second image information in the virtual cable space according to the position and posture information, where the second image information includes the virtual cable;

[0136] Step 1206, generating a mask image according to the mask corresponding to the bending area of ​​the virtual cable in the second image information;

[0137] Step 1208, determining a real cable area image from the first image information according to the mask image;

[0138] Step 1210: extract the cable backbone of the real cable area image to obtain a cable backbone curve corresponding to the real cable.

[0139] For example, according to the relevant regulations on cable installation, the cable simulation installation can be performed in the virtual cable space first, and a mask can be formed in the bending area of ​​the virtual cable. Exemplarily, the mask can be a spherical mask, the diameter of which can be three times the minimum bending radius of the cable, and the spherical mask contains the bending section of the virtual cable inside the sphere. According to the process requirements, when installing cables in the spacecraft, they need to be fixed to the cabin board at certain intervals by buckles, so the position offset between the real cable and the virtual cable will not be large.

[0140] The position information of the camera in the world coordinate system can be determined. The specific process can refer to the relevant description in the above embodiment, and the embodiment of the present disclosure will not be repeated here. After obtaining the position information of the camera in the world coordinate system, the virtual camera can be controlled to collect the second image information including the virtual cable in the virtual cable space with the position information.

[0141] After obtaining the second image information, the image outside the mask area in the second image information can be removed to obtain a mask image containing only the circular mask area. For example, the pixel value of the image outside the mask area in the second image information is set to 0, and the pixel value in the mask area is set to 1, thereby obtaining a mask image.

[0142] The mask image is superimposed on the first image information. For example, the mask image is multiplied by the first image information to obtain a real cable area image including the bending area of ​​the real cable. This can reduce the impact of a large amount of interference information in the first image information on detection, and concentrate the computing power of the device on processing the local image of the cable bending, thereby improving the calculation accuracy and detection efficiency.

[0143] After obtaining the real cable area image, the cable backbone can be extracted from the real cable image to obtain the cable backbone curve corresponding to the curved part of the real cable. For example, the threshold method can be used to extract the cable itself from the real cable image to obtain the cable itself, and the buckles can be detected at the same time. The buckles used to fix the curved part of the cable are used to cut the cable, and only the cable information of the curved part is retained; then, through the image thinning method (starting from the side boundary of the long strip area, the boundary points are gradually deleted until the area width is a single pixel), only the center line of the long strip cable area is retained, that is, the backbone curve of the cable. Fig. 9 As shown, the cable backbone curve obtained at this time (i.e., the center line of the cable in the real cable area image) is the projection of the cable backbone curve of the real cable in space on the first image information of the camera imaging plane. Then, by back-projecting the cable backbone curve in the plane image to the spatial plane, the curve of the cable backbone curve on the cable bending plane can be obtained, and then the least squares method is performed on the curve to obtain the bending radius of the real cable.

[0144] Exemplarily, after obtaining the real cable area image, the real cable area image can be grayscaled, and the cable can be separated from the background by threshold segmentation to obtain a first processed image. The first processed image is subjected to denoising to remove noise points to obtain a second processed image. The second processed image is subjected to opening and closing operations to eliminate areas that are too small and disconnect larger connected domains to obtain a third processed image. Since the buckle divides the cable into multiple segments, and the cable bending segment between the two buckles is the main part of the third processed image, the maximum connected domain method is used for detection to retain the largest connected area in the third processed image, that is, the cable bending segment, to obtain a fourth processed image. The fourth image is refined to extract the cable backbone curve. For details, please refer to Figure 13a to Figure 13d ,in, Fig.13a It shows the real cable area image. Fig.13b represents the first processed image, Fig.13c represents the fourth processed image, Fig.13d Shown is an image including the cable backbone curve.

[0145] In order to enable those skilled in the art to better understand the embodiments of the present disclosure, the embodiments of the present disclosure are described below through specific examples.

[0146] For example, refer to Fig.14 As shown, with respect to the real cable space being the spacecraft cabin to be inspected, a corresponding three-dimensional model of the spacecraft cabin can be established as a virtual cable space. The three-dimensional model includes models corresponding to objects such as cables, buckles, cabin panels, and equipment. The buckle model and the equipment model should be installed in fixed positions, and the cable model should be set strictly along the prescribed route to meet the corresponding minimum bending radius requirements, etc.

[0147] The camera can be calibrated to obtain the camera's intrinsic parameter matrix, and a virtual positioning marker image can be set in the virtual cable space, and a real positioning marker image can be posted at the same position in the real cable space. The positioning marker image can be detected by the camera, and the camera's posture information relative to the positioning marker image can be output. The same world coordinate system is created at the same position in the real cable space and the virtual cable space, and the posture of the marker plane coordinate system is recorded, and then the posture information of the camera in the world coordinate system, that is, the camera's extrinsic parameter matrix, is solved, and then the camera's projection matrix is ​​determined.

[0148] For example, it is possible to obtain an image of a real cable area, that is, a local image of a cable bending area. It is difficult to accurately extract the bending area of ​​the cable by directly detecting a complete image containing multiple cables. The embodiment of the present disclosure proposes an attention mechanism based on virtual-real space registration and a cable virtual model to specifically capture the image of the cable bending area, thereby improving the detection efficiency and reducing the false detection rate. The specific steps are as follows:

[0149] In the virtual cable space, a spherical mask area is generated at the bending position of the virtual cable model, and the bending section of the virtual cable is included in the mask area. A camera is used to capture an actual image containing the real cable, and a virtual camera is used to capture a virtual image containing the virtual cable and the spherical mask area from the same viewing angle in the virtual cable space. The virtual image is then processed to extract the spherical area of ​​the virtual image and form a mask image. The mask image is superimposed on the actual image to obtain an image of the real cable area with only the cable bending area retained, and the Hough circle detection method is used to detect the highlight area on the real cable area image with only the cable bending area retained, and the overall image is segmented to form several local images centered on the cable bending area.

[0150] After obtaining the local image of the cable bending area, it is necessary to identify and segment the cables in the local image, retaining only the bending section between the two buckles, and then thinning the cables to extract the backbone curve of the cables. Exemplarily, the local image can be grayed out, and the cable can be separated from the background by threshold segmentation to obtain a first processed image. The first processed image is subjected to denoising to remove noise points to obtain a second processed image, and the second processed image is subjected to opening and closing operations to eliminate too small areas and disconnect larger connected domains to obtain a third processed image. Since the buckle divides the cable into multiple sections, and the bending section of the cable between the two buckles is the main part of the image, the maximum connected domain method is used for detection to retain the largest connected area in the local image, that is, the bending section of the cable, to obtain a fourth processed image. The image is thinned to extract the backbone curve of the cable.

[0151] The bending of the cable can be approximately considered to occur on a spatial plane, and this spatial plane is defined as the cable bending plane. The bending of the cable in the plane image cannot fully reflect the bending degree of the cable in space, but based on the known camera posture and the cable bending plane, the cable backbone curve in the plane image can be back-projected onto the spatial plane through the back-projection method, thus obtaining the actual backbone curve of the cable in space.

[0152] For example, the two ends of the cable bending section are fixed by buckles, and the buckle installation precision is high. The buckle position in the actual environment is basically consistent with the buckle position in the virtual environment, so the cable bending plane in the actual environment is approximately consistent with the cable bending plane in the virtual environment. By analyzing the cable bending section in the virtual environment, the parameter expression of the cable bending plane in the world coordinate system can be obtained.

[0153] By using the reverse projection method, the pixel point set of the cable backbone curve can be reversely projected onto the spatial plane where the cable backbone theory is located to obtain the spatial point set of the cable backbone curve. Given the real-time projection matrix of the camera and the parameter expression of the cable bending plane in the world coordinate system, the coordinate information of the pixel points on the cable backbone curve in the world coordinate system can be obtained by reverse projection. The derivation process of reverse projection can refer to the relevant description of the aforementioned embodiment, and the embodiment of the present disclosure will not be repeated here.

[0154] Furthermore, the three-dimensional coordinates of the cable backbone spatial point set are converted into a two-dimensional coordinate point set in a new coordinate system on the cable bending plane to simplify the calculation process of the bending radius. The conversion process can be specifically referred to the relevant description of the aforementioned embodiment, and the embodiment of the present disclosure will not be repeated here. After obtaining the two-dimensional coordinate point set, it is necessary to process the twists and turns, bifurcations, burrs, etc. on the two-dimensional coordinate point set, and use local weighted regression to obtain a smooth curve of the cable backbone, and then use the least squares method to fit the curve, so as to calculate the bending radius of the cable segment.

[0155] The disclosed embodiment discloses a cable bending radius detection method based on the camera reverse projection principle and virtual assembly environment. Based on the positioning mark image, the camera's posture information in the world coordinate system can be calculated, and the positioning mark image with the same posture in the virtual and real cable space is used to match and align the virtual and real cable spaces. A spherical mask is added to the bending section position of the virtual cable model in the virtual cable space, and the circular mask area generated by the spherical mask in the virtual image of the same viewing angle of the camera is superimposed on the actual image, thereby obtaining a local image of the cable bending section, extracting the cable backbone curve through a visual processing method, and reversely projecting the cable backbone curve to the cable bending plane where the cable bending section is located based on the camera reverse projection principle. Finally, the spatial bending radius of the cable can be obtained according to the curve in the cable bending plane. The disclosed embodiment detects the cable bending radius based on a visual method, without direct contact, with fast detection speed and high flexibility. The spatial bending radius of the cable is calculated based on the camera posture and the plane image, without the need for three-dimensional reconstruction using a depth image or point cloud, with a small amount of calculation, and more accurate detection results.

[0156] It should be understood that although Figure 1-14 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-14At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0157] In one embodiment, Fig.15 As shown, a cable bending radius detection device is provided, including: an acquisition module 1502, a first determination module 1504, a processing module 1506, and a second determination module 1508, wherein:

[0158] An acquisition module 1502 is used to acquire first image information collected in a real cable space, wherein the first image information includes a positioning mark image and image information of a real cable, wherein the positioning mark image is an image preset at a preset position in the real cable space;

[0159] A first determination module 1504 is used to determine the projection matrix of the camera according to the coordinate information of the positioning mark image;

[0160] A processing module 1506 is used to perform a back-projection process on the real cable according to the projection matrix of the camera to obtain a curve corresponding to the real cable in the cable bending plane;

[0161] The second determining module 1508 is configured to determine the bending radius of the real cable according to the curve in the cable bending plane.

[0162] The above-mentioned cable bending radius detection device obtains the first image information collected in the real cable space, and the first image information includes the positioning mark image and the real cable. The positioning mark image is an image preset at a preset position in the real cable space, and the projection matrix of the camera can be determined according to the coordinate information of the positioning mark image. Furthermore, the real cable can be reversely projected according to the projection matrix of the camera to obtain the corresponding curve of the real cable in the cable bending plane, and the bending radius of the real cable can be determined according to the curve in the cable bending plane. The cable bending radius detection device provided by the embodiment of the present disclosure can realize the detection of the bending radius of the cable through the projection matrix of the camera and the collected plane image, without the need for complex equipment such as a depth camera, thereby reducing the calculation complexity and improving the detection efficiency and detection accuracy.

[0163] In one embodiment, the first determining module 1504 is further configured to:

[0164] Determine the coordinate information of the feature point in the world coordinate system according to the coordinate information of the feature point in the positioning mark image in the marking plane coordinate system and the preset position corresponding to the positioning mark image;

[0165] A projection matrix of a camera is determined according to the coordinate information of the feature point in the first image information and the coordinate information of the feature point in the world coordinate system.

[0166] In one embodiment, the processing module 1506 is further configured to:

[0167] Performing cable backbone extraction on the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information;

[0168] Establishing a cable bending plane according to coordinate information of a virtual cable corresponding to the real cable in the virtual cable space in a world coordinate system, wherein the virtual cable space is a three-dimensional model of the real cable space, and a unified world coordinate system is established between the real cable space and the virtual cable space;

[0169] According to the projection matrix of the camera, the cable backbone curve is subjected to a back-projection process to obtain a curve corresponding to the cable backbone curve in the cable bending plane.

[0170] In one embodiment, the bending area of ​​the virtual cable in the virtual cable space is included in a mask, and the processing module 1506 is further configured to:

[0171] Determine the position and posture information of the camera in the world coordinate system;

[0172] According to the position information, controlling the virtual camera to collect second image information in the virtual cable space, wherein the second image information includes the virtual cable;

[0173] generating a mask image according to the mask corresponding to the bending area of ​​the virtual cable in the second image information;

[0174] determining a real cable area image from the first image information according to the mask image;

[0175] A cable backbone is extracted from the real cable region image to obtain a cable backbone curve corresponding to the real cable.

[0176] In one embodiment, the processing module 1506 is further configured to:

[0177] For any pixel point in the cable backbone curve, determine the coordinate information of the pixel point in the world coordinate system according to the coordinate information corresponding to the pixel point in the first image information and the projection matrix;

[0178] Converting the coordinate information of each pixel point in the world coordinate system into two-dimensional coordinate information corresponding to each pixel point in the cable bending plane;

[0179] According to the two-dimensional coordinate information corresponding to each of the pixel points, a curve corresponding to the cable backbone curve in the cable bending plane is obtained.

[0180] In one embodiment, the second determining module 1508 is further configured to:

[0181] The curve is processed by using the least square method to obtain the bending radius of the real cable.

[0182] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0183] Acquire first image information collected in a real cable space, the first image information including a positioning mark image and image information of a real cable, the positioning mark image being an image preset at a preset position in the real cable space;

[0184] Determining the projection matrix of the camera according to the coordinate information of the positioning mark image;

[0185] Performing a back-projection process on the real cable according to the projection matrix of the camera to obtain a curve corresponding to the real cable in the cable bending plane;

[0186] The bending radius of the real cable is determined according to the curve in the cable bending plane.

[0187] For the specific definition of the cable bending radius detection device, please refer to the definition of the cable bending radius detection method above, which will not be repeated here. Each module in the above-mentioned cable bending radius detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0188] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.16As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a cable bending radius detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0189] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0190] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0191] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0192] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0193] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A cable bending radius detection method, characterized in that: The method comprises: Acquire first image information collected in a real cable space, the first image information including a positioning mark image and image information of a real cable, the positioning mark image being an image preset at a preset position in the real cable space; Determining the projection matrix of the camera according to the coordinate information of the positioning mark image; Performing cable backbone extraction on the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information; Establishing a cable bending plane according to coordinate information of a virtual cable corresponding to the real cable in the virtual cable space in a world coordinate system, wherein the virtual cable space is a three-dimensional model of the real cable space, and a unified world coordinate system is established between the real cable space and the virtual cable space; For any pixel point in the cable backbone curve, determine the coordinate information of the pixel point in the world coordinate system according to the coordinate information corresponding to the pixel point in the first image information and the projection matrix; Converting the coordinate information of each pixel point in the world coordinate system into two-dimensional coordinate information corresponding to each pixel point in the cable bending plane; Obtaining a curve corresponding to the cable backbone curve in the cable bending plane according to the two-dimensional coordinate information corresponding to each of the pixel points; The bending radius of the real cable is determined according to the curve in the cable bending plane.

2. The method according to claim 1, characterized in that Determining the projection matrix of the camera according to the coordinate information of the positioning mark image includes: Determine the coordinate information of the feature point in the world coordinate system according to the coordinate information of the feature point in the positioning mark image in the marking plane coordinate system and the preset position corresponding to the positioning mark image; A projection matrix of a camera is determined according to the coordinate information of the feature point in the first image information and the coordinate information of the feature point in the world coordinate system.

3. The method according to claim 1, characterized in that The bending area of ​​the virtual cable in the virtual cable space is contained in a mask, The extracting the cable backbone of the first image information to obtain a cable backbone curve corresponding to the real cable in the first image information includes: Determine the position and posture information of the camera in the world coordinate system; According to the position information, controlling the virtual camera to collect second image information in the virtual cable space, wherein the second image information includes the virtual cable; generating a mask image according to the mask corresponding to the bending area of ​​the virtual cable in the second image information; determining a real cable area image from the first image information according to the mask image; A cable backbone is extracted from the real cable region image to obtain a cable backbone curve corresponding to the real cable.

4. The method according to claim 1, characterized in that: The determining the bending radius of the real cable according to the curve in the cable bending plane includes: The curve is processed by using the least square method to obtain the bending radius of the real cable.

5. A cable bending radius detection device, characterized in that: The device comprises: An acquisition module, used to acquire first image information collected in a real cable space, wherein the first image information includes a positioning mark image and image information of a real cable, wherein the positioning mark image is an image preset at a preset position in the real cable space; A first determination module, used to determine the projection matrix of the camera according to the coordinate information of the positioning mark image; A processing module, used to extract the cable backbone of the first image information to obtain the cable backbone curve corresponding to the real cable in the first image information; establish a cable bending plane according to the coordinate information of the virtual cable corresponding to the real cable in the virtual cable space in the world coordinate system, wherein the virtual cable space is a three-dimensional model of the real cable space, and a unified world coordinate system is established between the real cable space and the virtual cable space; for any pixel point in the cable backbone curve, determine the coordinate information of the pixel point in the world coordinate system according to the coordinate information corresponding to the pixel point in the first image information and the projection matrix; convert the coordinate information of each pixel point in the world coordinate system into the two-dimensional coordinate information corresponding to each pixel point in the cable bending plane; obtain the curve corresponding to the cable backbone curve in the cable bending plane according to the two-dimensional coordinate information corresponding to each pixel point; The second determination module is used to determine the bending radius of the real cable according to the curve in the cable bending plane.

6. The device according to claim 5, characterized in that The first determining module is further used for: Determine the coordinate information of the feature point in the world coordinate system according to the coordinate information of the feature point in the positioning mark image in the marking plane coordinate system and the preset position corresponding to the positioning mark image; A projection matrix of a camera is determined according to the coordinate information of the feature point in the first image information and the coordinate information of the feature point in the world coordinate system.

7. The device according to claim 5, characterized in that The bending area of ​​the virtual cable in the virtual cable space is included in the mask, and the processing module is further used for: Determine the position and posture information of the camera in the world coordinate system; According to the position information, controlling the virtual camera to collect second image information in the virtual cable space, wherein the second image information includes the virtual cable; generating a mask image according to the mask corresponding to the bending area of ​​the virtual cable in the second image information; determining a real cable area image from the first image information according to the mask image; A cable backbone is extracted from the real cable region image to obtain a cable backbone curve corresponding to the real cable.

8. The device according to claim 5, characterized in that The second determining module is further used for: The curve is processed by using the least square method to obtain the bending radius of the real cable.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images