Wire harness bending radius measuring method, device, equipment and medium
Through binocular structured optical camera and convolutional neural network technology, the bending radius of the aircraft installation wiring harness is identified and calculated, and the problems of low efficiency and poor accuracy of traditional methods are solved, efficient and accurate bending radius measurement is achieved, and the quality of aircraft assembly is improved. Security.
Patent Information
- Application Number
- CN202410974354.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-19
AI Technical Summary
Traditional methods are inefficient, low accuracy, and prone to errors when measuring the bend radius of the installed aircraft wiring harness, which may cause aircraft safety risks.
A binocular structured light camera is used to collect two-dimensional images and three-dimensional point cloud data, and the contour points of the wire harness are identified through an instance segmentation model based on a convolutional neural network, the bending radius is calculated, and the K-mean clustering and curve fitting methods are used for accurate measurement.
It realizes more accurate and efficient measurement of the bending radius of the wiring harness, with the accuracy increased to 0.5mm and the detection time shortened to 10 seconds, solving the efficiency and accuracy problems of traditional methods.
Smart Images

Figure CN118913146A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of aircraft manufacturing, and provides a method, device, equipment and medium for measuring the bending radius of a wire harness. Background Art
[0002] As we all know, in the quality control of aircraft wiring harnesses, if the bending radius of the wiring harness is too small, it will not only fail to meet the design requirements, but may also cause the harness to break, thus causing aircraft safety risks. Therefore, accurate measurement of the bending radius of the wiring harness is a crucial part of ensuring the overall product quality. In traditional methods, the "gauge comparison" method is often used to measure the bending radius of the wiring harness. However, this measurement method has problems such as long operation time, low efficiency, low accuracy and easy errors.
[0003] Therefore, how to measure the bending radius of the wire harness more accurately and efficiently has become a problem that needs to be solved urgently. Summary of the invention
[0004] The present application provides a method, device, equipment and medium for measuring the bending radius of a wire harness, which are used to solve the problems of low efficiency and poor accuracy in measuring the bending radius of a wire harness in the prior art.
[0005] In one aspect, a method for measuring a bending radius of a wire harness is provided, the method comprising:
[0006] A binocular structured light camera is used to collect images of the target harness to obtain a two-dimensional image and the corresponding three-dimensional point cloud data of the target;
[0007] Inputting the two-dimensional image and the corresponding target three-dimensional point cloud data into a trained image segmentation model to obtain a plurality of contour points of the target harness; wherein the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network;
[0008] Extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness;
[0009] Calculating the bending radius corresponding to the contours of the wire harness on both sides according to the two characteristic points corresponding to the contours of the wire harness on both sides;
[0010] The bending radius of the target wire harness is calculated according to the bending radii corresponding to the wire harness profiles on both sides.
[0011] Optionally, before using a binocular structured light camera to capture an image of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data, the method further includes:
[0012] The binocular structured light camera is used to collect images of the wiring harness to be inspected from different angles and distances to obtain multiple collected images;
[0013] Batch importing the multiple acquired images into the segmentation and annotation software, annotating the wire harness contours in the multiple acquired images, and obtaining multiple annotated acquired images;
[0014] The multiple annotated collected images are used to train the initial image segmentation model to obtain a trained image segmentation model.
[0015] Optionally, the step of using a binocular structured light camera to capture an image of the target harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data includes:
[0016] Using a binocular structured light camera to collect images of the target wire harness to obtain the two-dimensional image and corresponding initial three-dimensional point cloud data;
[0017] Filtering the initial three-dimensional point cloud data to obtain target three-dimensional point cloud data;
[0018] Matching and aligning the two-dimensional image with the target three-dimensional point cloud data to obtain a matched two-dimensional image;
[0019] The matched two-dimensional image and the target three-dimensional point cloud data are stored.
[0020] Optionally, the step of storing the matched two-dimensional image and the target three-dimensional point cloud data includes:
[0021] The matched two-dimensional image is stored in the original size and png format;
[0022] The target three-dimensional point cloud data is stored in a preset data structure in an xyz format.
[0023] Optionally, the step of extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness, respectively, comprises:
[0024] For any two adjacent contour points, determine the direction of the contour point vector corresponding to the any two adjacent contour points;
[0025] For the directions of any two adjacent contour point vectors, determining the angular change amount of any two adjacent contour point vectors;
[0026] Performing cluster analysis on the plurality of contour points by using a K-means clustering method to determine a plurality of first contour point groups; wherein the first contour point groups include contour points and other points of the contours of the two side wire harnesses of the target wire harness;
[0027] performing outlier removal on the plurality of first contour point groups to obtain a plurality of second contour point groups;
[0028] According to the angle variation of any two adjacent contour point vectors, the plurality of second contour points are grouped to perform feature point extraction to obtain two feature points corresponding to the two side harness contours of the target harness respectively; wherein the feature points are endpoints or inflection points.
[0029] Optionally, the step of calculating the bending radius corresponding to the wire harness profiles on both sides according to the two feature points corresponding to the wire harness profiles on both sides respectively comprises:
[0030] According to the two feature points corresponding to the two side harness contours, the key points of the two side harness contours are extracted in sequence to determine the first section harness contour and the second section harness contour corresponding to the target harness; wherein the first section harness contour and the second section harness contour are located on the two sides of the target harness, respectively;
[0031] Calculating the bending radius of the first wire harness profile by using a curve fitting or curvature calculation method;
[0032] The bending radius of the second wire harness profile is calculated by using a curve fitting or curvature calculation method.
[0033] Optionally, the step of calculating the bending radius of the target wire harness according to the bending radii corresponding to the wire harness profiles on both sides includes:
[0034] The bending radius of the target wire harness is calculated by averaging the bending radius of the first wire harness profile and the second wire harness profile.
[0035] In one aspect, a wire harness bending radius measuring device is provided, the device comprising:
[0036] A point cloud data acquisition unit is used to acquire an image of the target harness using a binocular structured light camera to obtain a two-dimensional image and corresponding three-dimensional point cloud data of the target;
[0037] A contour point obtaining unit, used for inputting the two-dimensional image and the corresponding target three-dimensional point cloud data into a trained image segmentation model to obtain a plurality of contour points of the target line harness; wherein the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network;
[0038] A feature point obtaining unit, used for extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness;
[0039] A first bending radius calculation unit, used for calculating the bending radius corresponding to the contours of the wire harness on both sides according to two feature points corresponding to the contours of the wire harness on both sides;
[0040] The second bending radius calculation unit is used to calculate the bending radius of the target wire harness according to the bending radii corresponding to the wire harness profiles on both sides.
[0041] On the one hand, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein any one of the above methods is implemented when the processor executes the computer program.
[0042] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, any of the above methods is implemented.
[0043] Compared with the prior art, the beneficial effects of this application are:
[0044] In an embodiment of the present application, when measuring the bending radius of the wire harness, first, a binocular structured light camera can be used to capture images of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data; then, the two-dimensional image and the corresponding target three-dimensional point cloud data can be input into a trained image segmentation model to obtain multiple contour points of the target wire harness; wherein the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network; next, feature point extraction can be performed on the multiple contour points to obtain two feature points corresponding to the contours of the two sides of the target wire harness; then, the bending radii corresponding to the contours of the two side harnesses are calculated based on the two feature points corresponding to the contours of the two side harnesses; finally, the bending radius of the target wire harness can be calculated based on the bending radii corresponding to the contours of the two side harnesses. Therefore, in the embodiment of the present application, a "binocular structured light camera" is used to capture images of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data, and a trained image segmentation model is used to identify and extract the wire harness contour points to calculate the bending radius of the target wire harness. That is, the present application overcomes the limitations of the traditional "gauge comparison" method by combining binocular vision and structured light technology, and achieves more accurate and efficient measurement of the bending radius of the wire harness. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0046] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;
[0047] Figure 2A schematic diagram of a flow chart of a method for measuring the bending radius of a wire harness provided in an embodiment of the present application;
[0048] Figure 3 A schematic diagram of a connection between a cabin installation wiring harness and a binocular structured light camera provided in an embodiment of the present application;
[0049] Figure 4 A detection schematic diagram of an image segmentation model provided in an embodiment of the present application;
[0050] Figure 5 A schematic diagram of the direction angle value of the contour point vector provided in an embodiment of the present application;
[0051] Figure 6 A schematic diagram of the direction angle change of the contour point vector provided in an embodiment of the present application;
[0052] Figure 7 A schematic diagram of extracting characteristic points of an installed wiring harness provided in an embodiment of the present application;
[0053] Figure 8 A schematic diagram of a wire harness bending radius measuring device provided in an embodiment of the present application.
[0054] Markings in the figure: 10 - wire harness bending radius measuring device, 101 - processor, 102 - memory, 103 - I / O interface, 104 - database, 80 - wire harness bending radius measuring device, 801 - point cloud data acquisition unit, 802 - contour point acquisition unit, 803 - feature point acquisition unit, 804 - first bending radius calculation unit, 805 - second bending radius calculation unit, 806 - model training unit. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0056] As we all know, in the quality control of aircraft wiring harnesses, if the bending radius of the wiring harness is too small, it will not only fail to meet the design requirements, but may also cause the harness to break, thus causing aircraft safety risks. Therefore, accurate measurement of the bending radius of the wiring harness is a crucial part of ensuring the overall product quality. In traditional methods, the "gauge comparison" method is often used to measure the bending radius of the wiring harness. However, this measurement method has problems such as long operation time, low efficiency, low accuracy and easy errors.
[0057] Based on this, an embodiment of the present application provides a method for measuring the bending radius of a wire harness, in which, first, a binocular structured light camera can be used to capture images of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data; then, the two-dimensional image and the corresponding target three-dimensional point cloud data can be input into a trained image segmentation model to obtain multiple contour points of the target wire harness; wherein the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network; next, feature point extraction can be performed on multiple contour points to obtain two feature points corresponding to the contours of the two sides of the target wire harness; then, the bending radii corresponding to the contours of the two sides of the wire harness are calculated based on the two feature points corresponding to the contours of the two sides of the wire harness; finally, the bending radius of the target wire harness can be calculated based on the bending radii corresponding to the contours of the two sides of the wire harness. Therefore, in the embodiment of the present application, a "binocular structured light camera" is used to capture images of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data, and a trained image segmentation model is used to identify and extract the wire harness contour points to calculate the bending radius of the target wire harness. That is, the present application overcomes the limitations of the traditional "gauge comparison" method by combining binocular vision and structured light technology, and achieves more accurate and efficient measurement of the bending radius of the wire harness.
[0058] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.
[0059] like Figure 1 FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario may include a wire harness bending radius measuring device 10 .
[0060] The wire harness bending radius measuring device 10 can be used to measure the bending radius of the wire harness, for example, it can be a personal computer (PC), a server and a laptop. The wire harness bending radius measuring device 10 may include one or more processors 101, a memory 102, an I / O interface 103 and a database 104. Specifically, the processor 101 may be a central processing unit (CPU), or a digital processing unit, etc. The memory 102 may be a volatile memory, such as a random-access memory (RAM); the memory 102 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 102 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 102 may be a combination of the above memories. The memory 102 may store some program instructions of the wire harness bending radius measurement method provided in the embodiment of the present application, and these program instructions can be used to implement the steps of the wire harness bending radius measurement method provided in the embodiment of the present application when executed by the processor 101, so as to solve the problem of low efficiency and poor accuracy in measuring the bending radius of the wire harness in the prior art. The database 104 may be used to store data such as two-dimensional images, target three-dimensional point cloud data, trained image segmentation models, multiple contour points of the target wire harness, the bending radius corresponding to each of the two side wire harness contours, the bending radius of the target wire harness, and initial three-dimensional point cloud data involved in the solution provided in the embodiment of the present application.
[0061] In the embodiment of the present application, the wire harness bending radius measuring device 10 can obtain the two-dimensional image and the corresponding target three-dimensional point cloud data through the I / O interface 103, and then the processor 101 of the wire harness bending radius measuring device 10 will measure the wire harness bending radius more accurately and efficiently according to the program instructions of the wire harness bending radius measuring method provided in the embodiment of the present application in the memory 102. In addition, the two-dimensional image, the target three-dimensional point cloud data, the trained image segmentation model, multiple contour points of the target wire harness, the bending radius corresponding to each of the two sides of the wire harness contour, the bending radius of the target wire harness, and the initial three-dimensional point cloud data and other data can also be stored in the database 104.
[0062] Of course, the method provided in the embodiment of the present application is not limited to Figure 1The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be realized by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here. Below, the method of the embodiment of the present application will be introduced in conjunction with the accompanying drawings.
[0063] like Figure 2 FIG. 1 is a flow chart of a method for measuring the bending radius of a wire harness provided in an embodiment of the present application. The method can be performed by Figure 1 The method is performed by the wire harness bending radius measuring device 10, and specifically, the process of the method is described as follows.
[0064] Step 201: Use a binocular structured light camera to capture an image of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data.
[0065] In the embodiments of the present application, Figure 3 As shown, it is a connection schematic diagram of the cabin installation harness and the binocular structured light camera provided in the embodiment of the present application, wherein the binocular structured light camera is installed on a bracket. In practical applications, the binocular structured light camera can be set, and the binocular structured light camera can be connected to a control processing unit (for example, a personal computer, a server, etc.) to shoot the cabin installation harness. Specifically, the shooting, parameters, and storage results of the binocular structured light camera can be controlled by the control program of the control processing unit.
[0066] Furthermore, when measuring the bending radius of the harness, first, a binocular structured light camera can be used to scan the harness area in the cabin to capture the image of the target harness to obtain a two-dimensional image and the corresponding initial three-dimensional point cloud data; wherein, when performing image capture, it is necessary to ensure that the angle and position of the binocular structured light camera can cover the entire harness area. Then, the initial three-dimensional point cloud data can be processed and filtered to obtain the target three-dimensional point cloud data to remove possible noise points or abnormal points, thereby ensuring the accuracy of subsequent data analysis; next, the two-dimensional image can be matched and aligned with the target three-dimensional point cloud data to ensure a one-to-one correspondence between the two, thereby obtaining a matched two-dimensional image; then, a corresponding data structure can be created so that the matched two-dimensional image and the target three-dimensional point cloud data can be correctly called and processed in subsequent steps.
[0067] Finally, the matched two-dimensional image and the target three-dimensional point cloud data can be stored. Specifically, the matched two-dimensional image can be stored in the original size and png format; and the target three-dimensional point cloud data can be stored in the preset data structure in the xyz format.
[0068] Step 202: Input the two-dimensional image and the corresponding target three-dimensional point cloud data into the trained image segmentation model to obtain multiple contour points of the target line bundle.
[0069] In an embodiment of the present application, the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network (for example, a Yolov8 segmentation model).
[0070] Specifically, first, the corresponding matched two-dimensional image and the corresponding target three-dimensional point cloud data can be read from the memory to prepare for subsequent feature extraction. Then, the previously stored trained image segmentation model can be read from the memory, where the trained image segmentation model is stored in the best.pt file. Next, the two-dimensional image and the corresponding target three-dimensional point cloud data can be input into the trained image segmentation model to identify the target line bundle and its contour, thereby obtaining multiple contour points of the target line bundle, such as Figure 4 As shown, a detection schematic diagram of the image segmentation model provided in an embodiment of the present application, wherein, after the two-dimensional image and the corresponding target three-dimensional point cloud data are input into the trained image segmentation model, the trained image segmentation model can identify the target line bundle and its contour in the two-dimensional image through the recognition box.
[0071] Step 203: extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness.
[0072] Specifically, first, for any two adjacent contour points, the direction of the contour point vector corresponding to any two adjacent contour points can be determined (expressed as an angle value) to determine the direction and orientation of the harness contour, such as Figure 5 , which is a schematic diagram of the direction angle value of the contour point vector provided in an embodiment of the present application.
[0073] Then, the angle change of any two adjacent contour point vectors is determined according to the directions of any two adjacent contour point vectors, thereby providing a data basis for the angle change screening feature value for subsequent analysis, such as Figure 6 The figure is a schematic diagram of the direction angle change of the contour point vector provided in an embodiment of the present application.
[0074] Next, the K-means clustering method can be used to perform cluster analysis on multiple contour points to divide the multiple contour points into groups of contour points of the two side harness contours and other points, that is, multiple first contour point groups can be determined, for example, they can be divided into 3 first contour point groups; wherein the first contour point group includes contour points and other points of the two side harness contours of the target harness.
[0075] Then, outlier removal may be performed on the plurality of first contour point groups to obtain a plurality of second contour point groups, thereby ensuring that the obtained contour points can accurately represent contour features on both sides of the target harness.
[0076] Finally, based on the cluster analysis, multiple second contour points can be grouped and feature points can be extracted according to the angle change of any two adjacent contour point vectors, so as to further extract the feature points of the contour points of the two side harnesses. Thus, two feature points corresponding to the contours of the two side harnesses of the target harness are obtained, that is, the inflection points from the contours of the two sides of the harness to the edge of the diameter end. A total of 4 feature points are extracted, such as Figure 7 As shown, a schematic diagram of extracting the characteristic points of the installed wiring harness provided by the embodiment of the present application is provided, wherein the characteristic points are endpoints or inflection points. Thus, by determining the accuracy of the characteristic points, an important data basis is provided for the subsequent calculation of the bending radius of the target wiring harness, so that the bending radius of the target wiring harness can be accurately calculated later.
[0077] Step 204: Calculate the bending radius corresponding to the contours of the wire bundles on both sides according to the two characteristic points corresponding to the contours of the wire bundles on both sides.
[0078] Specifically, first, according to the two feature points corresponding to the two side harness contours respectively, the key points of the two side harness contours can be extracted in sequence to determine the first section harness contour and the second section harness contour corresponding to the target harness, so as to subsequently calculate the bending radius of the target harness; wherein the first section harness contour and the second section harness contour are respectively located on both sides of the target harness.
[0079] Then, a curve fitting or curvature calculation method may be used to calculate the bending radius of the first segment of the wire harness profile. For example, the bending radius of the first segment of the wire harness profile may be calculated by a "fitting plane circle" method. The specific calculation process is as follows:
[0080] Assume that the contour point set of the first segment of the line harness contour is {pt1, pt2,…, pt n}, the i-th contour point pt i It can be expressed as (x i ,y i ,z i ), based on this, by minimizing the objective function f as shown in formula (1), the best fitting plane parameters (a, b, c, d) of the harness contour points can be obtained, and the normal vector of the plane is n = (a, b, c).
[0081]
[0082] Then, according to the best fitting plane of the line bundle contour points, the projection point set {p1, p2, ..., p n}. Among them, the i-th projection point p i It can be obtained by the following formula (2).
[0083]
[0084] Next, the plane circle parameters can be calculated according to the projection points on the plane. Specifically, the plane circle parameters can be calculated by minimizing the objective function f as shown in the following formula (3):
[0085]
[0086] Among them, params is the optimized parameter, which includes the coordinates of the center c and the radius r of the circle, n·p i Represents point p i The projection on the normal vector n, d represents the offset of the plane, that is, d = -n·p i .
[0087] Furthermore, in an embodiment of the present application, the radius and center position of the circle can be fitted by the objective function in the above formula (3), and then the objective function can be minimized by adjusting the center coordinates and radius to obtain the best fitting circle radius (i.e., the bending radius of the first segment of the wiring harness profile).
[0088] Similarly, a curve fitting or curvature calculation method may also be used to calculate the bending radius of the second wire harness profile. For example, a “plane circle fitting” method may also be used to calculate the bending radius of the second wire harness profile.
[0089] Step 205: Calculate the bending radius of the target wire harness according to the bending radius corresponding to the wire harness contours on both sides.
[0090] Specifically, after calculating the bending radii corresponding to the wire harness profiles on both sides, that is, after calculating the bending radii of the first wire harness profile and the second wire harness profile, the bending radii of the first wire harness profile and the second wire harness profile can be directly averaged to calculate the bending radius of the target wire harness.
[0091] Finally, the calculated bending radius of the target wire harness may be output as a result, wherein the output result may include the recorded bending radius value and related information.
[0092] Furthermore, since the present application overcomes the limitations of the traditional "gauge comparison" method by combining binocular vision and structured light technology, it can achieve more accurate and efficient measurement of the bending radius of the wire harness.
[0093] In a possible implementation, in order to improve the segmentation accuracy of the image segmentation model, in an embodiment of the present application, before using a binocular structured light camera to capture the image of the target beam and obtain a two-dimensional image and corresponding target three-dimensional point cloud data, the image segmentation model can also be trained.
[0094] Specifically, first, in order to ensure the normal operation of equipment such as a binocular structured light camera (or industrial camera), parameters of the binocular structured light camera can be set. For example, parameters such as exposure time, resolution, and frame rate can be set to ensure accurate shooting of the wiring harness area.
[0095] Then, in order to improve the segmentation accuracy of the image segmentation model, a binocular structured light camera can be used to capture images of the wire harness to be inspected from different angles and distances to obtain multiple captured images. For example, a total of 104 images can be collected, and these images can include various parts of the wire harness. During the acquisition process, it is necessary to ensure that the wire harness in the area to be inspected is placed in a clearly visible position and there are no objects that block or obstruct the field of view. In addition, the image quality can be checked to rule out problems such as image blur, overexposure or underexposure, and any abnormal images can be recorded for subsequent processing.
[0096] Next, multiple acquired images can be batch-imported into segmentation and annotation software (e.g., segmentation and annotation tool Labelme software) to annotate the contours of the wire bundles in the multiple acquired images and ensure that the boundaries and details of each wire bundle are accurately marked, thereby obtaining multiple annotated acquired images.
[0097] Then, the multiple annotated captured images can be saved, that is, the annotation results can be saved and stored as a data set in the form of a file for subsequent analysis and model training, wherein the annotated captured images are saved in a json file format.
[0098] Next, you can prepare the labeled image dataset for training, that is, prepare these multiple labeled collected images, and perform necessary data cleaning and preprocessing on these multiple labeled collected images to improve the effect of model training. Then, you can determine the specific algorithm, network structure and training parameters of the training model to make full preparations for the training of the image segmentation model.
[0099] Finally, the initial image segmentation model is trained using multiple labeled collected images after data cleaning and preprocessing to obtain a trained image segmentation model, wherein the trained image segmentation model is stored in the form of a best.pt file.
[0100] To summarize, a “binocular structured light camera” is used to capture images of the target wire harness to obtain a two-dimensional image and the corresponding target three-dimensional point cloud data, and a trained image segmentation model is used to identify and extract the wire harness contour points to calculate the bending radius of the target wire harness. That is, this application overcomes the limitations of the traditional “gauge comparison” method by combining binocular vision and structured light technology, and achieves more accurate and efficient measurement of the bending radius of the wire harness.
[0101] Moreover, in view of the problems of time-consuming operation, low efficiency, low precision and easy error in the traditional "gauge comparison" method on aircraft, this application can not only realize the automation and precise measurement of the installed wire harness, but also realize the real-time monitoring of the wire harness manufacturing process, which is helpful to find problems in production and make timely adjustments, improve production efficiency and wire harness assembly quality. Compared with the traditional "gauge comparison" method, the accuracy is from centimeter level to 0.5mm, the detection time is shortened to within 10s (verification result), and the accuracy and efficiency are higher.
[0102] In addition, the present application proposes to use a model to identify the contour points of the wire harness, and train the model through deep learning and other technologies, so that the model can intelligently identify and extract the contour of the wire harness, and combine the two-dimensional image and three-dimensional point cloud data to comprehensively extract and analyze the contour features of the wire harness. The multi-dimensional data fusion method of the present application can more accurately capture the morphology and detail information of the wire harness. The technical solution of the present application has fast detection and high accuracy and reliability. It does not require actual contact with the wire harness, avoiding possible damage or deformation. It is not only suitable for measuring the bending radius of the wire harness, but can also be applied to other fields that require morphological measurement and feature extraction, such as parts processing, assembly, quality control, etc., providing new possibilities and prospects for the development of related fields.
[0103] Based on the same inventive concept, the embodiment of the present application provides a wire harness bending radius measuring device 80, such as Figure 8 As shown, the wire harness bending radius measuring device 80 includes:
[0104] The point cloud data acquisition unit 801 is used to collect images of the target wire harness using a binocular structured light camera to obtain a two-dimensional image and corresponding three-dimensional point cloud data of the target;
[0105] The contour point obtaining unit 802 is used to input the two-dimensional image and the corresponding target three-dimensional point cloud data into the trained image segmentation model to obtain multiple contour points of the target line harness; wherein the trained image segmentation model is generated based on the instance segmentation model of the convolutional neural network;
[0106] A feature point obtaining unit 803 is used to extract feature points from a plurality of contour points to obtain two feature points corresponding to the contours of the two sides of the target harness respectively;
[0107] A first bending radius calculation unit 804 is used to calculate the bending radius corresponding to the contours of the two side harnesses according to the two feature points corresponding to the contours of the two side harnesses;
[0108] The second bending radius calculation unit 805 is used to calculate the bending radius of the target wire harness according to the bending radii corresponding to the wire harness contours on both sides.
[0109] Optionally, the wire harness bending radius measuring device 80 further includes a model training unit 806, which is used to:
[0110] A binocular structured light camera is used to collect images of the wiring harness to be inspected from different angles and distances to obtain multiple collected images;
[0111] Batch import multiple acquired images into the segmentation and annotation software, annotate the wire harness contours in the multiple acquired images, and obtain multiple annotated acquired images;
[0112] The initial image segmentation model is trained using multiple annotated collected images to obtain a trained image segmentation model.
[0113] Optionally, the point cloud data obtaining unit 801 is further used for:
[0114] A binocular structured light camera is used to capture images of the target harness to obtain a two-dimensional image and the corresponding initial three-dimensional point cloud data;
[0115] Filter the initial three-dimensional point cloud data to obtain target three-dimensional point cloud data;
[0116] Match and align the two-dimensional image with the target three-dimensional point cloud data to obtain a matched two-dimensional image;
[0117] The matched two-dimensional image and target three-dimensional point cloud data are stored.
[0118] Optionally, the point cloud data obtaining unit 801 is further used for:
[0119] Store the matched two-dimensional image in original size and png format;
[0120] The target 3D point cloud data is stored in a preset data structure in xyz format.
[0121] Optionally, the feature point obtaining unit 803 is further used for:
[0122] For any two adjacent contour points, determine the direction of the contour point vector corresponding to any two adjacent contour points;
[0123] For the directions of any two adjacent contour point vectors, determine the angular change of any two adjacent contour point vectors;
[0124] A K-means clustering method is used to perform cluster analysis on multiple contour points to determine multiple first contour point groups; wherein the first contour point groups include contour points of the contours of the harnesses on both sides of the target harness and other points;
[0125] Perform outlier removal on the plurality of first contour point groups to obtain a plurality of second contour point groups;
[0126] According to the angle variation of any two adjacent contour point vectors, multiple second contour points are grouped to extract feature points, and two feature points corresponding to the two side harness contours of the target harness are obtained; wherein the feature points are endpoints or inflection points.
[0127] Optionally, the first bending radius calculation unit 804 is further configured to:
[0128] According to the two feature points corresponding to the contours of the harness on both sides, the key points of the contours of the harness on both sides are extracted in sequence to determine the first section of the contour of the harness and the second section of the contour of the harness corresponding to the target harness; wherein the first section of the contour of the harness and the second section of the contour of the harness are located on both sides of the target harness respectively;
[0129] Calculate the bending radius of the first section of the harness profile by using a curve fitting or curvature calculation method;
[0130] The bending radius of the second section of the harness profile is calculated using a curve fitting or curvature calculation method.
[0131] Optionally, the second bending radius calculation unit 805 is further used to:
[0132] The bending radius of the first section of the wire harness profile and the bending radius of the second section of the wire harness profile are averaged to calculate the bending radius of the target wire harness.
[0133] The wire harness bending radius measuring device 80 can be used to perform Figure 2-Figure 7 The method implemented in the embodiment shown in FIG. 8 is thus the function that can be realized by each functional module of the wire harness bending radius measuring device 80. Figure 2-Figure 7 The description of the illustrated embodiment is not repeated in detail.
[0134] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program part, which includes a program code. When the program part is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the following steps: Figure 2-Figure 7 The method performed in the illustrated embodiment.
[0135] It can be understood by those skilled in the art that all or part of the steps of the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiment are executed; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk. Alternatively, if the above integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent part, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention can be essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software part, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. And the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disk or optical disk.
[0136] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0137] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for measuring the bending radius of a wire harness, characterized in that: The method comprises: A binocular structured light camera is used to capture the image of the target harness to obtain a two-dimensional image and the corresponding three-dimensional point cloud data of the target; Inputting the two-dimensional image and the corresponding target three-dimensional point cloud data into a trained image segmentation model to obtain a plurality of contour points of the target harness; wherein the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network; Extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness; Calculating the bending radius corresponding to the contours of the wire harness on both sides according to the two characteristic points corresponding to the contours of the wire harness on both sides; The bending radius of the target wire harness is calculated according to the bending radii corresponding to the wire harness profiles on both sides.
2. The method according to claim 1, characterized in that Before using a binocular structured light camera to capture an image of the target wire harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data, the method further includes: The binocular structured light camera is used to collect images of the wire harness to be inspected from different angles and distances to obtain multiple collected images; Batch importing the multiple acquired images into the segmentation and annotation software, annotating the wire harness contours in the multiple acquired images, and obtaining multiple annotated acquired images; The multiple annotated collected images are used to train the initial image segmentation model to obtain a trained image segmentation model.
3. The method according to claim 1, characterized in that The step of using a binocular structured light camera to collect images of the target harness to obtain a two-dimensional image and corresponding target three-dimensional point cloud data includes: Using a binocular structured light camera to collect images of the target wire harness to obtain the two-dimensional image and corresponding initial three-dimensional point cloud data; Filtering the initial three-dimensional point cloud data to obtain target three-dimensional point cloud data; Matching and aligning the two-dimensional image with the target three-dimensional point cloud data to obtain a matched two-dimensional image; The matched two-dimensional image and the target three-dimensional point cloud data are stored.
4. The method according to claim 3, characterized in that The step of storing the matched two-dimensional image and the target three-dimensional point cloud data comprises: The matched two-dimensional image is stored in the original size and png format; The target three-dimensional point cloud data is stored in a preset data structure in an xyz format.
5. The method according to claim 1, characterized in that The step of extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness, respectively, comprises: For any two adjacent contour points, determine the direction of the contour point vector corresponding to the any two adjacent contour points; For the directions of any two adjacent contour point vectors, determining the angular change amount of any two adjacent contour point vectors; Performing cluster analysis on the plurality of contour points by using a K-means clustering method to determine a plurality of first contour point groups; wherein the first contour point groups include contour points and other points of the contours of the two side wire harnesses of the target wire harness; performing outlier removal on the plurality of first contour point groups to obtain a plurality of second contour point groups; According to the angle variation of any two adjacent contour point vectors, the plurality of second contour points are grouped to perform feature point extraction to obtain two feature points corresponding to the two side harness contours of the target harness respectively; wherein the feature points are endpoints or inflection points.
6. The method according to claim 1, characterized in that The step of calculating the bending radius corresponding to the contours of the wire harness on both sides respectively according to the two feature points corresponding to the contours of the wire harness on both sides respectively comprises: According to the two feature points corresponding to the two side harness contours, the key points of the two side harness contours are extracted in sequence to determine the first section harness contour and the second section harness contour corresponding to the target harness; wherein the first section harness contour and the second section harness contour are located on the two sides of the target harness, respectively; Calculating the bending radius of the first wire harness profile by using a curve fitting or curvature calculation method; The bending radius of the second wire harness profile is calculated by using a curve fitting or curvature calculation method.
7. The method according to claim 6, characterized in that The step of calculating the bending radius of the target wire harness according to the bending radius corresponding to the wire harness profiles on both sides includes: The bending radius of the target wire harness is calculated by averaging the bending radius of the first wire harness profile and the second wire harness profile.
8. A wire harness bending radius measuring device, characterized in that: The device comprises: A point cloud data acquisition unit is used to acquire an image of the target harness using a binocular structured light camera to obtain a two-dimensional image and corresponding three-dimensional point cloud data of the target; A contour point acquisition unit, used for inputting the two-dimensional image and the corresponding target three-dimensional point cloud data into a trained image segmentation model to obtain a plurality of contour points of the target line harness; wherein the trained image segmentation model is generated based on an instance segmentation model of a convolutional neural network; A feature point obtaining unit, used for extracting feature points from the plurality of contour points to obtain two feature points corresponding to the contours of the two side wire harnesses of the target wire harness; A first bending radius calculation unit, used for calculating the bending radius corresponding to the contours of the wire harness on both sides according to two feature points corresponding to the contours of the wire harness on both sides; The second bending radius calculation unit is used to calculate the bending radius of the target wire harness according to the bending radii corresponding to the wire harness profiles on both sides.
9. An electronic device, characterized in that: The device comprises: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute any method according to claims 1-7 according to the obtained program instructions.
10. A storage medium, characterized in that: The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute any one of the methods of claims 1-7.
Citation Information
Patent Citations
Method and equipment for measuring bending radius of cable
CN112595265A
Cable bending radius detection method and device
CN113935958A
Wire harness form characterization method, device, equipment and medium
CN114565699A
Method and equipment for determining bending degree of crimping pipe
CN117629103A
Optical fiber bending light-receiving head
JP2006235362A
Cited By
Methods for testing the straightness of structural steel
TWI915234B
