Wiring harness assembling method of vehicle roof and electronic equipment

Abstract

The embodiment of the invention relates to the technical field of robot control, and discloses a wire harness assembling method for a vehicle ceiling and electronic equipment, the wire harness assembling method for the vehicle ceiling is applied to a mechanical arm, and the method comprises the steps of obtaining a vehicle image of a current vehicle; determining a current vehicle type corresponding to the current vehicle according to the vehicle image; based on the current vehicle type, a preset parameter mapping library is determined and inquired, control parameters corresponding to the current vehicle type are determined, and the control parameters comprise an assembly track and force control parameters; and according to the assembly track and the force control parameters, a mechanical arm is controlled to conduct wire harness assembly on the car roof of the current car model. The vehicle model of the current vehicle is determined, the control parameters corresponding to the current vehicle model are determined by using the preset parameter mapping library, and the mechanical arm is controlled to perform wire harness assembly on the vehicle roof of the current vehicle model through the assembly track and the force control parameters, so that the assembly cost can be reduced.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a wiring harness assembly method and electronic device for a car roof. Background Technology

[0002] Flexible wiring harnesses, as components formed by the orderly or intertwined arrangement of multiple deformable linear objects such as wires, cables, and ropes, have been widely used in key fields such as industrial manufacturing, medical surgery, smart homes, and robot control due to their flexibility, bendability, and high integration. In industrial settings, flexible wiring harnesses are the core carriers for signal and power transmission within automotive electronics, and their assembly quality directly determines the performance stability of the vehicle.

[0003] Currently, the assembly of wiring harnesses for car roofs is usually done using robotic arms. However, the current method of assembling wiring harnesses for car roofs relies on fixed tooling for positioning, which makes it difficult to adapt to flexible production of multiple car models. If different car models are required, additional fixed tooling is needed, resulting in higher assembly costs. Summary of the Invention

[0004] This application provides a wiring harness assembly method and electronic device for a vehicle roof to reduce assembly costs.

[0005] The embodiments of this application provide the following technical solutions: In a first aspect, embodiments of this application provide a wiring harness assembly method for a vehicle roof, applied to a robotic arm, the method comprising: Obtain the vehicle image of the current vehicle; Based on the vehicle image, determine the current vehicle model corresponding to the current vehicle; Based on the current vehicle model, the preset parameter mapping library is queried to determine the control parameters corresponding to the current vehicle model. The control parameters include assembly trajectory and force control parameters. Based on the assembly trajectory and force control parameters, the robotic arm is controlled to assemble the wiring harness on the roof of the current vehicle model.

[0006] In some embodiments, The method also includes: Obtain the point cloud data of the current vehicle; Based on the current point cloud data of the vehicle, determine the real-time pose of the roof and wiring harness.

[0007] In some embodiments, Obtain the point cloud data of the current vehicle, including: Obtain a reference feature point cloud, which includes the point cloud of the reference region of the current vehicle body; Obtain the point cloud of the roof parts, which includes the point cloud of the roof parts area of ​​the vehicle; Based on the ICP algorithm, the point cloud of the reference feature and the point cloud of the roof parts are stitched together to obtain the complete point cloud data of the current vehicle.

[0008] In some embodiments, Based on the current vehicle point cloud data, determine the real-time pose of the roof and wiring harness, including: Key points were extracted using the SIFT3D algorithm, including key points on the vehicle body and key points on the roof. Match the key points of the vehicle body and the key points of the roof; The RANSAC algorithm combined with the PnP algorithm is used to calculate the real-time pose of the roof and wiring harness.

[0009] In some embodiments, Based on the assembly trajectory and force control parameters, the robotic arm is controlled to assemble the wiring harness on the roof of the current vehicle model, including: Based on the real-time pose of the roof and wiring harness, the robotic arm is controlled to move along the assembly trajectory to each path point of the assembly trajectory. At each path point of the assembly trajectory, the robotic arm is controlled to assemble the wiring harness based on force control parameters to assemble the wiring harness for the roof of the current vehicle model.

[0010] In some embodiments, The robotic arm includes a first robotic arm and a second robotic arm; The method also includes: Based on the real-time pose of the roof and wiring harness, the first working area corresponding to the first robotic arm and the second working area corresponding to the second robotic arm are divided. Based on the vehicle's point cloud data, obstacles in the first working area and obstacles in the second working area are identified; Based on obstacles in the first working area, path planning is performed on the first robotic arm to determine the first working trajectory; and based on obstacles in the second working area, path planning is performed on the second robotic arm to determine the second working trajectory.

[0011] In some embodiments, Based on obstacles in the first working area, path planning is performed on the first robotic arm to determine the first working trajectory, including: The Black Widow algorithm, optimized based on IBWOA, plans multiple first path points; Interpolate multiple first path points to determine the first working trajectory; Based on obstacles in the second working area, path planning is performed on the second robotic arm to determine the second working trajectory, including: The Black Widow algorithm, optimized based on IBWOA, plans multiple second path points; Interpolate multiple second path points to determine the second working trajectory.

[0012] In some embodiments, Force control parameters include the amount of pressure applied; The robotic arm includes an end effector and a pressure sensor, which is used to determine the pressure applied by the end effector. The method also includes: During the pressing process of the wire harness, the real-time pressing force feedback from the pressure sensor is obtained; Based on the PID control algorithm, the pressing force of the end effector is adjusted in real time according to the preset force and the real-time pressing force, and the position of the harness is adjusted in real time according to the preset harness position until the harness reaches the preset harness position.

[0013] In some embodiments, The method also includes: Assembly quality inspection of the wiring harness includes: Obtain an image of the gap between the wiring harness and the roof; Based on the gap image, determine the size of the gap between the wiring harness and the roof; If the gap size is less than the preset gap threshold, the wire harness assembly is deemed qualified. If the gap size is greater than or equal to the preset gap threshold, the wire harness assembly is determined to be unqualified, and the robotic arm is controlled to fine-tune the position of the wire harness until the gap size is less than the preset gap threshold.

[0014] Secondly, embodiments of this application provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When at least one program is executed by at least one processor, the at least one processor implements the above-described method for assembling the wiring harness for the roof.

[0015] Secondly, embodiments of this application provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When at least one program is executed by at least one processor, the at least one processor implements the above-described method for assembling the wiring harness for the roof.

[0016] Thirdly, embodiments of this application provide a non-volatile computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the aforementioned wiring harness assembly method for the vehicle roof.

[0017] Fourthly, embodiments of this application provide a computer program product comprising one or more lines of program code stored in a non-volatile computer-readable storage medium. A processor of an electronic device reads the program code from the non-volatile computer-readable storage medium and executes the program code to complete the steps of the above-described method for assembling the wiring harness of the roof liner.

[0018] The beneficial effects of this application embodiment are as follows: Unlike the prior art, this application embodiment provides a wiring harness assembly method for a car roof, applied to a robotic arm. The method includes: acquiring a vehicle image of the current vehicle; determining the current vehicle model based on the vehicle image; determining control parameters corresponding to the current vehicle model by querying a preset parameter mapping library based on the current vehicle model, wherein the control parameters include assembly trajectory and force control parameters; and controlling the robotic arm to assemble the wiring harness of the car roof of the current vehicle model according to the assembly trajectory and force control parameters.

[0019] By determining the current vehicle model, the corresponding control parameters are determined using a preset parameter mapping library. The assembly trajectory and force control parameters are used to control the robotic arm to assemble the wiring harness of the current vehicle's roof, thereby reducing assembly costs. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a wiring harness assembly method for a car roof provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the process of determining the real-time pose of a vehicle roof and wiring harness, as provided in an embodiment of this application. Figure 4 yes Figure 3 A detailed flowchart of step S301 in the process; Figure 5 yes Figure 3 A detailed flowchart of step S302 in the process; Figure 6 yes Figure 2 A detailed flowchart of step S204 in the process; Figure 7 This is a flowchart illustrating a method for determining the working trajectory of a robotic arm, as provided in an embodiment of this application. Figure 8yes Figure 7 A detailed flowchart of step S703 in the process; Figure 9 This is a schematic flowchart illustrating an embodiment of the present application for adjusting the position of a wire harness; Figure 10 This is a schematic diagram of a process for inspecting the assembly quality of a wire harness according to an embodiment of this application; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0023] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0024] Currently, the assembly of wiring harnesses for car roofs is usually done using robotic arms. However, the current method of assembling wiring harnesses for car roofs relies on fixed tooling for positioning, which makes it difficult to adapt to flexible production of multiple car models. If different car models are required, additional fixed tooling is needed, resulting in higher assembly costs.

[0025] Based on this, this application provides a wiring harness assembly method for a vehicle roof to reduce assembly costs.

[0026] Please see Figure 1 , Figure 1 This is a schematic diagram of an application scenario provided in an embodiment of this application.

[0027] like Figure 1 As shown, this application scenario includes a robotic arm that manipulates a flexible wiring harness to assemble the flexible wiring harness onto the roof of a vehicle.

[0028] In this embodiment of the application, the robotic arm is a dual-arm robotic arm, which is a component of a humanoid robot. For example, a humanoid robot includes a first robotic arm and a second robotic arm, which work together to assemble flexible wire harnesses.

[0029] The robotic arm includes a flexible gripper, on the inside of which a miniature high-precision pressure sensor is fitted to collect the pressure applied in real time.

[0030] In this embodiment, the humanoid robot includes a vision system consisting of three 3D structured light cameras and two depth cameras. The 3D structured light cameras are responsible for high-precision feature acquisition, while the depth cameras are responsible for scene completion and rapid localization. Electronic devices acquire data from the 3D structured light cameras and depth cameras, and perform real-time calculations based on this data to control the operation of the robotic arm.

[0031] Specifically, two structured light cameras are positioned on the left and right sides of the assembly station, at a 30-degree angle to the sides of the vehicle body, focusing on the sides of the vehicle body; one structured light camera is located at the top of the assembly station, covering the entire top area of ​​the vehicle body. Two depth cameras are installed on both sides of the entrance to the assembly station for auxiliary purposes.

[0032] Understandably, a depth camera can acquire three-dimensional depth information of a scene or object, that is, the depth of each pixel, and output an RGB image and a depth map.

[0033] In this embodiment, the depth camera includes a TOF camera and a binocular stereo camera.

[0034] Understandably, a 3D structured light camera is a type of depth camera. It actively projects coded light patterns onto the surface of an object, calculates the three-dimensional coordinates of each pixel by combining the pattern distortion, and finally outputs the object's 3D point cloud data or depth map.

[0035] Please refer to the following: Figure 2 , Figure 2 This is a schematic flowchart of a wiring harness assembly method for a car roof provided in an embodiment of this application.

[0036] like Figure 2 As shown, the process of assembling the wiring harness for the car roof includes the following steps S201-S204: Step S201: Obtain the vehicle image of the current vehicle; Specifically, a vehicle image of the current vehicle is acquired using a depth camera, and this vehicle image includes an RGB image.

[0037] Step S202: Determine the current vehicle model based on the vehicle image; Specifically, by identifying features in vehicle images, the current vehicle model can be determined. For example, a training dataset can be constructed, which includes multiple vehicle images, each corresponding to a labeled vehicle model. Based on the training dataset, the YOLO model is trained to obtain the trained YOLO model. The vehicle images are then input into the trained YOLO model to perform vehicle model recognition and determine the current vehicle model.

[0038] Step S203: Based on the current vehicle model, determine the preset parameter mapping library and determine the control parameters corresponding to the current vehicle model. The control parameters include assembly trajectory and force control parameters.

[0039] It is understandable that the assembly trajectory and force control parameters of the wiring harness assembly are different for different vehicle models. Therefore, this application pre-constructs a parameter mapping library, which includes multiple vehicle models and control parameters corresponding to each vehicle model. The control parameters include the assembly trajectory and force control parameters.

[0040] By pre-building a parameter mapping library, when switching vehicle models, the vision system automatically identifies the current vehicle model, queries the parameter mapping library, and calls the corresponding control parameters for the current model. This allows the robotic arm to assemble the wiring harness of the current model's roof based on the corresponding control parameters. As a result, the assembly of roofs for multiple vehicle models can be achieved without refactoring the tooling, thus reducing assembly costs.

[0041] Step S204: Based on the assembly trajectory and force control parameters, control the robotic arm to assemble the wiring harness on the roof of the current vehicle model.

[0042] Understandably, before assembling the wiring harness, it's necessary to determine the location of the vehicle's roof and the wiring harness. Specifically, this requires obtaining the real-time pose of the roof and the wiring harness.

[0043] Please refer to the following: Figure 3 , Figure 3 This is a flowchart illustrating the process of determining the real-time pose of a vehicle roof and wiring harness, as provided in an embodiment of this application.

[0044] like Figure 3 As shown, the process for determining the real-time pose of the roof and wiring harness includes the following steps S301-S302: Step S301: Obtain the point cloud data of the current vehicle.

[0045] Please refer to the following: Figure 4 , Figure 4 yes Figure 3 A detailed flowchart of step S301 is shown.

[0046] like Figure 4As shown, step S301: Obtaining the point cloud data of the current vehicle includes the following steps S311-S313: Step S311: Obtain the reference feature point cloud, wherein the reference feature point cloud includes the point cloud of the reference region of the current vehicle body.

[0047] Specifically, point cloud data of the reference area of ​​the current vehicle body is acquired by a depth camera. The reference area of ​​the vehicle body includes the sheet metal area of ​​the body assembly station (including all mounting holes, locating pins, and frame contours). For example, the depth camera is fixed above the assembly station or at the end of a robotic arm. By controlling the depth camera, a set of point clouds is acquired from the sheet metal area of ​​the body assembly station to obtain the reference feature point cloud, that is, the original point cloud of the body sheet metal. Its format is .pcd. The reference feature point cloud contains the (X,Y,Z) coordinates and normal vector of each pixel.

[0048] Furthermore, by controlling the depth camera to collect 3 to 5 sets of point clouds around the sheet metal area of ​​the vehicle assembly station, different perspectives can be covered to avoid occlusion.

[0049] Step S312: Obtain the point cloud of the roof parts, wherein the point cloud of the roof parts includes the point cloud of the roof parts area of ​​the vehicle.

[0050] Specifically, point cloud data of the vehicle roof component area is acquired using a 3D structured light camera. This roof component area includes the mounting surface, positioning holes, assembly wiring harness, and other areas of the roof component. For example, by placing the roof on a tooling table and scanning the component area with a 3D structured light camera, the point cloud of the roof component is obtained. That is, the point cloud data of the roof component area is in .pcd format, which contains the (X,Y,Z) coordinates and normal vector of each pixel.

[0051] Understandably, in order to assemble the wire harness, it is necessary to focus on scanning the wire harness to be assembled, such as focusing on the endpoints, intersections, and fixed buckle positions of the wire harness, in order to obtain the wire harness point cloud, that is, the original point cloud of the wire harness, in .pcd format. This wire harness point cloud contains the (X,Y,Z) coordinates and normal vector of each point.

[0052] Step S313: Based on the ICP algorithm, stitch together the reference feature point cloud and the point cloud of the roof parts to obtain the complete point cloud data of the current vehicle.

[0053] Specifically, the ICP algorithm, or Iterative Closest Point algorithm, is the core algorithm for point cloud stitching and alignment. The ICP algorithm finds the optimal spatial transformation relationship (translation + rotation) between two point clouds through iterative calculation, making the two point clouds overlap as much as possible.

[0054] Point cloud stitching and alignment are performed using the ICP algorithm to obtain the complete point cloud data of the current vehicle, including the following steps (1)-(4): Step (1): Take the point cloud of the car roof part as the source point cloud and the reference feature point cloud as the target point cloud. Extract the FPFH feature of each pixel in the source point cloud and the target point cloud respectively. The fast point feature histogram (FPFH) is used to describe the curvature, concavity and convexity and neighborhood distribution features of the local point cloud.

[0055] Step (2): Perform feature matching and RANSAC coarse registration on the FPFH features of the source point cloud and the FPFH features of the target point cloud to obtain the coarse registration result.

[0056] Specifically, by comparing the FPFH features of the source point cloud and the target point cloud, we find point pairs with similar features, use the RANSAC algorithm to remove incorrect matching pairs (such as points that are accidentally similar in shape but do not actually correspond), calculate a rough spatial transformation matrix (how much to translate and rotate), and move the source point cloud (the point cloud of the car roof part) according to this spatial transformation matrix so that the source point cloud and the target point cloud (the reference feature point cloud) are roughly aligned.

[0057] Step (3): Based on the coarse registration results, perform ICP fine registration.

[0058] Specifically, the optimal transformation matrix is ​​obtained through iterative calculation using a point-to-surface ICP algorithm for precise registration. This forces matching of core features such as mounting holes / locating pins to achieve ICP precision registration. For example, core feature points such as mounting holes and locating pins in the source point cloud are forced to precisely align with corresponding feature surfaces in the target point cloud. The optimal transformation matrix is ​​obtained through an iterative process, which includes: calculating the distance from the point to the surface each time → adjusting the transformation matrix to reduce the distance → repeating until the distance is minimized, resulting in the optimal transformation matrix. This iterative process significantly improves alignment accuracy, meeting the point cloud requirements at the vehicle part assembly level.

[0059] Step (4): Merge the baseline feature point cloud and the roof part point cloud to obtain the complete point cloud data of the current vehicle.

[0060] Specifically, by merging the precisely transformed point cloud of the roof component with the baseline feature point cloud into the same three-dimensional coordinate system, the final result is the complete point cloud data of the current vehicle, which can fully describe the three-dimensional shape of the vehicle (including the roof).

[0061] Step S302: Determine the real-time pose of the roof and wiring harness based on the current point cloud data of the vehicle.

[0062] Please refer to the following: Figure 5 , Figure 5 yes Figure 3A detailed flowchart of step S302 in the process.

[0063] like Figure 5 As shown, step S302: Based on the current vehicle point cloud data, determine the real-time pose of the roof and wiring harness, including the following steps S321-S323: Step S321: Extract key points using the SIFT3D algorithm, where key points include key points of the vehicle body and key points of the roof.

[0064] Specifically, the complete point cloud data of the current vehicle is acquired, i.e., the stitched complete point cloud. Gaussian convolutions of different scales (σ is the scale factor) are performed on the stitched complete point cloud (body + roof + wiring harness) to generate a multi-scale point cloud pyramid to construct a 3D scale space. The reference feature regions (mounting holes, positioning pins) are segmented from the body point cloud and extracted separately using SIFT3D (to reduce interference from irrelevant regions). Feature filtering is performed on the extracted key points, retaining only the key points located on the mounting holes / positioning pins / frame contours to output the body key point set.

[0065] Furthermore, feature regions (positioning holes, mounting surfaces) matching the vehicle body are segmented from the point cloud of the roof; key points are extracted using SIFT3D parameters identical to those of the vehicle body (ensuring scale / rotation consistency); features matching the key point types of the vehicle body are retained (e.g., positioning hole key points only match mounting hole key points) to output a set of key points for the roof.

[0066] Furthermore, the curvature of the bundle point cloud is calculated, and extreme curvature points (where the bundle bends) are selected. Connectivity analysis is used to extract the bundle endpoints (points with only 1 neighbor) and intersections (points with ≥3 neighbor) to output a set of bundle key points.

[0067] Step S322: Match the key points of the vehicle body and the key points of the roof.

[0068] Specifically, a 3D feature descriptor is calculated for each keypoint, and an initial set of matching pairs is output through KNN nearest neighbor matching. Then, a set of valid matching pairs is selected based on constraint rules, including: (1) Distance constraints, for example: calculate the 3D Euclidean distance of the matching pairs and remove matching pairs with a distance > 5mm (the matching distance of the body / roof features should be ≤ 5mm).

[0069] (2) Normal vector constraint, for example: calculate the angle between the normal vectors of the key points of the matching pair and remove matching pairs with an angle > 10° (the normal vectors of rigid features should be consistent).

[0070] (3) Topological constraints, for example: verify the neighborhood topological relationship of the matching pair. If the matching relationship of the neighborhood key points is inconsistent, the matching pair is removed.

[0071] Furthermore, it is necessary to match the key points of the wiring harness. For example, relative matching based on the roof pose is required, including: initially calculating the roof pose based on the effective matching set, transforming the key points of the wiring harness from the roof coordinate system to the vehicle coordinate system, performing FPFH matching with the key points of the wiring harness, and establishing a correspondence.

[0072] Step S323: Use the RANSAC algorithm combined with the PnP algorithm to calculate the real-time pose of the roof and wiring harness.

[0073] Specifically, based on the 3D-3D corresponding point pairs of the effective matching pair set, outliers are eliminated using RANSAC, and then the pose of the roof / wiring harness (rotation matrix R + translation vector T) is solved using the PnP algorithm (Perspective-n-Point, PnP), outputting the real-time pose in the vehicle body coordinate system.

[0074] In the embodiments of this application, the PnP algorithm includes the 3D-3D PnP algorithm, namely the EPnP algorithm.

[0075] In this embodiment, SIFT3D ensures the scale / rotation invariance of key points, enabling adaptation to point clouds from different viewpoints. Furthermore, multi-layer matching and filtering ensures the accuracy of corresponding point pairs. RANSAC+EPnP balances robustness and accuracy, calculating the absolute pose of the roof, then deriving the relative pose of the wire harness, and finally outputting real-time pose data that meets industrial assembly requirements, thereby improving pose accuracy.

[0076] After obtaining the real-time pose of the roof and wiring harness, the roof and wiring harness can be assembled using the real-time pose of the roof and wiring harness.

[0077] For details, please refer to [link / reference]. Figure 6 , Figure 6 yes Figure 2 A detailed flowchart of step S204 is shown.

[0078] like Figure 6 As shown, step S204: Based on the assembly trajectory and force control parameters, control the robotic arm to assemble the wiring harness on the roof of the current vehicle model, including the following steps S241-S242: Step S241: Based on the real-time pose of the roof and wiring harness, control the robotic arm to move along the assembly trajectory to each path point of the assembly trajectory.

[0079] Specifically, based on the real-time pose of the roof and wiring harness, the predefined assembly trajectory is dynamically corrected, and the robotic arm is controlled to move precisely to each path point to ensure the positioning accuracy of the path points and to ensure a smooth movement process.

[0080] Step S242: At each path point of the assembly trajectory, control the robotic arm to assemble the wiring harness based on force control parameters to assemble the wiring harness for the roof of the current vehicle model.

[0081] Specifically, at each path point (especially critical assembly nodes, such as buckle pressing and plug insertion), the assembly force and displacement of the robotic arm are adaptively adjusted based on feedback from pressure sensors to ensure that the assembly actions are compliant (no overpressure, no false assembly).

[0082] Furthermore, the robotic arm includes a first robotic arm and a second robotic arm, which simultaneously assemble the wiring harness on the vehicle roof. Specifically, the working trajectory of the robotic arm needs to be determined; please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a flowchart illustrating how to determine the working trajectory of a robotic arm, as provided in an embodiment of this application.

[0083] like Figure 7 As shown, the process for determining the working trajectory of the robotic arm includes the following steps S701-S703: Step S701: Based on the real-time pose of the roof and wiring harness, divide the first working area corresponding to the first robotic arm and the second working area corresponding to the second robotic arm.

[0084] Specifically, based on the real-time pose of the roof and wiring harness, the entire assembly area is divided into two independent areas that are non-overlapping, fully covered, and adapted to the workspace of the dual robotic arms. These are the first working area and the second working area. For example, the first working area and the second working area are divided according to the symmetrical assembly of the roof. The first robotic arm is assigned to the left side of the roof, and the second robotic arm is assigned to the right side of the roof.

[0085] In this embodiment of the application, by dividing the work area into two areas, collisions between the first robotic arm and the second robotic arm during collaborative assembly can be avoided, while ensuring assembly efficiency.

[0086] Step S702: Based on the vehicle's point cloud data, determine the obstacles in the first working area and the obstacles in the second working area.

[0087] Specifically, static or dynamic obstacles are extracted from the complete point cloud data of the current vehicle to provide an obstacle avoidance map for path planning, so as to ensure that the first and second robotic arms do not collide with the vehicle body, tooling, wiring harness or another robotic arm during the movement.

[0088] Static obstacles include body panels, roof frames, tooling fixtures, assembled wiring harnesses, and foam cushioning pads, while dynamic obstacles include the movement area of ​​another robotic arm.

[0089] Step S703: Based on the obstacles in the first working area, perform path planning for the first robotic arm to determine the first working trajectory; and based on the obstacles in the second working area, perform path planning for the second robotic arm to determine the second working trajectory.

[0090] For details, please refer to [link / reference]. Figure 8 , Figure 8 yes Figure 7 A detailed flowchart of step S703 is shown.

[0091] like Figure 8 As shown, step S703 involves: performing path planning for the first robotic arm based on obstacles in the first working area to determine the first working trajectory; and performing path planning for the second robotic arm based on obstacles in the second working area to determine the second working trajectory, including the following steps S731-S734: Step S731: Based on the IBWOA-optimized Black Widow algorithm, plan multiple first path points; It should be noted that IBWOA is an improved version of the basic Black Widow Optimization Algorithm (BWOA), which is prone to getting trapped in local optima and has slow convergence. It improves global search, convergence speed and optimization accuracy by integrating multiple strategies and is often used for solving complex function optimization and engineering constraint problems.

[0092] Specifically, the Black Widow algorithm, optimized based on IBWOA, plans multiple first path points, including: Extract information from multiple obstacles, determine the first working area corresponding to the first robotic arm, and set the initial standby position of the first robotic arm and the starting position for wire harness assembly.

[0093] Next, configure the IBWOA algorithm parameters, specifically including: (1) Initialize the parameters of the improved black widow optimization algorithm, specifically including: population size N=30 (each individual in the population corresponds to a candidate path), maximum number of iterations T=50, mating probability Pc=0.7, mutation probability Pm=0.05, selection pressure factor α=2.0 (controlling the genetic weight of excellent individuals); the improvement strategy is to introduce an adaptive weight factor (which decreases linearly with the number of iterations, with an initial weight of 0.9 and a weight of 0.4 at the end of the iteration) to improve the convergence accuracy of the algorithm in the later stage.

[0094] (2) Encoding and initializing population individuals, specifically including: using real number encoding, each population individual is represented by the three-dimensional coordinates (x1, y1, z1, x2, y2, z2, ..., x) of n path points. n ,y n ,z n The initial population is composed of n, which is the number of initial path points (preset n=8, which can be dynamically adjusted according to the complexity of the work area). Based on the starting point, the ending point and the boundary of the work area, the initial population is generated by random sampling to ensure that the candidate path corresponding to each individual does not exceed the movement range of the robotic arm, and the distance between the initial path point and the obstacle is ≥ 1.2 times the safe distance.

[0095] (3) Algorithm iterative optimization and path point selection, specifically including: ① Constructing the fitness function: Construct a multi-objective fitness function F to comprehensively evaluate the safety, feasibility, and economy of the path. The formula is: F=ω1×D+ω2×S+ω3×θ, where: D is the total path length (the smaller the better); S is the minimum distance between the path point and the obstacle (the larger the better; if S < safe distance, then F is set to infinity and the individual is directly eliminated); θ is the sum of the attitude angle deviations of the end effector of the robotic arm at each path point (the smaller the better); weight coefficients ω1=0.4, ω2=0.5, ω3=0.1 (verified through multiple experiments, prioritizing obstacle avoidance safety).

[0096] ② Implement the iterative process, including: Selection operation: The roulette wheel selection method is used to select the top 1 / 3 of the population individuals with the best fitness as the parents based on the fitness function value.

[0097] Mating process: Parent individuals are randomly paired and gene fragments are exchanged at random locations to generate offspring individuals.

[0098] Mutation operation: Perform small-scale random mutations (mutation range ±0.5mm) on some genes (path point coordinates) of offspring individuals to improve population diversity and avoid getting trapped in local optima; Adaptive weight update: As the number of iterations increases, the weight of individuals with better fitness is gradually increased to accelerate the convergence speed.

[0099] ③ Termination condition and optimal path point extraction: When the number of iterations reaches the maximum number of iterations T, or the fluctuation of the optimal fitness function value in 10 consecutive iterations is ≤0.01, the iteration terminates; extract the set of discrete path points corresponding to the optimal individual, denoted as the first candidate path point group (containing n path points, sorted in the order of movement).

[0100] (4) Verification and correction of path points, specifically including: kinematic verification of the first candidate path point group, and solving the problem through forward and inverse kinematics of the robotic arm (using numerical iteration method, with a solution accuracy of 10).-3 (mm) Confirm that the joint angles of the robotic arm corresponding to each path point are within the allowable range; if there is a path point where the joint exceeds the limit, re-verify by fine-tuning the coordinates of the point (fine-tuning range ≤ 1mm) until all path points meet the kinematic constraints, and finally obtain multiple first path points.

[0101] Step S732: Interpolate multiple first path points to determine the first working trajectory, including: (1) Determine the interpolation algorithm, for example: select the third-order B-spline interpolation algorithm.

[0102] (2) Perform interpolation calculation based on interpolation algorithm to generate the first working trajectory, including: based on the basis function formula of B spline interpolation, substitute the three-dimensional coordinates of each first path point to calculate the interpolated continuous trajectory function (x(t), y(t), z(t)), where t is the interpolation parameter (t∈[0,1]); through discretization processing (take t step size 0.01), generate a high-density continuous path point set, which is the original data of the first working trajectory, thereby generating the first working trajectory, which includes multiple first path points.

[0103] Step S733: Based on the IBWOA-optimized Black Widow algorithm, plan multiple second path points; Step S734: Interpolate multiple second path points to determine the second working trajectory.

[0104] It is understandable that the process of determining the second working trajectory for the second robotic arm is the same as the process of determining the first working trajectory, and will not be elaborated here.

[0105] Please refer to the following: Figure 9 , Figure 9 This is a schematic diagram of a process for adjusting the position of a wire harness according to an embodiment of this application.

[0106] like Figure 9 As shown, the process for adjusting the position of the wire harness includes the following steps S901-S903: Step S901: During the pressing process of the wire harness, the real-time pressing force feedback from the pressure sensor is obtained; Specifically, a miniature high-precision pressure sensor is attached to the inside of the flexible gripper carried by the robotic arm. The pressure sensor collects the current pressing force in real time to obtain the real-time pressing force.

[0107] Step S902: Based on the PID control algorithm, adjust the pressing force of the end effector in real time according to the preset force and the real-time pressing force. Specifically, a preset force is determined. For example, if the end effector is a flexible gripper, the preset force is set to 0.18-0.20 MPa. The industrial control computer has a built-in PID control module with initial parameters set as follows: proportional coefficient Kp=5.0, integral coefficient Ki=0.5, and derivative coefficient Kd=0.8. The calculation logic is as follows: first, the deviation value e(t) between the preset force and the real-time pressing force is calculated. If e(t)>0 (real-time pressure is insufficient), the PID module outputs a positive adjustment to control the flexible gripper to increase the pressing force. If e(t)<0 (real-time pressure is too high), a negative adjustment is output to control the flexible gripper to decrease the pressing force. If |e(t)|≤0.01 MPa (pressure deviation is within the allowable range), the current pressing force is maintained unchanged.

[0108] Step S903: Adjust the position of the wire harness in real time according to the preset wire harness position until the wire harness reaches the preset wire harness position.

[0109] Specifically, after each position fine-tuning action is completed, the depth camera immediately acquires a gap image of the current area, calculates the gap size, and if the gap size is greater than or equal to the preset gap threshold, the position fine-tuning action is performed again until the adjusted gap size is less than the preset gap threshold. In addition, the position of the adjusted wire harness is controlled to reach the preset wire harness position.

[0110] Please refer to the following: Figure 10 , Figure 10 This is a schematic diagram of a process for inspecting the assembly quality of a wire harness, provided in an embodiment of this application.

[0111] like Figure 10 As shown, the assembly quality inspection process for the wire harness includes the following steps S1001-S1006: Step S1001: Obtain an image of the gap between the wiring harness and the roof; Specifically, a depth camera is used to acquire images of the gap between the wiring harness and the roof lining. These images include the gap between the installed wiring harness and the roof lining. For example, the depth camera is mounted at the end of a robotic arm. After receiving a signal indicating that the wiring harness assembly is complete, the signal triggers the depth camera to start image acquisition. During image acquisition, the depth camera on the robotic arm moves along a preset trajectory (parallel to the direction of the wiring harness extension), acquiring one frame of image every 5mm to obtain images of the gap between the wiring harness and the roof lining.

[0112] Step S1002: Determine the size of the gap between the wiring harness and the roof lining based on the gap image; Specifically, before determining the size of the gap between the wiring harness and the roof based on the gap image, the method also includes: The acquired raw gap images are preliminarily processed, including: removing image noise using a Gaussian filtering algorithm (filter kernel size 5×5); retaining only the effective image containing the area where the wire harness fits with the roof through image cropping, and removing irrelevant background; and performing coordinate calibration on the depth image to convert pixel coordinates into world coordinates to facilitate subsequent calculation of gap size.

[0113] Specifically, based on the gap image, the size of the gap between the wiring harness and the roof is determined, including the following steps (1)-(3): Step (1): Extract the edges of the preprocessed slit image.

[0114] Specifically, the Canny edge detection algorithm is used to extract edges from the preprocessed gap image. For example, the algorithm parameters of the Canny edge detection algorithm are set to a high threshold of 180 and a low threshold of 60 to extract the contour information of the wire harness edge and the roof edge to obtain edge contour information. For the depth image, the gradient thresholding method is used to extract the depth change region. This depth change region is the candidate region for the gap between the wire harness and the roof, so as to achieve the preliminary location of the gap.

[0115] Step (2): The edge contours extracted from the preprocessed gap image are fused and compared with the depth change regions extracted from the depth image. Regions that simultaneously satisfy edge continuity and depth change greater than the depth change threshold (0.05mm) are selected as valid gap regions. Also, pseudo gap regions caused by wire harness surface wrinkles, car roof surface defects, etc. are removed (pseudo gap judgment criteria: region width less than 0.03mm or length less than 2mm).

[0116] Step (3): Measure the size of the effective gap area based on the calibrated world coordinates.

[0117] Specifically, along the direction of the gap, a measurement point is selected at 2mm intervals, and the vertical distance between the edge of the wire harness and the edge of the roof at each measurement point (i.e., the gap width at that point) is calculated. The gap widths of all measurement points are statistically analyzed, and the average value is calculated as the final gap size of the current detection area.

[0118] Step S1003: Is the gap size greater than or equal to the preset gap threshold? Specifically, it is determined whether the gap size is greater than or equal to a preset gap threshold. For example, if the preset gap threshold is 1mm, if the gap size is greater than or equal to the preset gap threshold, then proceed to step S1004: determine that the wire harness assembly is unqualified; if the gap size is less than the preset gap threshold, then proceed to step S1006: determine that the wire harness assembly is qualified.

[0119] Step S1004: Determine that the wire harness assembly is defective; Step S1005: Control the robotic arm to fine-tune the position of the wire harness until the gap size is less than the preset gap threshold; Specifically, the industrial control computer generates a fine-tuning path and parameters for the robotic arm based on the location of gaps whose size is greater than or equal to a preset gap threshold. The fine-tuning direction is towards the direction of reducing the gap (i.e., the direction in which the wiring harness fits into the roof). The initial fine-tuning step size is determined based on the maximum gap width (0.3mm when the gap width is ≥1.5mm; 0.2mm when the gap width is 1.0mm≤1.5mm; and 0.1mm when the gap width is 0.5mm≤1.0mm). The fine-tuning range is limited to an area with a radius of 5mm centered on the current wiring harness position to avoid over-adjustment that could damage the wiring harness or cause misalignment of other assembly parts.

[0120] Furthermore, the industrial control computer sends fine-tuning instructions to the robotic arm, which include parameters such as fine-tuning path, step size, and speed (fine-tuning speed controlled within 5mm / s). The robotic arm carries a flexible gripper (grip pressure controlled at 0.1-0.2MPa to avoid damaging the wire harness sheath) to perform fine-tuning operations. During the fine-tuning process, the depth camera acquires gap images in real time and transmits them to the industrial control computer. After each fine-tuning step size is completed, the gap size calculation in step S1002 is immediately repeated until the adjusted gap size is less than the preset gap threshold.

[0121] Step S1006: Confirm that the wire harness assembly is qualified.

[0122] In this embodiment, closed-loop assembly quality detection can correct dynamic interference (such as assembly gap fluctuations) in real time. Compared with traditional open-loop assembly, the solution in this application can significantly improve anti-interference and stability. Simultaneously, the collaborative division of labor between the two robotic arms reduces the redundancy of individual robot movements, shortens assembly time, and balances high precision and real-time performance. Real-vehicle assembly experiments have verified that this application can robustly handle complex scenarios involving the assembly of roof panels for multiple vehicle models, and it exhibits strong generalization ability, enabling rapid adaptation to the assembly needs of new vehicle models.

[0123] In this embodiment of the application, a method for assembling a wiring harness for a car roof is provided. The method includes: acquiring a vehicle image of the current vehicle; determining the current vehicle model based on the vehicle image; determining control parameters corresponding to the current vehicle model by querying a preset parameter mapping library based on the current vehicle model, wherein the control parameters include an assembly trajectory and force control parameters; and controlling a robotic arm to assemble the wiring harness for the car roof of the current vehicle model according to the assembly trajectory and force control parameters.

[0124] By determining the current vehicle model, the corresponding control parameters are determined using a preset parameter mapping library. The assembly trajectory and force control parameters are used to control the robotic arm to assemble the wiring harness of the current vehicle's roof, thereby reducing assembly costs.

[0125] Please refer to the following: Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0126] Among them, electronic equipment includes industrial control computers.

[0127] like Figure 11 As shown, the electronic device 110 includes one or more processors 111 and a memory 112. Wherein, Figure 11 Take a processor 111 as an example.

[0128] Processor 111 and memory 112 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.

[0129] The processor 111 provides computing and control capabilities to control the electronic device 110 to perform corresponding tasks. For example, it controls the electronic device 110 to perform the wiring harness assembly method for the roof lining in any of the above method embodiments. The method includes: acquiring a vehicle image of the current vehicle; determining the current vehicle model based on the vehicle image; determining the control parameters corresponding to the current vehicle model by querying a preset parameter mapping library based on the current vehicle model, wherein the control parameters include assembly trajectory and force control parameters; and controlling a robotic arm to assemble the wiring harness for the roof lining of the current vehicle model based on the assembly trajectory and force control parameters.

[0130] By determining the current vehicle model, the corresponding control parameters are determined using a preset parameter mapping library. The assembly trajectory and force control parameters are used to control the robotic arm to assemble the wiring harness of the current vehicle's roof, thereby reducing assembly costs.

[0131] Processor 111 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0132] Memory 112, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the wiring harness assembly method for the car roof in the embodiments of this application. Processor 111 can implement the wiring harness assembly method for the car roof in any of the above method embodiments by running the non-transitory software programs, instructions, and modules stored in memory 112. Specifically, memory 112 may include volatile memory (VM), such as random access memory (RAM); memory 112 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 112 may also include combinations of the above types of memory.

[0133] Memory 112 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 112 may optionally include memory remotely located relative to processor 111, and these remote memories may be connected to processor 111 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0134] One or more modules are stored in memory 112. When executed by one or more processors 111, they perform the wiring harness assembly method for the roof lining in any of the above method embodiments, for example, the method described above. Figure 2 The steps shown.

[0135] This application also provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors. For example, the one or more processors can execute the wiring harness assembly method for the roof in any of the above method embodiments, such as performing the steps described above.

[0136] This application also provides a computer program product, which includes one or more lines of program code stored in a non-volatile computer-readable storage medium. The processor of the electronic device reads the program code from the non-volatile computer-readable storage medium and executes the program code to complete the steps of the above-described method for assembling the wiring harness of the car roof.

[0137] The apparatus or device embodiments described above are merely illustrative. The unit modules described as separate components may or may not be physically separate, and the components shown as module units may or may not be physical units; that is, they may be located in one place or distributed across multiple network module units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for a computer device (which may be a personal computer, server, or network device, etc.) to execute the various embodiments or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above. For the sake of brevity, they are not provided in detail; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for assembling a wiring harness for a vehicle roof, characterized in that, Applied to robotic arms, the method includes: Obtain the vehicle image of the current vehicle; Based on the vehicle image, determine the current vehicle model corresponding to the current vehicle; Based on the current vehicle model, a preset parameter mapping library is queried to determine the control parameters corresponding to the current vehicle model, wherein the control parameters include assembly trajectory and force control parameters; Based on the assembly trajectory and force control parameters, the robotic arm is controlled to assemble the wiring harness on the roof of the current vehicle model.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the point cloud data of the current vehicle; Based on the current point cloud data of the vehicle, the real-time pose of the roof and wiring harness is determined.

3. The method according to claim 2, characterized in that, The acquisition of the point cloud data of the current vehicle includes: Obtain a reference feature point cloud, wherein the reference feature point cloud includes the point cloud of a reference region of the body of the current vehicle; Obtain a point cloud of roof components, wherein the point cloud of roof components includes the point cloud of the roof component area of ​​the vehicle; Based on the ICP algorithm, the reference feature point cloud and the roof component point cloud are stitched together to obtain the complete point cloud data of the current vehicle.

4. The method according to claim 3, characterized in that, Determining the real-time pose of the roof and wiring harness based on the current vehicle's point cloud data includes: Key points are extracted using the SIFT3D algorithm, including key points of the vehicle body and key points of the roof. Match the key points of the vehicle body and the key points of the roof; The RANSAC algorithm combined with the PnP algorithm was used to calculate the real-time pose of the roof and wiring harness.

5. The method according to any one of claims 2-4, characterized in that, The step of controlling the robotic arm to assemble the wiring harness on the roof of the current vehicle model according to the assembly trajectory and force control parameters includes: Based on the real-time pose of the roof and wiring harness, the robotic arm is controlled to move according to the assembly trajectory to each path point of the assembly trajectory. At each path point of the assembly trajectory, the robotic arm is controlled to assemble the wiring harness based on the force control parameters, so as to assemble the wiring harness for the roof of the current vehicle model.

6. The method according to claim 1, characterized in that, The robotic arm includes a first robotic arm and a second robotic arm; The method further includes: Based on the real-time pose of the roof and wiring harness, the first working area corresponding to the first robotic arm and the second working area corresponding to the second robotic arm are divided. Based on the point cloud data of the vehicle, obstacles in the first working area and obstacles in the second working area are determined; Based on the obstacles in the first working area, path planning is performed on the first robotic arm to determine a first working trajectory; and based on the obstacles in the second working area, path planning is performed on the second robotic arm to determine a second working trajectory.

7. The method according to claim 6, characterized in that, The step of performing path planning for the first robotic arm based on obstacles in the first working area to determine the first working trajectory includes: The Black Widow algorithm, optimized based on IBWOA, plans multiple first path points; Interpolate multiple first path points to determine the first working trajectory; The step of performing path planning for the second robotic arm based on obstacles in the second working area to determine the second working trajectory includes: The Black Widow algorithm, optimized based on IBWOA, plans multiple second path points; Interpolate multiple second path points to determine the second working trajectory.

8. The method according to claim 1, characterized in that, The force control parameters include the pressure applied. The robotic arm includes an end effector and a pressure sensor, the pressure sensor being used to determine the pressing force of the end effector; The method further includes: During the pressing process of the wire harness, the real-time pressing force feedback from the pressure sensor is obtained; Based on the PID control algorithm, the pressing force of the end effector is adjusted in real time according to the preset force and the real-time pressing force, and the position of the wiring harness is adjusted in real time according to the preset wiring harness position until the wiring harness reaches the preset wiring harness position.

9. The method according to claim 1, characterized in that, The method further includes: The assembly quality inspection of the wire harness includes: Obtain an image of the gap between the wiring harness and the vehicle roof; Based on the gap image, determine the size of the gap between the wiring harness and the roof; If the gap size is less than a preset gap threshold, the wire harness assembly is deemed qualified. If the gap size is greater than or equal to a preset gap threshold, the wire harness assembly is determined to be unqualified, and the robotic arm is controlled to fine-tune the position of the wire harness until the gap size is less than the preset gap threshold.

10. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, such that at least one of the processors implements claim 1 9. Any one of the methods described.

Images