Robot movement trajectory generation method and system for engine detection

Abstract

The application discloses a robot movement track generation method and system for engine detection, the method comprising: generating a 3D digital contour model of the engine through original images of different perspectives of the engine; generating a motion waypoint list of the robot according to the 3D digital contour model; inputting the motion waypoint list into a robot controller to generate an initial robot movement track of the engine; traversing a current motion waypoint from the initial robot movement track, controlling an end camera of the robot to move to the current motion waypoint to take a picture to obtain a target image; inputting the target image into a pre-trained detection model to output a confidence of the target image; optimizing the initial robot movement track according to the confidence to obtain a final movement track; and storing a mapping relationship between an order number of the engine and the final movement track. Therefore, the application can improve the generation efficiency of the robot movement track, thereby guaranteeing the production efficiency and improving the reliability of the detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motor control, and in particular to a robot movement track generation method and system for engine detection. BACKGROUND

[0002] With the diversification and individualization of market demand, the product models on the engine production line are increasingly diversified. Even for the same engine model, the positions, quantities and sizes of the external parts will be significantly different due to the requirements of different customer order numbers. In order to ensure that there is no misassembly or missing assembly problem in the assembly process of the engine, comprehensive and accurate photographic detection of the engine appearance is required. The detection process is performed by a camera carried at the end of a robot according to a movement track to take pictures of each part of the engine.

[0003] In the related art, technicians need to manually plan and set hundreds of robot movement tracks according to the specific requirements of each engine model and its order number.

[0004] However, the process of manually setting the robot movement track is extremely time-consuming, and it takes more than 30 minutes to set each track on average. Moreover, as the number of engine models and order numbers continues to increase, the number of tracks required for setting also grows exponentially, which greatly limits the production efficiency. In addition, the manual setting of the track is difficult to achieve high consistency and repeatability, and the tracks set by different technicians may be different, thereby affecting the reliability of the detection results. SUMMARY

[0005] The embodiments of the present application provide a robot movement track generation method and system for engine detection. To have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This part is not a general review, nor is it intended to determine the key / important components or delineate the protection scope of these embodiments. Its only purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0006] In a first aspect, the embodiments of the present application provide a robot movement track generation method for engine detection, the method comprising:

[0007] generating a 3D digital contour model of the engine from original images of different perspectives of the engine;

[0008] generating a motion waypoint list of the robot according to the 3D digital contour model;

[0009] transmitting the motion waypoint list into a robot controller to generate an initial robot movement track of the engine, and traversing a current motion waypoint from the initial robot movement track to control the end camera of the robot to move to the current motion waypoint for photographing to obtain a target image;

[0010] inputting the target image into the pre-trained detection model to output a confidence of the target image; and optimizing an initial robot moving track according to the confidence to obtain a final moving track.

[0011] storing a mapping relationship between the order number of the engine and the final moving track.

[0012] Optionally, the 3D digital contour model of the engine is generated through original images of different perspectives of the engine, and the 3D digital contour model of the engine includes:

[0013] the original images of different perspectives are preprocessed to obtain high-quality images of different perspectives;

[0014] edge extraction is performed on the preprocessed original images to obtain edge binary images of different perspectives which reserve contour edges of the appearance of the engine;

[0015] the edge binary images of different perspectives are taken as masks to extract foreground parts of the engine from the high-quality images of different perspectives to obtain foreground images of different perspectives;

[0016] feature point matching between images of different perspectives is performed based on the foreground images of different perspectives to generate an initial three-dimensional feature point cloud of the appearance of the engine;

[0017] the 3D digital contour model of the engine is constructed according to the initial three-dimensional feature point cloud.

[0018] Optionally, the initial three-dimensional feature point cloud of the appearance of the engine is generated through feature point matching between images of different perspectives based on the foreground images of different perspectives, and the initial three-dimensional feature point cloud of the appearance of the engine includes:

[0019] key feature points in the foreground images of different perspectives which are used to represent geometric features of the engine are extracted, and feature descriptors of the key feature points are calculated to obtain a plurality of feature descriptors of the foreground images of different perspectives;

[0020] the foreground images of different perspectives are matched through the plurality of feature descriptors of the foreground images of different perspectives to determine a plurality of matching point pairs;

[0021] erroneous matching pairs in the plurality of matching point pairs are removed to obtain a plurality of cross-perspective feature point pairs;

[0022] three-dimensional space coordinates of the cross-perspective feature points in each cross-perspective feature point pair are calculated as the initial three-dimensional feature point cloud of the appearance of the engine according to intrinsic and extrinsic parameters of an end camera of the robot.

[0023] Optionally, the 3D digital contour model of the engine is constructed according to the initial three-dimensional feature point cloud, and the 3D digital contour model of the engine includes:

[0024] According to the initial three-dimensional feature point cloud, an initial 3D grid model of the engine is established;

[0025] The initial 3D grid model is smoothed by using a Laplace smoothing algorithm;

[0026] The local of the initial 3D grid model after smoothing is fine-tuned by using appearance detail features representing geometric features of the engine, and a 3D digital contour model of the engine is obtained.

[0027] Optionally, according to the 3D digital contour model, a motion waypoint list of the robot is generated, including:

[0028] Each surface of the 3D digital contour model is traversed;

[0029] The normal vector of each surface is calculated based on the direction of the normal vector pointing to the outside of the engine;

[0030] According to the normal vector direction of the normal vector of each surface, each vertex of each surface is translated by a preset distance along the normal vector direction to obtain the vertex coordinates of each surface after offset;

[0031] The vertex coordinates of each surface after offset are reconnected to generate a photographing reference surface grid model of the motion reference surface of the end camera of the robot;

[0032] According to the photographing reference surface grid model, a motion waypoint set of the robot photographing is generated;

[0033] According to the motion waypoint set, a motion waypoint list of the robot is generated.

[0034] Optionally, according to the photographing reference surface grid model, a motion waypoint set of the robot photographing is generated, including:

[0035] The photographing reference surface grid model is subjected to collision detection and line-of-sight analysis to adjust the local of the photographing reference surface grid model to obtain an optimized target photographing reference surface grid model;

[0036] The engine is divided into multiple layers along the Z-axis of the target photographing reference surface grid model by a preset number of layers, each layer corresponding to a horizontal slice of the photographing reference surface, and the waypoints in each layer are uniformly distributed along the horizontal direction;

[0037] The outer surface of the engine in each layer is divided into multiple logical regions according to the structure to obtain multiple logical regions in each layer, and the multiple logical regions include a front end cover region, a rear end cover region, a shell side surface region, and a pipeline integration region;

[0038] In each logical region of each layer, motion waypoints at different positions are planned according to a preset grid density, and a preset number of transition motion waypoints are inserted at the junctions of different logical regions in each layer to obtain a motion waypoint set of the robot photographing.

[0039] Optionally, according to the set of motion points, a motion point list of the robot is generated, comprising:

[0040] A target space coordinate system is established with the geometric center of the 3D digital contour model as the origin;

[0041] The coordinates of each motion point in the set of motion points are converted into three-dimensional coordinates in the target space coordinate system to obtain the three-dimensional coordinates of each motion point;

[0042] The camera pose angle, point number, and photograph trigger signal of each motion point are determined;

[0043] The three-dimensional coordinates of each motion point and the camera pose angle, point number, and photograph trigger signal are packaged into a robot motion point list of the robot.

[0044] Optionally, according to the confidence, the initial robot movement trajectory is optimized to obtain a final movement trajectory, comprising:

[0045] In the case where the confidence is greater than or equal to a preset threshold, the photograph trigger signal of the current motion point is set to an open state, and the step of traversing the current motion point from the initial robot movement trajectory is continued until all the motion points in the initial robot movement trajectory are traversed, and a photograph trigger signal setting result of the motion point is obtained; or,

[0046] In the case where the confidence is less than the preset threshold, the step of controlling the end camera of the robot to move to the current motion point for photographing is continued;

[0047] In the case where the number of photographs is greater than a preset number and the confidence is less than the preset threshold, the current motion point is marked as an invalid point, the photograph trigger signal of the current motion point is set to a closed state, and the step of traversing the current motion point from the initial robot movement trajectory is continued until all the motion points in the initial robot movement trajectory are traversed, and a photograph trigger signal setting result of the motion point is obtained;

[0048] According to the photograph trigger signal setting result of the motion point, the initial robot movement trajectory is updated to obtain a final movement trajectory.

[0049] Optionally, according to the photograph trigger signal setting result of the motion point, the initial robot movement trajectory is updated to obtain a final movement trajectory, comprising:

[0050] The motion points with the photograph trigger signal in the closed state in the initial robot movement trajectory are removed;

[0051] The motion points with the photograph trigger signal in the open state before and after the removed motion points are connected to obtain a final movement trajectory.

[0052] In a second aspect, the embodiments of the present application provide a robot movement track generation system for engine detection, the system comprising:

[0053] a 3D digital contour model generation module configured to generate a 3D digital contour model of the engine based on original images of the engine from different perspectives;

[0054] a motion waypoint list generation module configured to generate a motion waypoint list of the robot based on the 3D digital contour model;

[0055] a photographing module configured to input the motion waypoint list into a robot controller to generate an initial robot movement track of the engine, and to control an end camera of the robot to move to a current motion waypoint to take a target image based on the initial robot movement track and the current motion waypoint;

[0056] a track optimization module configured to input the target image into a pre-trained detection model to output a confidence of the target image, and to optimize the initial robot movement track based on the confidence to obtain a final movement track;

[0057] a parameter storage module configured to store a mapping relationship between an order number of the engine and the final movement track.

[0058] The technical solutions provided by the embodiments of the present application can have the following beneficial effects:

[0059] In the embodiments of the present application, on the one hand, the 3D digital contour model is generated based on the original images of the engine from different perspectives, the motion waypoint list of the robot is generated, and then the initial robot movement track is generated. This process does not require manual setting of the track one by one, greatly reducing the time cost of track setting. At the same time, as the engine models and order numbers increase, this process can quickly adapt and generate the corresponding track, avoiding a significant increase in time and labor costs due to the increase in the number of manual settings, thereby significantly improving the production efficiency. On the other hand, the robot end camera moves strictly according to the generated track, ensuring the accuracy of the photographing position and angle. In addition, by evaluating the confidence of the target image and optimizing the robot movement track accordingly, the accuracy of the detection result is further improved, ensuring that the verified track can be used for subsequent detection of engines with the same order number, thereby improving the reliability of the detection result.

[0060] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0061] The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate embodiments consistent with the application and, together with the description, further serve to explain the principles of the application.

[0062] Figure 1 is a flow diagram of a robot movement trajectory generation method for engine detection provided by an embodiment of the present application;

[0063] Figure 2 is a process diagram of a model generation process of a 3D digital contour model provided by the present application;

[0064] Figure 3 is a process diagram of a robot movement waypoint list generation process of a robot provided by the present application;

[0065] Figure 4 is a process diagram of a trajectory optimization process provided by the present application;

[0066] Figure 5 is a structural diagram of a robot movement trajectory generation system for engine detection provided by the present application;

[0067] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following description and drawings are illustrative of the specific embodiments of the present application and are not intended to be limiting thereof.

[0069] It should be noted that the described embodiments are merely a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0070] The following description refers to the accompanying drawings, which are incorporated in and constitute a part of this specification. Unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementations described in the following example embodiments are illustrative of all implementations consistent with the present application. They are not meant to limit the scope of the application as described in the claims.

[0071] In the description of the present application, it should be understood that the terms "first", "second" and the like are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance. The above terms can be understood in their specific meanings in the present application by the person of ordinary skill in the art according to the specific circumstances. In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise specified. The association relationship of the associated objects is described, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0072] The present application provides a robot movement trajectory generation method and system for engine detection to solve the problems existing in the above related technical problems. In the embodiments of the present application, on the one hand, a 3D digital contour model of the engine is generated by using original images of different perspectives of the engine, a movement waypoint list of the robot is generated, and then an initial robot movement trajectory is generated. This process does not require manual setting of the trajectory one by one, greatly reducing the time cost of trajectory setting. At the same time, with the increase of engine models and order numbers, this process can quickly adapt and generate the corresponding trajectory, avoiding the substantial increase in time and labor costs caused by the increase in the number of manual settings, thereby significantly improving the production efficiency. On the other hand, the robot end camera moves strictly according to the generated trajectory, ensuring the accuracy of the shooting position and angle. In addition, by evaluating the confidence of the target image and optimizing the robot movement trajectory accordingly, the accuracy of the detection result is further improved, ensuring that the verified trajectory can be used for subsequent engine detection of the same order number, thereby improving the reliability of the detection result. The following exemplary embodiments are described in detail.

[0073] The following will be described in detail with reference to the accompanying drawings Figure 1 -Appendix Figure 4 The robot movement trajectory generation method for engine detection provided by the embodiments of the present application is described in detail. The method can be realized by relying on a computer program and can run on a robot movement trajectory generation system for engine detection based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool class application.

[0074] Please refer to Figure 1 A flowchart of a robot movement trajectory generation method for engine detection is provided for the embodiments of the present application. As Figure 1 shown, the method of the embodiments of the present application can include the following steps:

[0075] S101, a 3D digital contour model of the engine is generated by using original images of different perspectives of the engine;

[0076] The original images of different perspectives of the engine refer to two-dimensional images taken from multiple different angles (such as front, back, left, right, up, and down) of the engine. These images are directly collected from industrial cameras and contain all the details of the appearance of the engine. The 3D digital contour model is a three-dimensional virtual model used to accurately represent the external contour and geometry of the engine.

[0077] In some embodiments of the present application, the collection process of the original images of different perspectives of the engine includes: using an industrial-grade high-definition 2D camera (resolution not less than 4K). Appearance images are sequentially taken from the front, back, left, right, up, and down of the engine, ensuring that the images cover all the details of the external surface of the engine, and the light intensity of the shooting environment is uniform to avoid glare and shadow interference. The captured images are stored as original image data, obtaining the original images of different perspectives of the engine.

[0078] In some embodiments of the present application, the specific process of generating the 3D digital contour model of the engine through the original images of different perspectives of the engine includes: pre-processing the original images of different perspectives to obtain high-quality images of different perspectives; performing edge extraction on the pre-processed original images to obtain edge binary images of different perspectives that retain the contour edges of the appearance of the engine; using the edge binary images of different perspectives as masks to extract the foreground part of the engine from the high-quality images of different perspectives to obtain foreground images of different perspectives; performing matching of feature points between different perspective images based on the foreground images of different perspectives to generate an initial three-dimensional feature point cloud of the appearance of the engine; and constructing the 3D digital contour model of the engine according to the initial three-dimensional feature point cloud.

[0079] Specifically, when pre-processing the original images of different perspectives, an adaptive median filtering algorithm is used to suppress noise in the original images of different perspectives, and the size of the filtering window is dynamically adjusted according to the noise density of the image (range 3x3-7x7), which retains the edge details of the appearance of the engine while eliminating Gaussian noise and salt and pepper noise; the Retinex algorithm is used to optimize the brightness uniformity of the image, improve the contrast of the surface texture and edges of the engine, and solve the problem of image detail blur caused by uneven lighting.

[0080] Specifically, when performing edge extraction on the pre-processed original images, the Canny operator is used for edge extraction, and double thresholds (low threshold 50-80, high threshold 150-200) are set to accurately identify the contour edges of the appearance of the engine and generate edge binary images.

[0081] In some embodiments of the present application, the specific process of generating the initial three-dimensional feature point cloud of the engine appearance based on the foreground images of different perspectives includes: extracting key feature points in the foreground images of different perspectives for representing the geometric features of the engine, calculating the feature descriptors of the key feature points, and obtaining a plurality of feature descriptors of the foreground images of different perspectives; performing matching of the feature descriptors on the foreground images of different perspectives based on the plurality of feature descriptors of the foreground images of different perspectives, to determine a plurality of matching point pairs; eliminating the false matching pairs in the plurality of matching point pairs, to obtain a plurality of cross-perspective feature point pairs; and calculating the three-dimensional space coordinates of the cross-perspective feature points in each cross-perspective feature point pair based on the intrinsic and extrinsic parameters of the end camera of the robot, as the initial three-dimensional feature point cloud of the engine appearance.

[0082] Specifically, when the key feature points in the foreground images of different perspectives for representing the geometric features of the engine are extracted, the SIFT (Scale-Invariant Feature Transform) algorithm is used to extract the key feature points in the 2D images of different perspectives, including the engine shell joints, bolt holes, pipeline interfaces, and protruding structures, etc. Each feature point records its pixel coordinates, scale parameter and direction vector, to ensure the scale invariance and rotation invariance of the feature points.

[0083] Specifically, when the matching of the feature descriptors is performed on the foreground images of different perspectives, the FLANN (Fast Library for Approximate Nearest Neighbors) algorithm is first used to realize the matching of the feature points between different perspective images, then the Euclidean distance of the feature point descriptors is calculated to screen the matching pairs, and then the RANSAC (Random Sample Consensus) algorithm is used to eliminate the false matching pairs, to retain the correct feature point correspondence and establish the spatial correlation between the images of different perspectives. Finally, based on the camera intrinsic parameters (focal length and principal point coordinates) and extrinsic parameters (shooting poses of different perspectives), the three-dimensional space coordinates of the cross-perspective feature points in each cross-perspective feature point pair are calculated based on the principle of triangulation, as the initial three-dimensional feature point cloud of the engine appearance.

[0084] In some embodiments of the present application, the specific process of constructing the 3D digital contour model of the engine based on the initial three-dimensional feature point cloud includes: establishing an initial 3D mesh model of the engine based on the initial three-dimensional feature point cloud; performing smoothing processing on the initial 3D mesh model by using the Laplace smoothing algorithm; fine-tuning the local part of the initial 3D mesh model after the smoothing processing by using the appearance detail features for representing the geometric features of the engine, to obtain the 3D digital contour model of the engine.

[0085] Specifically, the initial 3D mesh model contains the design size, structural parameters and appearance features of the engine as the constraint reference for 3D contour generation; at the same time, camera calibration parameters, image acquisition environment parameters and the like are integrated to form a basic database for digital modeling. The Laplace smoothing algorithm is used to smooth the initial 3D mesh model, eliminate the noise and sawtooth on the surface of the mesh, and improve the smoothness of the contour; at the same time, based on the appearance detail features (such as the bending radius of the pipeline, the fillet of the shell and the like) in the model, the local details of the 3D mesh model are fine-tuned to ensure that the contour details are consistent with the actual engine appearance.

[0086] For example Figure 2 As shown, Figure 2 is a process diagram of a model generation process of a 3D digital contour model provided by the present application, and the original images of the engine are taken from multiple perspectives. The original images are subjected to noise suppression and brightness optimization to obtain high-quality images. The contour edges of the engine are extracted from the preprocessed images to generate edge binary images. The edge binary images are used as masks to extract the foreground part of the engine from the high-quality images. Feature point matching is performed between the foreground images at different perspectives to generate matching point pairs. The three-dimensional space coordinates are calculated according to the matching point pairs to generate an initial three-dimensional feature point cloud. Based on the initial three-dimensional feature point cloud, the 3D digital contour model of the engine is generated and optimized.

[0087] S102, generating a motion waypoint list of the robot according to the 3D digital contour model;

[0088] The motion waypoint list is a list containing all key positions (waypoints) that the robot needs to pass through when performing a task. Each waypoint contains three-dimensional coordinates (X, Y, Z), a camera pose angle (to ensure that the camera lens is perpendicular to the engine surface) and a photograph trigger signal. It can be used to guide the end effector (such as a camera) of the robot to move along the predetermined path and ensure comprehensive photographing detection of the outer surface of the engine.

[0089] In some embodiments of the present application, the specific process of generating the motion waypoint list of the robot according to the 3D digital contour model includes: traversing each surface of the 3D digital contour model; calculating the normal vector of each surface based on the direction of the normal vector pointing to the outside of the engine; translating each vertex of each surface along the normal vector direction by a preset distance according to the normal vector direction of the normal vector of each surface to obtain the vertex coordinates of each surface after offset; reconnecting the vertex coordinates of each surface after offset to generate a photograph reference surface mesh model of the motion reference surface of the end camera of the robot; generating a motion waypoint set of the robot photographing according to the photograph reference surface mesh model; and generating the motion waypoint list of the robot according to the motion waypoint set.

[0090] Specifically, the normal vector of each face of the 3D digital contour model is calculated (ensuring that the normal vector direction points to the outside of the engine); then, according to the normal vector direction, the vertices of each face are translated along the normal vector direction by 20 cm to obtain the offset vertex coordinates; finally, the offset vertices are reconnected to generate a photographing reference surface mesh model parallel to the 3D contour of the engine and spaced 20 cm apart, which is the photographing reference surface mesh model of the motion reference surface of the end camera of the robot.

[0091] In some embodiments of the present application, the specific process of generating a set of motion waypoints for robot photographing according to the photographing reference surface mesh model includes: performing collision detection and line-of-sight analysis on the photographing reference surface mesh model to adjust the local part of the photographing reference surface mesh model to obtain an optimized target photographing reference surface mesh model; dividing the engine into multiple layers along the Z-axis of the target photographing reference surface mesh model by a preset number of layers, each layer corresponding to a horizontal slice of the photographing reference surface, and the waypoints in each layer being uniformly distributed along the horizontal direction; dividing the outer surface of each layer into multiple logical regions according to the structure to obtain multiple logical regions of each layer, the multiple logical regions including a front end cover region, a rear end cover region, a shell side region, and a pipeline integration region; and planning motion waypoints at different positions in each logical region of each layer according to a preset grid density, and inserting a preset number of transition motion waypoints at the boundaries of different logical regions of each layer to obtain the set of motion waypoints for robot photographing.

[0092] Specifically, when performing collision detection and line-of-sight analysis on the photographing reference surface mesh model, the photographing reference surface is adjusted locally for special structures on the outer surface (such as protruding pipelines, recessed interfaces, and arc-shaped shells). For example, for a cylindrical pipeline with a diameter of 5 cm, the corresponding photographing reference surface needs to be offset along the tangent direction of the outer circle of the pipeline to ensure that the camera and the pipeline surface always maintain a distance of 20 cm; for a recessed interface with a depth of 3 cm, the reference surface needs to extend into the recessed interior to avoid a photographing blind area for the camera.

[0093] Specifically, when dividing the engine into multiple layers, the motion waypoints are planned according to the area and detail density of the outer surface of the engine to ensure that the images captured by adjacent motion waypoints have an overlap area of 10%-15% (to avoid missing shots), and the waypoint density increases with the complexity of the appearance details. For example, 1 waypoint is planned every 20cm x 20cm in the flat shell region of the engine, and 1 waypoint is planned every 10cm x 10cm in the detail-intensive regions such as bolt holes and pipeline interfaces. Secondly, the engine is divided into 3-5 layers along the vertical direction (Z-axis) (according to the height of the engine), each layer corresponding to a horizontal slice of the photographing reference surface, and the waypoints in each layer being uniformly distributed along the horizontal direction (X-axis, Y-axis) to ensure that the entire outer surface of the layer is covered.

[0094] Specifically, the outer surface of the engine is divided into four regions according to the structure, i.e., a front end cover region, a rear end cover region, a shell side region, and a pipeline integration region. Motion points are independently planned for each region, and transition points are added at the junctions of the regions (to avoid trajectory breaks) to obtain a set of motion points for the robot to take pictures. For example, 2-3 transition points are additionally planned at the junction of the front end cover region and the shell side region to ensure smooth movement of the camera.

[0095] In some embodiments of the present application, the specific process of generating the motion point list of the robot according to the set of motion points includes: establishing a target space coordinate system with the geometric center of the 3D digital contour model as the origin; converting the coordinates of each motion point in the set of motion points to three-dimensional coordinates in the target space coordinate system to obtain the three-dimensional coordinates of each motion point; determining the camera attitude angle, point number, and picture taking trigger signal of each motion point; and packaging the three-dimensional coordinates and the camera attitude angle, point number, and picture taking trigger signal of each motion point into the robot motion point list of the robot.

[0096] Specifically, a coordinate system is established with the geometric center of the 3D digital contour model as the origin (the X axis is along the length direction of the engine, the Y axis is along the width direction, and the Z axis is along the height direction). The three-dimensional coordinates (X, Y, Z) and the camera attitude angle (to ensure that the camera lens is perpendicular to the outer surface of the engine, with an attitude angle error of ≤±2°) of each point are calculated through the vertex coordinates of the picture taking reference surface grid model and the point distribution rule to form a robot point list containing the point number, three-dimensional coordinates, attitude angle, and picture taking trigger signal. The point list corresponds to the engine order number one by one.

[0097] For example Figure 3 As shown, Figure 3 is a process schematic diagram of a process of generating a robot motion point list of a robot provided by the present application. First, each surface of the 3D model is processed one by one. The normal vector of each surface is calculated to ensure that the direction points to the outside of the engine. Each vertex is translated by a preset distance along the normal vector direction to obtain the coordinates of the offset vertex. The offset vertex is reconnected to form a picture taking reference surface grid model. Collision detection and line of sight analysis are performed. The local structure is adjusted to obtain an optimized picture taking reference surface grid model. The Z axis is layered, and each layer corresponds to a horizontal slice of the picture taking reference surface. The logical regions (front end cover region, rear end cover region, etc.) are divided according to the structure. Points are planned in each logical region of each layer, and transition points are inserted at the junctions. A target space coordinate system is established. The point coordinates are converted to the target coordinate system. The camera attitude angle, point number, and picture taking trigger signal of each point are determined. The motion point list is packaged.

[0098] S103, transmit the motion point list to the robot controller to generate an initial robot movement trajectory of the engine, and traverse the current motion point from the initial robot movement trajectory to control the end camera of the robot to move to the current motion point for shooting to obtain a target image;

[0099] Wherein, the robot controller is a hardware and software system for receiving instructions and controlling the movement of the robot. It can parse the motion point list and convert it into specific actions that the robot can perform. The initial robot movement trajectory is a robot movement path generated according to the motion point list. It is a preliminary motion planning to ensure that the robot can move according to the predetermined sequence of points. The current motion point is the point that the robot needs to reach currently during movement. The end camera is a camera installed at the end of the robot for shooting appearance images of the engine. The target image is the appearance image of the engine shot at each point.

[0100] In some embodiments of the present application, the generated motion point list (containing point number, three-dimensional coordinates, attitude angle, and shooting trigger signal) is transmitted to the robot controller through a communication interface (such as Ethernet, serial communication). The robot controller plans the movement path of the robot according to the sequence of points in the point list. The initial robot movement trajectory is generated to ensure that the robot can reach each point in the predetermined order. The robot controller starts the robot and traverses each point in the initial trajectory. When reaching each point, the controller checks the three-dimensional coordinates and attitude angle of the current point to ensure that the end camera of the robot is in the correct position and attitude. The controller triggers the shooting signal to make the end camera shoot at the current point to obtain the target image.

[0101] Specifically, an association database containing order number-robot movement trajectory is established. Wherein, the order number is used as a unique index to associate the corresponding trajectory of the engine, ensuring that different order numbers can quickly match exclusive trajectory data. For new order number engines, after completing 3D contour generation and trajectory point planning for the first time, the complete set of trajectory parameters is automatically bound to the order number and entered into the database; if the structure of the existing order number engine changes (such as pipeline layout adjustment), the trajectory parameters are corrected synchronously through digital twin model update triggering to ensure that the trajectory data in the database is consistent with the actual product. When the end camera of the robot reaches the target point and triggers the first shooting, the target image is obtained.

[0102] S104, input the target image into the pre-trained detection model to output the confidence of the target image; according to the confidence, optimize the initial robot movement trajectory to obtain a final movement trajectory;

[0103] In some embodiments of the present application, the target image is input into a pre-trained detection model (a trained yoloV10 algorithm model), and a confidence level is output. It should be noted that the yoloV10 algorithm model can be used as an existing model that has been trained. There are many existing schemes for training the yoloV10 algorithm model, and the model training process will not be described here.

[0104] In some embodiments of the present application, according to the confidence level, the initial robot movement trajectory is optimized to obtain a specific process of the final movement trajectory, which includes: in the case that the confidence level is greater than or equal to a preset threshold, setting a photograph trigger signal of a current motion waypoint to an open state, and continuing to execute the step of traversing the current motion waypoint from the initial robot movement trajectory until all motion waypoints in the initial robot movement trajectory are traversed, and a photograph trigger signal setting result of the motion waypoint is obtained.

[0105] In some embodiments of the present application, in the case that the confidence level is less than the preset threshold, the step of controlling the end camera of the robot to move to the current motion waypoint for photographing is continued; in the case that the number of photographing times is greater than a preset number and the confidence levels are all less than the preset threshold, the current motion waypoint is marked as an invalid waypoint, the photograph trigger signal of the current motion waypoint is set to a closed state, and the step of traversing the current motion waypoint from the initial robot movement trajectory is continued until all motion waypoints in the initial robot movement trajectory are traversed, and a photograph trigger signal setting result of the motion waypoint is obtained; and according to the photograph trigger signal setting result of the motion waypoint, the initial robot movement trajectory is updated to obtain a final movement trajectory.

[0106] Specifically, if the confidence level is greater than or equal to a set threshold, the photograph trigger signal of the motion waypoint is retained, and valid photographing is completed; if the confidence level is less than the threshold, secondary photographing is triggered (with an interval of 0.3 seconds to ensure image acquisition stability), and the detection process is repeated until 5 times of photographing and detection are completed. At this time, invalid motion waypoint determination and signal cancellation are performed: if the target to be detected is not recognized in the continuous 5 times of detection, the system determines that there is no target to be detected at the position corresponding to the waypoint, and automatically cancels the photograph trigger signal of the waypoint.

[0107] In some embodiments of the present application, according to the photograph trigger signal setting result of the motion waypoint, the initial robot movement trajectory is updated to obtain a specific process of the final movement trajectory, which includes: removing the motion waypoints in the initial robot movement trajectory whose photograph trigger signals are in a closed state; connecting the motion waypoints before and after the removed motion waypoints whose photograph trigger signals are in an open state to obtain a final movement trajectory.

[0108] Specifically, in the current engine inspection task, the system cancels invalid waypoints in real time based on the photo-triggered signal cancellation result, generating a dynamic execution path of "valid waypoints - optimized trajectory". For example, if the original trajectory contains 120 waypoints, after 5 detections and the cancellation of 15 invalid waypoints, the system automatically adjusts the trajectory interpolation curve, directly connecting the valid waypoints before and after the invalid waypoints, forming an optimized trajectory containing only 105 valid waypoints, reducing the robot's idle time.

[0109] For example Figure 4 As shown, Figure 4 This is a schematic flowchart illustrating a trajectory optimization process provided in this application. First, each waypoint is checked individually. It is determined whether the confidence level of the current waypoint is greater than or equal to a threshold. If yes, the process continues to the next waypoint and takes more photos. If not, the process continues to take photos of the current waypoint. The number of photos taken of the current waypoint is checked against a preset number. If yes, it is marked as an invalid waypoint, and the photo-taking signal is turned off. It is then determined whether the traversal is complete, meaning all waypoints have been processed. If so, invalid waypoints are removed from the initial trajectory, and valid waypoints are connected to obtain the optimized robot movement trajectory.

[0110] S105 stores the mapping relationship between the engine's order number and its final movement trajectory.

[0111] The engine order number is a unique identifier for the engine model and configuration. Order numbers are typically generated by customers or manufacturers based on specific requirements, reflecting the engine's specifications, configuration, and functions. The mapping relationship establishes an association between the engine order number and the corresponding final motion trajectory for rapid retrieval and application. Through this mapping relationship, the system can quickly retrieve the corresponding robot motion trajectory based on the order number, avoiding duplicate trajectory generation and improving production efficiency.

[0112] In some embodiments of this application, a mapping table is created to associate each order number with its corresponding final movement trajectory. The mapping table can be a database table, a collection of key-value pairs, or any other form of data structure used to store and retrieve mapping relationships. Storing the mapping relationships in a database ensures data persistence and accessibility. An index is created for each order number and its corresponding final movement trajectory to facilitate fast retrieval.

[0113] In the embodiments of the present application, on the one hand, a 3D digital contour model is generated by using original images of different perspectives of the engine, a motion point list of the robot is generated, and then an initial robot movement trajectory is generated. This process does not require manual setting of the trajectory one by one, greatly reducing the time cost of trajectory setting. At the same time, as the number of engine models and order numbers increases, this process can quickly adapt and generate the corresponding trajectory, avoiding a substantial increase in time and labor costs due to the increase in the number of manual settings, thereby significantly improving production efficiency. On the other hand, the robot end camera moves strictly according to the generated trajectory, ensuring the accuracy of the shooting position and angle. In addition, by evaluating the confidence of the target image and optimizing the robot movement trajectory accordingly, the accuracy of the detection result is further improved, ensuring that the verified trajectory can be used when detecting the same order number engine in the future, thereby improving the reliability of the detection result.

[0114] The following is an embodiment of the system of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the system embodiments of the present application, please refer to the method embodiments of the present application.

[0115] Please refer to Figure 5 which shows a structure diagram of a robot movement trajectory generation system for engine detection provided by an exemplary embodiment of the present application. The robot movement trajectory generation system for engine detection can be realized by software, hardware or a combination of both to become all or part of an electronic device. The system 1 includes a 3D digital contour model generation module 10, a motion point list generation module 20, a photographing module 30, a trajectory optimization module 40, and a parameter storage module 50.

[0116] The 3D digital contour model generation module 10 is configured to generate a 3D digital contour model of the engine by using original images of different perspectives of the engine.

[0117] The motion point list generation module 20 is configured to generate a motion point list of the robot according to the 3D digital contour model.

[0118] The photographing module 30 is configured to input the motion point list into a robot controller to generate an initial robot movement trajectory of the engine, and traverse a current motion point from the initial robot movement trajectory to control the end camera of the robot to move to the current motion point for photographing to obtain a target image.

[0119] The trajectory optimization module 40 is configured to input the target image into a pre-trained detection model to output a confidence of the target image, and optimize the initial robot movement trajectory according to the confidence to obtain a final movement trajectory.

[0120] The parameter storage module 50 is configured to store a mapping relationship between the order number of the engine and the final movement trajectory.

[0121] It should be noted that the robot movement track generation system for engine detection provided in the above embodiment is only exemplified by the above division of functional modules when performing the robot movement track generation method for engine detection. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the electronic device is divided into different functional modules to complete all or part of the functions described above. In addition, the robot movement track generation system for engine detection and the robot movement track generation method for engine detection provided in the above embodiment belong to the same concept, which embodies the realization process details of the method embodiment, which will not be repeated here.

[0122] The serial numbers of the embodiments of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0123] In the embodiments of the present application, on the one hand, a 3D digital contour model is generated by using original images of different perspectives of the engine, a movement waypoint list of the robot is generated, and then an initial robot movement track is generated. This process does not require manual setting of the track one by one, greatly reducing the time cost of track setting. At the same time, with the increase of engine models and order numbers, this process can quickly adapt and generate corresponding tracks, avoiding a substantial increase in time and labor costs due to the increase in the number of manual settings, thereby significantly improving production efficiency. On the other hand, the robot end camera moves strictly according to the generated track, ensuring the accuracy of the shooting position and angle. In addition, by evaluating the confidence of the target image and optimizing the robot movement track accordingly, the accuracy of the detection result is further improved, ensuring that the verified track can be used for subsequent engine detection of the same order number, thereby improving the reliability of the detection result.

[0124] The present application also provides a computer readable medium having program instructions stored thereon, which, when executed by a processor, implement the robot movement track generation method for engine detection provided by each of the above method embodiments.

[0125] The present application also provides a computer program product containing instructions which, when executed on a computer, cause the computer to perform the robot movement track generation method for engine detection of the above various method embodiments.

[0126] Please refer to Figure 6 The present application provides a structural schematic diagram of an electronic device. As shown in Figure 6 The electronic device 1000 can include at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0127] The communication bus 1002 is configured to realize the connection communication between the components.

[0128] The user interface 1003 can include a display, a camera, and optionally a standard wired interface and a wireless interface.

[0129] The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0130] The processor 1001 can include one or more processing cores. The processor 1001 connects various parts in the electronic device 1000 through various interfaces and lines, executes various functions of the electronic device 1000 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1001 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU is mainly used to process the operating system, the user interface, and the application program; the GPU is used to render and draw the content to be displayed on the display; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 1001, but can be realized by a separate chip.

[0131] The memory 1005 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1005 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above various method embodiments, etc.; the data storage area can store data involved in the above various method embodiments, etc. The memory 1005 can also be at least one storage system located away from the aforementioned processor 1001. As shown in Figure 6 The memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a robot movement trajectory generation application for engine detection.

[0132] In Figure 6 In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an interface for user input and obtain user input data; and the processor 1001 can be used to call the robot movement trajectory generation application for engine detection stored in the memory 1005 and specifically perform the following operations:

[0133] Generate a 3D digital contour model of the engine from the original images of different perspectives of the engine;

[0134] Generate a motion waypoint list of the robot according to the 3D digital contour model;

[0135] Input the motion waypoint list into the robot controller to generate an initial robot movement trajectory of the engine, and traverse the current motion waypoint from the initial robot movement trajectory to control the end camera of the robot to move to the current motion waypoint to take a picture and obtain a target image;

[0136] Input the target image into a pre-trained detection model to output a confidence of the target image; and according to the confidence, optimize the initial robot movement trajectory to obtain a final movement trajectory;

[0137] Store a mapping relationship between an order number of the engine and the final movement trajectory.

[0138] In one embodiment, when the processor 1001 performs the operation of generating a 3D digital contour model of the engine from the original images of different perspectives of the engine, it specifically performs the following operations:

[0139] The original images of different perspectives are preprocessed to obtain high-quality images of different perspectives;

[0140] The preprocessed original images are edge extracted to obtain edge binary images of different perspectives which reserve the contour edges of the engine appearance;

[0141] The edge binary images of different perspectives are used as masks to extract the foreground part of the engine from the high-quality images of different perspectives to obtain foreground images of different perspectives;

[0142] Based on the foreground images of different perspectives, the matching of feature points between images of different perspectives is performed to generate an initial three-dimensional feature point cloud of the engine appearance;

[0143] According to the initial three-dimensional feature point cloud, a 3D digital contour model of the engine is constructed.

[0144] In one embodiment, when the processor 1001 performs the matching of feature points between images of different perspectives based on the foreground images of different perspectives to generate an initial three-dimensional feature point cloud of the engine appearance, the following operations are specifically performed:

[0145] The key feature points in the foreground images of different perspectives which are used to represent the geometric features of the engine are extracted, and the feature descriptors of the key feature points are calculated to obtain a plurality of feature descriptors of the foreground images of different perspectives;

[0146] Through the plurality of feature descriptors of the foreground images of different perspectives, the matching of feature descriptors is performed on the foreground images of different perspectives to determine a plurality of matching point pairs;

[0147] The false matching pairs in the plurality of matching point pairs are eliminated to obtain a plurality of cross-perspective feature point pairs;

[0148] According to the intrinsic and extrinsic parameters of the end camera of the robot, the three-dimensional space coordinates of the cross-perspective feature points in each cross-perspective feature point pair are calculated as the initial three-dimensional feature point cloud of the engine appearance.

[0149] In one embodiment, when the processor 1001 performs the construction of a 3D digital contour model of the engine according to the initial three-dimensional feature point cloud, the following operations are specifically performed:

[0150] According to the initial three-dimensional feature point cloud, an initial 3D mesh model of the engine is established;

[0151] The initial 3D mesh model is smoothed by using a Laplacian smoothing algorithm;

[0152] The local part of the smoothed initial 3D mesh model is fine-tuned by using appearance detail features which are used to represent the geometric features of the engine to obtain a 3D digital contour model of the engine.

[0153] In one embodiment, the processor 1001, when generating the motion waypoint list of the robot according to the 3D digital contour model, specifically performs the following operations:

[0154] traversing each surface of the 3D digital contour model;

[0155] calculating the normal vector of each surface based on the direction of the normal vector pointing to the outside of the engine;

[0156] translating each vertex of each surface in the direction of the normal vector by a preset distance according to the direction of the normal vector of the normal vector of each surface, to obtain the offset vertex coordinates of each surface;

[0157] reconnecting the offset vertex coordinates of each surface to generate a photographing reference surface grid model of the motion reference surface of the end camera of the robot;

[0158] generating a motion waypoint set of the robot photographing according to the photographing reference surface grid model;

[0159] generating the motion waypoint list of the robot according to the motion waypoint set.

[0160] In one embodiment, the processor 1001, when generating the motion waypoint set of the robot photographing according to the photographing reference surface grid model, specifically performs the following operations:

[0161] performing collision detection and line-of-sight analysis on the photographing reference surface grid model to adjust the local of the photographing reference surface grid model to obtain an optimized target photographing reference surface grid model;

[0162] dividing the engine into multiple layers along the Z-axis of the target photographing reference surface grid model through a preset number of layers, each layer corresponding to a horizontal slice of the photographing reference surface, and the waypoints in each layer being uniformly distributed in the horizontal direction;

[0163] dividing the outer surface of each layer into multiple logical regions according to the structure to obtain multiple logical regions of each layer, the multiple logical regions including a front end cover region, a rear end cover region, a shell side region, and a pipeline integration region;

[0164] planning motion waypoints at different positions in each logical region of each layer according to a preset grid density, and inserting a preset number of transition motion waypoints at the boundaries of different logical regions of each layer to obtain the motion waypoint set of the robot photographing.

[0165] In one embodiment, the processor 1001, when generating the motion waypoint list of the robot according to the motion waypoint set, specifically performs the following operations:

[0166] establishing a target space coordinate system with the geometric center of the 3D digital contour model as the origin;

[0167] transforming the coordinates of each motion waypoint in the set of motion waypoints into three-dimensional coordinates under the target space coordinate system to obtain three-dimensional coordinates of each motion waypoint;

[0168] determining the camera pose angle, waypoint number, and photograph triggering signal of each motion waypoint;

[0169] packaging the three-dimensional coordinates of each motion waypoint and the camera pose angle, waypoint number, and photograph triggering signal into a robot motion waypoint list of the robot.

[0170] In one embodiment, the processor 1001, when optimizing the initial robot movement trajectory according to the confidence to obtain the final movement trajectory, specifically performs the following operations:

[0171] In the case where the confidence is greater than or equal to the preset threshold, the photograph triggering signal of the current motion waypoint is set to an open state, and the step of traversing the current motion waypoint from the initial robot movement trajectory is continued to be performed until the motion waypoints in the initial robot movement trajectory are all traversed, and a photograph triggering signal setting result of the motion waypoints is obtained; or,

[0172] In the case where the confidence is less than the preset threshold, the step of controlling the end camera of the robot to move to the current motion waypoint for photographing is continued to be performed;

[0173] In the case where the number of photographs is greater than the preset number and the confidence is less than the preset threshold, the current motion waypoint is marked as an invalid waypoint, the photograph triggering signal of the current motion waypoint is set to a closed state, and the step of traversing the current motion waypoint from the initial robot movement trajectory is continued to be performed until the motion waypoints in the initial robot movement trajectory are all traversed, and a photograph triggering signal setting result of the motion waypoints is obtained;

[0174] According to the photograph triggering signal setting result of the motion waypoints, the initial robot movement trajectory is updated to obtain the final movement trajectory.

[0175] In one embodiment, the processor 1001, when updating the initial robot movement trajectory according to the photograph triggering signal setting result of the motion waypoints to obtain the final movement trajectory, specifically performs the following operations:

[0176] eliminating the motion waypoints in the initial robot movement trajectory whose photograph triggering signals are in a closed state;

[0177] connecting the motion waypoints before and after the eliminated motion waypoints whose photograph triggering signals are in an open state to obtain the final movement trajectory.

[0178] In the embodiments of the present application, on the one hand, the 3D digital contour model is generated by using the original images of different perspectives of the engine, the movement point list of the robot is generated, and then the initial robot movement trajectory is generated. This process does not require manual setting of the trajectory one by one, greatly reducing the time cost of trajectory setting. At the same time, with the increase of engine models and order numbers, this process can quickly adapt and generate the corresponding trajectory, avoiding the substantial increase in time and labor costs caused by the increase in the number of manual settings, thereby significantly improving the production efficiency. On the other hand, the robot end camera moves strictly according to the generated trajectory, ensuring the accuracy of the shooting position and angle. In addition, by evaluating the confidence of the target image and optimizing the robot movement trajectory accordingly, the accuracy of the detection result is further improved, ensuring that the verified trajectory can be used when detecting the same order number engine in the future, thereby improving the reliability of the detection result.

[0179] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program for generating the robot movement trajectory for engine detection can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium of the program for generating the robot movement trajectory for engine detection can be a disk, an optical disk, a read-only memory, a random access memory, etc.

[0180] The above only discloses the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. A robot movement trajectory generation method for engine detection, characterized by, The method comprises: generating a 3D digital contour model of the engine through original images of different perspectives of the engine; generating a motion waypoint list of the robot according to the 3D digital contour model; the generating of the motion waypoint list of the robot according to the 3D digital contour model comprises: traversing each surface of the 3D digital contour model; calculating a normal vector of each surface based on the direction of the normal vector pointing to the outside of the engine; translating each vertex of each surface by a preset distance along the direction of the normal vector of the normal vector of each surface to obtain the vertex coordinates of each surface after offset; reconnecting the vertex coordinates of each surface after offset to generate a photographing reference surface grid model of a motion reference surface of an end camera of the robot; generating a motion waypoint set of the robot photographing according to the photographing reference surface grid model; and generating a motion waypoint list of the robot according to the motion waypoint set; inputting the motion waypoint list into a robot controller to generate an initial robot movement trajectory of the engine, and traversing a current motion waypoint from the initial robot movement trajectory to control the end camera of the robot to move to the current motion waypoint for photographing to obtain a target image; inputting the target image into a pre-trained detection model to output a confidence of the target image; and optimizing the initial robot movement trajectory according to the confidence to obtain a final movement trajectory; storing a mapping relationship between an order number of the engine and the final movement trajectory.

2. The method of claim 1, wherein, The generating of the 3D digital contour model of the engine through original images of different perspectives of the engine comprises: preprocessing the original images of different perspectives to obtain high-quality images of different perspectives; performing edge extraction on the preprocessed original images to obtain edge binary images of different perspectives retaining the contour edges of the appearance of the engine; extracting the foreground part of the engine from the high-quality images of different perspectives by taking the edge binary images of different perspectives as masks to obtain foreground images of different perspectives; performing matching of feature points between images of different perspectives based on the foreground images of different perspectives to generate an initial three-dimensional feature point cloud of the appearance of the engine; constructing the 3D digital contour model of the engine according to the initial three-dimensional feature point cloud.

3. The method of claim 2, wherein, The performing of the matching of feature points between images of different perspectives based on the foreground images of different perspectives to generate the initial three-dimensional feature point cloud of the appearance of the engine comprises: extracting key feature points in the foreground images of different perspectives for representing the geometric features of the engine, calculating feature descriptors of the key feature points to obtain a plurality of feature descriptors of the foreground images of different perspectives; performing matching of feature descriptors on the foreground images of different perspectives through the plurality of feature descriptors of the foreground images of different perspectives to determine a plurality of matching point pairs; eliminating the false matching pairs existing in the plurality of matching point pairs to obtain a plurality of cross-perspective feature point pairs; According to the intrinsic and extrinsic parameters of the end camera of the robot, three-dimensional space coordinates of the cross-view feature points in each cross-view feature point pair are calculated as an initial three-dimensional feature point cloud of the engine appearance.

4. The method of claim 2, wherein, The 3D digital contour model of the engine is constructed according to the initial three-dimensional feature point cloud, including: An initial 3D grid model of the engine is established according to the initial three-dimensional feature point cloud; The initial 3D grid model is smoothed by using a Laplace smoothing algorithm; The local of the initial 3D grid model after smoothing is fine-tuned by appearance detail features for representing geometric features of the engine, to obtain the 3D digital contour model of the engine.

5. The method of claim 1, wherein, The set of motion waypoints of the robot photographing is generated according to the photographing reference surface grid model, including: Collision detection and line-of-sight analysis are performed on the photographing reference surface grid model to adjust the local of the photographing reference surface grid model, to obtain an optimized target photographing reference surface grid model; The engine is divided into multiple layers along the Z-axis of the target photographing reference surface grid model by a preset number of layers, each layer corresponding to a horizontal slice of the photographing reference surface, and the waypoints in each layer being uniformly distributed along the horizontal direction; The outer surface of each layer of the engine is divided into multiple logical regions according to the structure, to obtain multiple logical regions of each layer, and the multiple logical regions include a front end cover region, a rear end cover region, a shell side surface region, and a pipeline integration region; Motion waypoints at different positions in each logical region of each layer are planned according to a preset grid density, and a preset number of transition motion waypoints are inserted at the boundaries of different logical regions in each layer, to obtain the set of motion waypoints of the robot photographing.

6. The method of claim 1, wherein, The motion waypoint list of the robot is generated according to the set of motion waypoints, including: A target space coordinate system is established with the geometric center of the 3D digital contour model as the origin; The coordinates of each motion waypoint in the set of motion waypoints are converted to three-dimensional coordinates in the target space coordinate system, to obtain the three-dimensional coordinates of each motion waypoint; The camera pose angle, waypoint number, and photographing trigger signal of each motion waypoint are determined; The three-dimensional coordinates of each motion waypoint and the camera pose angle, waypoint number, and photographing trigger signal are packaged into the robot motion waypoint list of the robot.

7. The method of claim 1, wherein, The initial robot movement trajectory is optimized according to the confidence to obtain a final movement trajectory, including: In the case that the confidence is greater than or equal to a preset threshold, the photographing trigger signal of the current motion waypoint is set to an open state, and the step of traversing the current motion waypoint from the initial robot movement trajectory is continued to be executed until the motion waypoints in the initial robot movement trajectory are all traversed, to obtain a photographing trigger signal setting result of the motion waypoints; or In the case that the confidence is less than the preset threshold, the step of moving the end camera of the robot to the current motion waypoint for photographing is continued to be executed. In a case that the number of photographing times is greater than the preset number and the confidence levels are all less than the preset threshold, the current motion waypoint is marked as an invalid waypoint, the photographing trigger signal of the current motion waypoint is set to an off state, and the step of traversing the current motion waypoint from the initial robot movement trajectory is continuously performed until all motion waypoints in the initial robot movement trajectory are traversed, and a photographing trigger signal setting result of the motion waypoints is obtained; According to the photographing trigger signal setting result of the motion waypoints, the initial robot movement trajectory is updated to obtain a final movement trajectory.

8. The method of claim 7, wherein, The updating of the initial robot movement trajectory according to the photographing trigger signal setting result of the motion waypoints to obtain the final movement trajectory comprises: eliminating the motion waypoints in the initial robot movement trajectory with the photographing trigger signal in an off state; connecting the motion waypoints before and after the eliminated motion waypoints with the photographing trigger signal in an on state to obtain the final movement trajectory.

9. A robot movement trajectory generation system for engine inspection implemented using the method of any one of claims 1-8, characterized by, The system comprises: a 3D digital contour model generation module configured to generate a 3D digital contour model of the engine through original images of the engine from different perspectives; a motion waypoint list generation module configured to generate a motion waypoint list of a robot according to the 3D digital contour model; a photographing module configured to input the motion waypoint list into a robot controller to generate an initial robot movement trajectory of the engine, traverse a current motion waypoint from the initial robot movement trajectory, control an end camera of the robot to move to the current motion waypoint to take a photograph, and obtain a target image; a trajectory optimization module configured to input the target image into a pre-trained detection model to output a confidence level of the target image, and optimize the initial robot movement trajectory according to the confidence level to obtain a final movement trajectory; a parameter storage module configured to store a mapping relationship between an order number of the engine and the final movement trajectory.

Images