Control method, system, storage medium and terminal of engine detection robot

Abstract

This invention discloses a control method, system, storage medium, and terminal for an engine inspection robot, combining a digital simulation system with a physical snake robot motion control system to form a twin system. The physical robot motion control system includes a monocular vision system and a laser ranging system. The control method includes constructing a digital twin model of the physical robot and a virtual engine model in the digital simulation system; initializing the standard state of the physical snake robot based on deep learning and HOG feature extraction algorithms; automatically planning the robot's route to the target task point using a shortest distance algorithm; fitting the actual motion path of the physical robot into the system during task execution; and using deep learning to continuously optimize the planned robot motion path to achieve iterative control. This invention applies digital twin technology to robot control, enabling continuous iteration in control accuracy, making control more precise, intuitive, and efficient.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and more specifically, to a control method, system, storage medium, and terminal for an engine testing robot. Background Technology

[0002] Digital twins are a novel concept proposed in the industrial manufacturing field in the 21st century. They refer to the digital reconstruction of physical entities in virtual space, the real-time transmission of key data collected from the physical entity back to the virtual space for data processing, and the generation of feedback to the physical entity. The integration of digital twins with the Internet of Things (IoT) is an inevitable trend. IoT technology provides the technical support for data exchange between the physical entity and the virtual twin model required for digital twins. At the same time, digital twin technology addresses the shortcomings of IoT technology, which primarily relies on single sensors for controlling complex objects.

[0003] As the application of digital twin technology expands across various fields, its potential in robot control is gradually gaining attention. In aero-engine inspection, the complex internal structure of the engine makes the movement of snake-like robots extremely difficult. Current research focuses on using obstacle avoidance algorithms based on mathematical models to ensure obstacle avoidance. However, this approach is insufficient when facing complex obstacle environments.

[0004] Because snake-like robots are soft robots, they possess unique compliantness, which provides a significant advantage in their operation within complex aircraft engines. However, this very characteristic also makes traditional methods—based on information collected by robot sensors to determine the robot's posture and position for control—less accurate and intuitive, significantly increasing the difficulty of performing complex actions. Furthermore, locating snake-like robots within aircraft engines is extremely challenging. While high-power nuclear magnetic resonance imaging (NMR) can be used for robot localization, its large size and high cost cannot be ignored. This means that current snake-like robot control methods may fail to complete detection tasks properly due to inaccurate sensor data or operational errors. Summary of the Invention

[0005] To address the above-mentioned technical deficiencies, this invention provides a control method and control system for an engine inspection robot. It applies digital twin technology to the control of a snake robot, enabling continuous iteration in control precision, making the control more accurate, intuitive, and efficient.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a control method for a snake-like robot used for engine inspection, characterized in that a digital simulation system and a physical snake-like robot motion control system are combined to form a twin system; the physical snake-like robot motion control system includes a sensing module; the sensing module includes a monocular vision system and a laser ranging system; the control method includes the following steps:

[0007] Step S11: Construct a digital twin model of the physical snake robot in the digital simulation system based on the engineering drawings of the physical snake robot;

[0008] Step S12: Collect the engine structure and data to be tested, and construct a virtual engine model in the digital simulation system;

[0009] Step S13: Place the physical snake robot at the positioning point and put it in the initialization standard state, and simultaneously start the digital simulation system to record the position parameters;

[0010] Step S14: Using the monocular vision system and laser ranging system, the front end of the physical snake robot makes micro-displacements in various directions to obtain images and distance data of the engine area set for positioning.

[0011] Step S15: Based on deep learning and HOG feature extraction algorithm, compare the acquired image with the set image to adjust the physical snake robot to the standard state set in the digital simulation system.

[0012] Step S16: Obtain the target task point of the snake robot, and automatically plan the route for the snake robot to move to the target task point based on the digital simulation system and the shortest distance algorithm;

[0013] Step S17: During task execution, the actual movement path of the physical robot is fitted to the digital simulation system, and deep learning is used to continuously optimize the snake robot movement path planned by the digital simulation system to achieve iterative control: If the task is successfully completed, the image information obtained by the physical snake robot sensor is sent back to the digital simulation system, and the digital simulation system is continuously optimized; if the task fails, the digital simulation system will record the failure and automatically correct it, or provide feedback to the controller for manual intervention.

[0014] Furthermore, in steps S14 to S17, the localization of the snake robot model by combining the monocular vision system and the laser ranging system includes the following steps:

[0015] Step S21: Perform grayscale processing on the image information captured by the monocular vision system;

[0016] Step S22: Perform Gamma correction on the image information;

[0017] Step S23: Calculate the gradient in the image information using the gradient algorithm and draw the gradient histogram;

[0018] Step S24: Extract edge features using the HOG feature extraction algorithm, and determine the outer contour edge of the engine and the blade edge position by comparing the weights of the gradient histogram, i.e., the cell position of the feature localization point in the image; confirm or correct the position of the snake robot by comparing with the target position.

[0019] Step S25: Determine the distance between the snake robot and the propeller using the image information captured by the monocular vision system and the ranging system. Then, the three-dimensional spatial coordinates P of the monocular vision system camera can be determined. camera Represented as [x, y, z], the position of the snake robot can be determined and corrected through a position transformation algorithm.

[0020] Furthermore, in step S22, the formula for Gamma correction of the image information is:

[0021] Y(x, y) = I(x, y) γ (γ=0.5)

[0022] In the above formula, x and y are the coordinates of the positioning point obtained by the monocular vision system, Y(x, y) represents the encoded value of the pixel, and I(x, y) represents the brightness of the pixel.

[0023] Further, in step S23, the formula for the gradient algorithm is:

[0024] H x (x,y)=I(x+1,y)-I(x-1,y)

[0025] H y (x,y)=I(x,y+1)-I(x,y-1)

[0026]

[0027]

[0028] In the above formula, x and y are the coordinates of the positioning point obtained using the monocular vision system, and H x (x, y) and H y (x, y) represent the difference in brightness between two pixels in the horizontal and vertical directions, respectively, which is the distance between the two pixels when calculating the gradient; I(x, y) represents the brightness of the pixel; S(x, y) represents the length of the gradient vector; α(x, y) represents the angle of the gradient vector.

[0029] Furthermore, in step S24, the robot position is corrected based on the camera coordinates using the following transformation algorithm:

[0030]

[0031] In the above formula, T camera P is a four-dimensional coordinate transformation matrix; camera The fourth dimension of the coordinate system is l, represented as [x, y, z, l], where x and y are the coordinates of the positioning point obtained using the monocular vision system, and z is the depth information obtained by the laser ranging system; the position coordinate information P of the snake robot can be obtained through calculation. robot .

[0032] Secondly, the present invention provides a control system for a snake-like robot for engine inspection, characterized in that the control method for implementing the above-described snake-like robot for engine inspection includes a digital simulation system and a physical snake-like robot motion control system; the digital simulation system includes:

[0033] The model building module is used to build digital twin models and virtual engine models of snake robots;

[0034] The route planning module is used to automatically plan the route for the snake robot to move to the target task point based on the target task point;

[0035] The image and position information processing module is used to process the real-time image and position information of the physical snake robot, which facilitates the positioning and correction of the robot's position.

[0036] The motion control system includes:

[0037] The management module is used to monitor and control the physical snake robot in real time, with a personal computer as the control center.

[0038] The control module directly controls the motor torque output and reducer operation of the snake robot;

[0039] The sensing module includes a monocular vision system and a laser ranging system. The monocular vision system captures images in real time, and the laser ranging system measures relative distances.

[0040] Furthermore, the management module includes a remote control system based on a 5G IoT communication module.

[0041] Furthermore, the control module includes a master control module composed of an STM32 microcontroller and a slave control module composed of multiple integrated joint products and their control boards.

[0042] Thirdly, the present invention provides a computer storage medium having a computer program, characterized in that, when the computer program is executed by a processor, it implements the control method of the snake robot for engine inspection as described above.

[0043] Finally, the present invention provides a smart terminal, characterized in that it includes: one or more memories and one or more processors;

[0044] The one or more memories are used to store computer programs;

[0045] The one or more processors, connected to the memory, are used to run the computer program to execute the control method for the snake robot for engine inspection as described above.

[0046] Compared with the prior art, the above invention has the following advantages or beneficial effects:

[0047] (1) This invention applies digital twin technology to the control of snake robots, enabling continuous iteration in the accuracy of their control, making the control more precise, intuitive and efficient;

[0048] (2) The present invention utilizes a snake robot initialization and positioning control method based on HOG feature extraction algorithm, which reduces the cost of positioning the snake robot in the engine;

[0049] (3) The present invention combines a laser ranging system with a monocular vision system, which ensures that the sensing module is small in size while meeting the high accuracy requirements, and helps to fit and iterate digital twin data.

[0050] (4) The present invention can adopt a relatively mature 5G Internet of Things communication method, which makes the control and data transmission of snake robot more stable and with low latency, and ensures the reliability of the control of the device in complex environments. Attached Figure Description

[0051] The invention, its features and advantages will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0052] Figure 1 This is a flowchart of a snake robot control method in one embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the control system architecture of a snake robot in one embodiment of the present invention;

[0054] Figure 3 This is a connection diagram of the control module and the sensing module in one embodiment of the present invention. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but these are not intended to limit the scope of the invention.

[0056] In the following detailed description, numerous specific details are set forth to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that well-known algorithms and models are not shown in detail to avoid obscuring the spirit of the invention.

[0057] Furthermore, the execution order of actions, steps, etc. in the apparatus and methods shown in the claims, specification, and drawings can be implemented in any order, unless a specific order is explicitly specified, and as long as the output of the preceding processing is not used in the subsequent processing.

[0058] Example 1

[0059] See Figure 1 This embodiment provides a control method for a snake-like robot used for engine inspection, which combines a digital simulation system with a physical snake-like robot motion control system to form a twin system; the physical snake-like robot motion control system includes a sensing module; the sensing module includes a monocular vision system and a laser ranging system; the control method includes the following steps:

[0060] Step S11: Construct a digital twin model of the physical snake robot in the digital simulation system based on the engineering drawings of the physical snake robot, so as to facilitate the overall construction of the digital twin model, and then use it to fit the path planning data.

[0061] Step S12: Collect the engine structure and data to be tested, and construct a virtual engine model in the digital simulation system;

[0062] Step S13: Place the physical snake robot at the positioning point and put it in the initialization standard state, and simultaneously start the digital simulation system to record the position parameters;

[0063] Step S14: Using the monocular vision system and laser ranging system, the front end of the physical snake robot makes micro-displacements in various directions to obtain images and distance data of the engine area set for positioning.

[0064] Step S15: Based on deep learning and HOG feature extraction algorithm, compare the acquired image with the set image to adjust the physical snake robot to the standard state set in the digital simulation system.

[0065] Step S16: Obtain the target task point of the snake robot, and automatically plan the route for the snake robot to move to the target task point based on the digital simulation system and the shortest distance algorithm;

[0066] Step S17: During task execution, the actual movement path of the physical robot is fitted to the digital simulation system, and deep learning is used to continuously optimize the snake robot movement path planned by the digital simulation system to achieve iterative control: If the task is successfully completed, the image information obtained by the physical snake robot sensor is sent back to the digital simulation system, and the digital simulation system is continuously optimized; if the task fails, the digital simulation system will record the failure and automatically correct it, or provide feedback to the controller for manual intervention.

[0067] The digital simulation system is preferably run on the cloud, but it can also be run locally.

[0068] As a preferred technical solution, further: in steps S14 to S17, the localization of the snake robot model by combining the monocular vision system and the laser ranging system includes the following steps:

[0069] Step S21: Perform grayscale processing on the image information captured by the monocular vision system;

[0070] Step S22: Perform Gamma correction on the image information;

[0071] Step S23: Calculate the gradient in the image information using the gradient algorithm and draw the gradient histogram;

[0072] Step S24: Extract edge features using the HOG feature extraction algorithm, and determine the outer contour edge of the engine and the blade edge position by comparing the weights of the gradient histogram, i.e., the cell position of the feature localization point in the image; confirm or correct the position of the snake robot by comparing with the target position.

[0073] Step S25: Determine the distance between the snake robot and the propeller using the image information captured by the monocular vision system and the ranging system. This allows the determination of the three-dimensional spatial coordinates P of the monocular vision system camera. camera Represented as [x, y, z], the position of the snake robot can be determined and corrected through a position transformation algorithm.

[0074] The laser ranging system can use existing LiDAR (Light Detection and Ranging) systems.

[0075] Preferably, in step S22, the formula for Gamma correction of the image information is:

[0076] Y(x, y) = I(x, y) γ (γ=0.5)

[0077] In the above formula, x and y are the coordinates of the positioning point obtained by the monocular vision system, Y(x, y) represents the encoded value of the pixel, and I(x, y) represents the brightness of the pixel.

[0078] Preferably, in step S23, the formula for the gradient algorithm is:

[0079] H x (x,y)=I(x+1,y)-I(x-1,y)

[0080] H y (x,y)=I(x,y+1)-I(x,y-1)

[0081]

[0082]

[0083] In the above formula, x and y are the coordinates of the positioning point obtained using the monocular vision system, and H x (x, y) and H y (x, y) represent the difference in brightness between two pixels in the horizontal and vertical directions, respectively, which is the distance between the two pixels when calculating the gradient; I(x, y) represents the brightness of the pixel; S(x, y) represents the length of the gradient vector; α(x, y) represents the angle of the gradient vector.

[0084] Preferably, in step S24, the robot position is corrected based on camera coordinates using the following transformation algorithm:

[0085]

[0086] In the above formula, T camera P is a four-dimensional coordinate transformation matrix; camera The fourth dimension of the coordinate system is l, represented as [x, y, z, l], where x and y are the coordinates of the positioning point obtained using the monocular vision system, and z is the depth information obtained by the laser ranging system; the position coordinate information P of the snake robot can be obtained through calculation. robot .

[0087] Example 2

[0088] See Figure 2 This embodiment provides a control system for a snake-like robot used for engine inspection, used to implement the control method for the snake-like robot used for engine inspection as described in Embodiment 1, including a digital simulation system and a physical snake-like robot motion control system; the digital simulation system includes:

[0089] The model building module is used to build digital twin models and virtual engine models of snake robots;

[0090] The route planning module is used to automatically plan the route for the snake robot to move to the target task point based on the target task point;

[0091] The image and position information processing module is used to process the real-time image and position information of the physical snake robot, which facilitates the positioning and correction of the robot's position.

[0092] The motion control system includes:

[0093] The management module is used to monitor and control the physical snake robot in real time, with a personal computer as the control center.

[0094] The control module directly controls the motor torque output and reducer operation of the snake robot;

[0095] The sensing module includes a monocular vision system and a laser ranging system. The monocular vision system captures images in real time, and the laser ranging system measures relative distances. Data from the sensing module is simultaneously uploaded to a cloud-based digital simulation system.

[0096] Preferably, the management module includes a remote control system based on a 5G IoT communication module, which can be selected with IoT communication solutions such as 5G communication modules to meet different scenario requirements. The control module includes a master control module composed of an STM32 microcontroller and a slave control module composed of multiple integrated joint products and their control boards. See also Figure 3 The core of the main control module is an STM32 microcontroller, including a 5G wireless communication module and a reset circuit. The 5G IoT wireless communication module ensures low-latency, high-volume data transmission. The slave control module consists of multiple integrated joint products, directly controlling the movement of the snake robot. The power source is a DC brushless motor. Meanwhile, the slave control module's control board supplements the STM32 microcontroller's motor control; the reducer lowers the rotational speed while increasing output torque, ensuring high robot controllability.

[0097] Example 3

[0098] This embodiment provides a computer storage medium having a computer program, which, when executed by a processor, implements the control method for the snake robot used for engine inspection as described in Embodiment 1.

[0099] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may include, but is not limited to, floppy disks, optical disks, CD-ROMs (Read-Only Optical Disk Memory), magneto-optical disks, ROMs (Read-Only Memory), RAMs (Random Access Memory), EPROMs (Erasable Programmable Read-Only Memory), EEPROMs (Electrically Erasable Programmable Read-Only Memory), magnetic cards or optical cards, flash memory, or other types of media / machine-readable media suitable for storing machine-executable instructions. The computer-readable storage medium can be a product not connected to a computer device or a component used in a computer device.

[0100] Example 4

[0101] This embodiment provides a smart terminal, including: one or more memories and one or more processors;

[0102] The one or more memories are used to store computer programs;

[0103] The one or more processors are connected to the memory and are used to run the computer program to perform the control method for the snake robot for engine inspection as described in Embodiment 1.

[0104] Optionally, the memory may include, but is not limited to, high-speed random access memory and non-volatile memory. For example, one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices; the processor 42 may include, but is not limited to, a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0105] Optionally, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0106] Those skilled in the art should understand that variations can be implemented by combining existing technology with the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention, and will not be elaborated here either.

[0107] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and the devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the present invention. This does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the present invention's technical solutions still fall within the protection scope of the present invention.

[0108] Those skilled in the art will recognize that the units, i.e., algorithm steps, of the various examples described in connection with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A control method for a serpentine robot for engine inspection, characterized by, The digital simulation system is combined with the physical snake robot motion control system to form a twin system; the physical snake robot motion control system comprises a sensing module; the sensing module comprises a monocular vision system and a laser ranging system; the control method comprises the following steps: Step S11, a digital twin model of the physical snake robot is constructed in the digital simulation system according to engineering drawings of the physical snake robot; Step S12, collect the engine structure and data to be detected, and construct a virtual engine model in the digital simulation system; Step S13, place the physical snake robot at a positioning point and make it in an initial standard state, and start the digital simulation system to record the position parameters synchronously; Step S14, use the monocular vision system and the laser ranging system to obtain the image and distance data of the engine region to be positioned by slightly moving the front end of the physical snake robot in each direction; Step S15, based on deep learning and HOG feature extraction algorithm, compare the obtained image with the set image, so that the physical snake robot is adjusted to the standard state set in the digital simulation system; Step S16, obtain the target task point of the snake robot, and automatically plan the route of the snake robot moving to the target task point relying on the digital simulation system and the shortest distance algorithm; Step S17, in the task execution process, fit the actual motion route of the physical snake robot in the digital simulation system, and continuously optimize the motion path of the snake robot planned by the digital simulation system by using deep learning, so as to realize the iteration of control: if the task is successfully performed, the image information obtained by the sensor of the physical snake robot is returned to the digital simulation system, and the digital simulation system is continuously optimized; if the task fails, the digital simulation system will record the fault and automatically correct it, or feed back to the controller for manual intervention; In steps S14 to S17, the snake robot model positioning combined with the monocular vision system and the laser ranging system comprises the following steps: Step S21, perform gray processing on the image information captured by the monocular vision system; Step S22, perform Gamma correction on the image information; Step S23, calculate the gradient in the image information by using a gradient algorithm, and draw a gradient histogram; Step S24, extract the edge features by using a HOG feature extraction algorithm, compare the weight of the gradient histogram to determine the position of the outer contour edge and the paddle edge of the engine, that is, the position of the feature positioning point in the cell of the image; compare with the target position to confirm or correct the position of the snake robot; Step S25, the image information captured by the monocular vision system and the distance information of the serpentine robot and the paddle determined by the ranging system, that is, the three-dimensional space coordinates P camera represented as [x, y, z] can be determined by a position transformation algorithm, and the position of the serpentine robot can be confirmed and corrected.

2. The control method of a serpentine robot for engine inspection according to claim 1, characterized by, The formula for performing Gamma correction on the image information in step S22 is: , In the above formula, x, y are coordinates of positioning points acquired by using the monocular vision system, represents an encoding value of a pixel point, represents the brightness of a pixel point.

3. The control method of a serpentine robot for engine inspection according to claim 1, characterized by, In step S23, the formula of the gradient algorithm is: ， In the above formula, x, y are the coordinates of the positioning point acquired by the monocular vision system, and respectively represent the difference value of the luminance of two pixel points in the horizontal and vertical directions, that is, the distance between two pixels in the gradient calculation; represents the luminance of the pixel point; represents the length of the gradient vector; represents the angle of the gradient vector.

4. The control method of a serpentine robot for engine inspection according to claim 1, characterized by, In step S24, the robot position is corrected based on the camera coordinates by using the following transformation algorithm: ， In the above formula, T camera is a four-dimensional coordinate transformation matrix; The coordinate is supplemented with a fourth dimension , expressed as , wherein x, y are the coordinates of the positioning points obtained by the monocular vision system, and z is the depth information obtained by the laser ranging system; the position coordinate information of the snake robot can be obtained by calculation .

5. A control system for a serpentine robot for engine inspection, characterized by, The control method of the snake robot for engine detection according to any one of claims 1 to 4, comprising a digital simulation system and a physical snake robot motion control system; the digital simulation system comprises: A model construction module is configured to construct a digital twin model of the snake robot and a virtual engine model; A route planning module is configured to automatically plan a route for the snake robot to move to a target task point according to the target task point; An image and position information processing module is configured to process images and position information of real-time positions of the physical snake robot, so as to facilitate positioning and correction of the positions of the robot; The action control system comprises: A management module is configured to take a personal computer as a control center to monitor and control the physical snake robot in real time; A control module is configured to directly control motor torque output of the snake robot and operation of a speed reducer; A sensing module comprises a monocular vision system and a laser ranging system, the monocular vision system is configured to capture images in real time, and the laser ranging system is configured to measure relative distances.

6. The control system of a serpentine robot for engine inspection according to claim 5, wherein The management module comprises a remote control system based on a 5G Internet of Things communication module.

7. The control system of a serpentine robot for engine inspection according to claim 5, wherein The control module comprises a master control module formed by an STM32 single-chip microcomputer, and a plurality of slave control modules formed by integrated joint products and control boards thereof.

8. A computer storage medium having a computer program thereon, characterized in that The computer program is executed by a processor to implement the control method of the snake robot for engine detection according to any one of claims 1 to 4.

9. A smart terminal, characterized by Comprise: One or more memories and one or more processors; The one or more memories are configured to store a computer program; The one or more processors are connected to the memories and are configured to run the computer program to perform the control method of the snake robot for engine detection according to any one of claims 1 to 4.

Images