A vision servo control method, device, equipment, and non-volatile storage medium for cutterhead changing in a tunnel boring machine.
By using a data-driven vision servo control method, the problems of accuracy and robustness of vision servo control in complex environments were solved, and high-precision operation of cutterhead changing in tunnel boring machines was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2024-11-21
- Publication Date
- 2026-05-26
AI Technical Summary
Existing visual servo control methods lack accuracy and robustness under conditions of high pollution, occlusion, and complex lighting, and also lack real-time performance and flexibility.
By adopting a data-driven approach, motion control commands are output through image acquisition, processing, compression, and feature vector generation, combined with feedback error judgment, to achieve high-precision and robust visual servo control.
The accuracy and robustness of visual servo control have been improved in complex environments, enhancing the operational capabilities of the tunnel boring machine cutterhead changing robot.
Smart Images
Figure CN119458338B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of servo motion control technology, mainly for visual servo control of tunnel boring machine cutterhead changing robots, and particularly to a visual servo control method, device, equipment and non-volatile storage medium for tunnel boring machine cutterhead changing. Background Technology
[0002] Servo motion control is a motion control technology, which can be simply described as a closed-loop motion control technology. It features closed-loop control that combines the output of motion control commands with the feedback of motion control execution effects.
[0003] Currently, feedback on the execution effect of servo motion control systems is usually obtained through position feedback via encoders, or through image or video capture by vision systems.
[0004] Currently, visual servoing control systems rely on the extraction of geometric features for motion control. However, due to complex operating conditions, such as high pollution, occlusion, and complex lighting conditions, the aforementioned geometric features may not be accurately extracted, leading to a decrease in the accuracy and robustness of visual servoing. Common visual servoing control methods, such as position-based visual servoing and image-based visual servoing, both suffer from significant dependence on environmental conditions.
[0005] To address these issues, primarily visual servoing control under conditions of high pollution, occlusion, and complex lighting, a data-driven approach has been introduced. However, most existing methods rely heavily on large-scale data acquisition and lack real-time performance and flexibility optimization. Therefore, developing a visual servoing method capable of handling complex environments with high real-time performance and robustness has become an urgent problem to solve. Summary of the Invention
[0006] In view of this, this application proposes a visual servo control method, device, equipment and non-volatile storage medium for cutterhead changing in a tunnel boring machine.
[0007] According to one aspect of this application, a vision servo control method for cutterhead changing in a tunnel boring machine is provided, comprising:
[0008] Step S1: Acquire image data and obtain the image to be processed;
[0009] Step S2: Process and compress the image to be processed, and output the feature vector and the compressed image;
[0010] Step S3: Process the feature vector and the compressed image and output motion control commands;
[0011] Step S4: Perform feedback checks on the execution of the motion control command and obtain the first feedback error. Determine the first feedback error. If the first feedback error result meets the first standard, then execute step S41; otherwise, execute step S42.
[0012] Step S41: The first feedback error meets the first standard, and the entire process ends;
[0013] Step S42: If the first feedback error does not meet the first standard, proceed to step S5;
[0014] Step S5: Obtain the execution action described in the current step S4, and obtain the second feedback error. Determine the second feedback error. If the result of the second feedback error meets the second standard, then execute step S51; otherwise, execute step S52.
[0015] Step S51: The second feedback error meets the second standard, and the entire process ends;
[0016] Step S52: If the second feedback error does not meet the second standard, proceed to step S1.
[0017] In one possible implementation, step S2 further includes:
[0018] Step S21: Divide the unprocessed image into a grid and output the first image to be processed;
[0019] Step S22: Obtain the first motion direction of the first image to be processed;
[0020] Step S23: Encode the first image to be processed and combine it with the first motion direction to form the data to be processed.
[0021] In one possible implementation, step S2 further includes:
[0022] Step S24: Record each grid region in the image to be processed and obtain first motion feature vector data in at least two motion dimensions;
[0023] Step S25: Perform autoencoding on the image data to be processed, and obtain the feature vector and the compressed image.
[0024] In one possible implementation, step S3 further includes:
[0025] Step S31: Combine the vector data and the compressed image with known training data to obtain the first nearest neighbor feature label;
[0026] Step S32: Determine the movement direction of the first neighboring label and obtain the movement direction of the first grid;
[0027] Step S33: Output the motion direction of the first grid as the motion control command.
[0028] In one possible implementation, step S4 further includes:
[0029] Step S401: Establish an oscillation counter to acquire the change in motion speed during the motion process and output the first feedback error;
[0030] Step S402: Establish a first error determiner to monitor whether the first feedback error meets the first standard.
[0031] In one possible implementation, step S5 further includes:
[0032] Step S501: Establish a processor to decompose the motion control command into a first motion speed and a first motion direction;
[0033] Step S502: Establish a second error calculator and output the first motion speed and the first motion direction as the second feedback error;
[0034] Step S503: Establish a second error arbiter to monitor whether the second feedback error meets the second standard.
[0035] In one possible implementation, step S4 further includes:
[0036] The calculation process includes:
[0037] Step S4011: Set a counting variable and a maximum motion value;
[0038] Step S4012: If the first motion feature vector and / or the first neighboring label and / or the image to be processed and / or the motion control command are out of phase and / or have opposite signs, count the counting variables;
[0039] Step S4013: Specify the counting time for statistical analysis of the count variable, and combine the count variable with the counting time to convert it into a first oscillation frequency;
[0040] Step S4014: Calculate the first oscillation frequency and the maximum value of motion, and output the first feedback error.
[0041] According to another aspect of this application, a vision servo control device for cutterhead changing in a tunnel boring machine is provided, characterized in that it includes: an image acquisition module, an image processing module, a servo control module, and a vibration detection module;
[0042] The image acquisition module is used to acquire image data;
[0043] The image processing module is used to perform network partitioning, encoding, vector calculation, and compression of the image and output motion control commands.
[0044] The servo control module is used to execute the motion control command and determine whether the motion control command has been executed to the set parameters;
[0045] The oscillation detection module is used to detect and determine the oscillation frequency.
[0046] According to another aspect of this application, a vision servo control device for cutterhead changing in a tunnel boring machine is characterized in that it includes: a processor and a memory for storing processor-executable instructions, wherein the processor is configured to implement any of the methods described above when executing the executable instructions.
[0047] According to another aspect of this application, a vision servo control non-volatile storage medium for cutterhead changing in a tunnel boring machine is provided, characterized in that the non-volatile storage medium includes stored computer program instructions, which, when executed by a processor, implement the method described in any one of the above methods.
[0048] Other features and aspects of this application will become clearer from the following detailed description of exemplary embodiments with reference to the accompanying drawings.
[0049] This application is applicable to situations where the accuracy and robustness of the visual servo system are improved during the cutterhead changing process of a tunnel boring machine in complex scenarios. Under harsh conditions such as changes in lighting, pollution, and partial occlusion, the servo control system of this application can achieve high-precision and high-robustness visual servo control, effectively improving the operation capability of the cutterhead changing robot of the tunnel boring machine. Attached Figure Description
[0050] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.
[0051] Figure 1 A flowchart illustrating a vision servo control method for cutterhead changing in a tunnel boring machine according to an embodiment of this application is shown.
[0052] Figure 2This diagram shows the main structure of a vision servo control device for cutterhead changing in a tunnel boring machine, according to an embodiment of this application.
[0053] Figure 3 This diagram shows the main structure of a vision servo control device for cutterhead changing in a tunnel boring machine, according to an embodiment of this application. Detailed Implementation
[0054] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.
[0055] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.
[0056] Furthermore, to better illustrate this disclosure, numerous specific details are provided in the following detailed embodiments. Those skilled in the art will understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this application.
[0057]
Method Implementation Examples
[0058] Figure 1 This illustration shows a flowchart of a vision servo control method for cutterhead changing in a tunnel boring machine, according to an embodiment of this application. Figure 1 As shown, the method includes: Step S1: Acquire image data and obtain the image to be processed; Step S2: Process and compress the image to be processed and output feature vectors and compressed images; Step S3: Process the feature vectors and compressed images and output motion control commands; Step S4: Perform feedback checks on the execution of the motion control commands and obtain a first feedback error. The first feedback error is then judged. If the result of the first feedback error meets a first standard, step S41 is executed; otherwise, step S42 is executed. Step S41: If the first feedback error meets the first standard, the entire process ends. Step S42: If the first feedback error does not meet the first standard, step S5 is executed. Step S5: Obtain the execution action in the current step S4 and obtain a second feedback error. The second feedback error is then judged. If the result of the second feedback error meets a second standard, step S51 is executed; otherwise, step S52 is executed. Step S51: If the second feedback error meets the second standard, the entire process ends. Step S52: If the second feedback error does not meet the second standard, step S1 is executed.
[0059] In another preferred embodiment, the image data is acquired using a monocular vision device for sampling.
[0060] In another preferred embodiment, the image data is acquired using a binocular vision device for sampling.
[0061] In another preferred embodiment, the image data is acquired using a charge-coupled device (CCD) image acquisition device for sampling.
[0062] In another preferred embodiment, the image data is acquired using a solid-state imaging device (CMOS) image acquisition device for sampling.
[0063] In another preferred embodiment, the images are sampled using a location-based visual servoing system.
[0064] In another preferred embodiment, the images are acquired by sampling using an image-based visual servoing method.
[0065] In another preferred embodiment, the image to be processed is further processed by edge finding and / or halftone binarization and / or error diffusion binarization and / or edge extraction and / or grayscale correction and / or feature finding and / or brightness gain and / or enhanced denoising and / or de-denoising.
[0066] In another preferred embodiment, the first feedback error is ±0.05% to 30% of the difference between the oscillation frequency and the maximum value of the motion.
[0067] In another preferred embodiment, the second feedback error is: ±0.05% to 30% of the difference between the current speed and the set speed and ±0.05% to 30% of the difference between the current direction of motion and the actual direction.
[0068] Furthermore, in another preferred embodiment, step S2 further includes: step S21: dividing the unprocessed image into a grid and outputting a first image to be processed; step S22: obtaining a first motion direction of the first image to be processed; step S23: encoding the first image to be processed and combining it with the first motion direction to form data to be processed.
[0069] In another preferred embodiment, the image to be processed is divided into grid regions. Further, the image to be processed is divided into 4 grid regions; further, the image to be processed is divided into 8 grid regions; further, the image to be processed is divided into 9 grid regions; further, the image to be processed is divided into 16 grid regions; further, the image to be processed is divided into 25 grid regions; further, the image to be processed is divided into 36 grid regions; further, the image to be processed is divided into 49 grid regions; further, the image to be processed is divided into 64 grid regions; further, the image to be processed is divided into 81 grid regions; further, the image to be processed is divided into up to 4096 grid regions; after the image to be processed is divided into grid regions, a first image to be processed is output.
[0070] In another preferred embodiment, images to be processed are acquired from different locations and divided into grid regions.
[0071] In another preferred embodiment, the first motion direction of each grid is obtained by comparing images of the to be processed acquired at different locations and comparing images of different grid regions.
[0072] In another preferred embodiment, each grid region has four different positional motion feature data directions. Further, the first motion direction includes: each of the following motion directions: up and / or down and / or left and / or right and / or forward and / or backward.
[0073] In another preferred embodiment, each grid region outputs a first motion direction through an expert system; further, the first motion direction includes a vector combination of different directions of two motion axes in any adjacent three-dimensional space.
[0074] Furthermore, in another preferred embodiment, step S2 further includes: step S24: recording each grid region in the image to be processed and obtaining first motion feature vector data of at least two motion dimensions; step S25: performing self-encoding on the image data to be processed and obtaining feature vectors and compressed images.
[0075] In another preferred embodiment, the first motion vector data includes the speed of motion and the direction of motion.
[0076] In another preferred embodiment, the direction of movement includes: each of the following directions of movement: up and / or down and / or left and / or right and / or forward and / or backward.
[0077] In another preferred embodiment, the grid region image and the first motion vector are encoded by an autoencoder to generate data to be processed.
[0078] In another preferred embodiment, the autoencoder is a machine learning-based autoencoder, further wherein the autoencoder is performed using a backpropagation algorithm and an optimization algorithm.
[0079] In another preferred embodiment, the optimization algorithm employs the gradient descent algorithm.
[0080] In another preferred embodiment, the autoencoder is an encoding algorithm performed by a feedforward neural network.
[0081] In another preferred embodiment, the autoencoder employs a supervised learning computer training algorithm.
[0082] In another preferred embodiment, the autoencoder employs an unsupervised learning-based computer training algorithm.
[0083] In another preferred embodiment, the autoencoder is a stacked autoencoder.
[0084] In another preferred embodiment, the autoencoder is a variable autoencoder.
[0085] In another preferred embodiment, a first motion direction is generated by comparing the feature vector after compression by the autoencoder with the clustering direction.
[0086] Furthermore, in another preferred embodiment, step S3 further includes: step S31: obtaining a first neighboring feature label by combining the vector data and the compressed image with known training data; step S32: determining the motion direction of the first neighboring label and obtaining the first grid motion direction; step S33: outputting the first grid motion direction as a motion control command.
[0087] In another preferred embodiment, the feature vector is input to a K-nearest neighbor classifier (hereinafter referred to as: KNN classifier) along with the compressed image. The KNN classifier compares the feature vector of the current grid with the nearest neighbors in the large dataset and / or the already trained dataset. The first nearest feature label is obtained by judging the labels in the neighbor data.
[0088] In another preferred embodiment, the KNN classifier obtains the first motion direction by the direction of the first neighboring label.
[0089] In another preferred embodiment, the KNN classifier outputs motion control commands via a first motion direction, and further, the motion control commands include: motion speed and motion direction.
[0090] In another preferred embodiment, the first motion direction is output as a motion control command by using a Jacobian matrix to convert the first grid motion direction into a motion control command.
[0091] In another preferred embodiment, the Jacobian matrix is calculated in real time within each period.
[0092] In another preferred embodiment, the Jacobian matrix is obtained through offline sampling and training.
[0093] Furthermore, in another preferred embodiment, step S4 further includes: step S401: establishing an oscillation counter to acquire the change in motion speed during the motion process and outputting a first feedback error; step S402: establishing a first error determiner to monitor whether the first feedback error meets a first standard.
[0094] Furthermore, in another preferred embodiment, the calculation process in step S4 further includes: step S4011: setting a counting variable and a maximum motion value; step S4012: if the first motion feature vector and / or the first neighboring label and / or the image to be processed and / or the motion control command are out of phase and / or have opposite signs, counting the counting variable; step S4013: specifying the counting time for statistical analysis of the counting variable, and combining the counting variable with the counting time to convert it into a first oscillation frequency; step S4014: performing a calculation between the first oscillation frequency and the maximum motion value, and outputting it as a first feedback error.
[0095] In another preferred embodiment, a counting variable and a threshold are set to record the number of oscillations of the motion control command.
[0096] In another preferred embodiment, when the count variable reaches the number of oscillations, the first feedback error is determined to meet the first criterion.
[0097] In another preferred embodiment, the counting variable is counted each time the first motion feature vector and / or the first neighboring label and / or the image to be processed and / or the motion control command is detected to have the opposite sign to the previous first motion feature vector and / or the first neighboring label and / or the image to be processed and / or the motion control command component.
[0098] Furthermore, in another preferred embodiment, step S5 further includes: step S501: establishing an arithmetic unit for decomposing the motion control command into a first motion speed and a first motion direction; step S502: establishing a second error arithmetic unit for outputting the first motion speed and the first motion direction as a second feedback error; step S503: establishing a second error arbitrator for monitoring whether the second feedback error meets the second standard.
[0099] In another preferred embodiment, the first motion speed and the first motion direction are sent to the motion actuator via an inverse kinematics algorithm.
[0100] In another preferred embodiment, the second criterion includes: errors in motion speed and errors in motion direction.
[0101] In another preferred embodiment, the error in motion speed is ±0.00% to 50.00%.
[0102] In another preferred embodiment, the error in the direction of motion is ±0.00% to 50.00%.
[0103] [Device Example]
[0104] Figure 2 An apparatus according to an embodiment of this application is shown, comprising: an image acquisition module 100, an image processing module 200, a servo control module 300, and an oscillation detection module 400; the image acquisition module 100 is used to acquire image data; the image processing module 200 is used to perform network partitioning, encoding, vector calculation, and compression on the image and output motion control commands; the servo control module 300 is used to execute motion control commands and determine whether the motion control commands have been executed to the set parameters; the oscillation detection module 400 is used to detect oscillation frequency and make a judgment.
[0105]
Equipment Implementation Example
[0106] Figure 3 An apparatus according to an embodiment of this application is shown. For example... Figure 3 As shown, the device 800 includes a processor 810 and a memory 820 for storing executable instructions of the processor 810. The processor 810 is configured to implement any of the aforementioned vision servo control methods for cutterhead changing in a tunnel boring machine when executing the executable instructions.
[0107] It should be noted here that the number of processors 810 can be one or more. Furthermore, the device 800 in this embodiment may also include an input device 830 and an output device 840. The processors 810, memory 820, input device 830, and output device 840 can be connected via a bus or other means, without specific limitations here.
[0108] The memory 820, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the visual servo control method for cutterhead changing in a tunnel boring machine under complex scenarios, as described in this application embodiment. The processor 810 executes the software programs or modules stored in the memory 820, thereby enabling various functional applications and data processing of the device 800. The input device 830 can be used to receive input digital numbers or signals. These signals can be key signals related to user settings and function control of the device / terminal / server. The output device 840 may include various execution devices such as a display screen and motion actuators.
[0109] [Storage Media Examples]
[0110] According to a fourth aspect of this disclosure, a non-volatile computer-readable storage medium is also provided, on which computer program instructions are stored, which, when executed by a processor 810, implement any of the preceding methods for visual servo control of cutterhead changing in a tunnel boring machine.
[0111] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A vision servo control method for cutterhead changing in a tunnel boring machine, characterized in that: Step S1: Acquire image data and obtain the image to be processed; Step S2: Process and compress the image to be processed, and output the feature vector and the compressed image; Step S3: Process the feature vector and the compressed image and output motion control commands; Step S4: Perform feedback checks on the execution of the motion control command and obtain the first feedback error. Determine the first feedback error. If the first feedback error result meets the first standard, then execute step S41; otherwise, execute step S42. Step S41: The first feedback error meets the first standard, and the entire process ends; Step S42: If the first feedback error does not meet the first standard, proceed to step S5; Step S5: Obtain the execution action described in the current step S4, and obtain the second feedback error. Determine the second feedback error. If the result of the second feedback error meets the second standard, then execute step S51; otherwise, execute step S52. Step S51: The second feedback error meets the second standard, and the entire process ends; Step S52: If the second feedback error does not meet the second standard, proceed to step S1; The first feedback error is obtained by calculating the first oscillation frequency and the maximum value of the motion; The second feedback error is the difference between the current speed expected by the motion control command and the actual speed, and the difference between the current motion direction expected by the motion control command and the actual direction.
2. The method according to claim 1, characterized in that, Step S2 further includes: Step S21: Divide the image to be processed into a grid and output the first image to be processed; Step S22: Obtain the first motion direction of the first image to be processed; Step S23: Encode the first image to be processed and combine it with the first motion direction to form a second image to be processed.
3. The method according to claim 2, characterized in that, Step S2 further includes: Step S24: Record each grid region in the second image to be processed, and obtain first motion feature vector data with at least two motion dimensions; Step S25: Perform autoencoding on the second image to be processed, and obtain the feature vector and the compressed image.
4. The method according to claim 3, characterized in that, Step S3 further includes: Step S31: Obtain the first nearest neighbor feature label by combining the feature vector with the compressed image using known training data; Step S32: Determine the motion direction of the first neighboring feature label and obtain the motion direction of the first grid; Step S33: Output the motion direction of the first grid as the motion control command.
5. The method according to claim 4, characterized in that, Step S4 also includes: Step S401: Establish an oscillation counter to acquire the change in motion speed during the motion process and output the first feedback error; Step S402: Establish a first error determiner to monitor whether the first feedback error meets the first standard.
6. The method according to claim 1, characterized in that, Step S5 further includes: Step S501: Establish a processor to decompose the motion control command into a first motion speed and a first motion direction; Step S502: Establish a second error calculator and output the first motion speed and the first motion direction as the second feedback error; Step S503: Establish a second error arbiter to monitor whether the second feedback error meets the second standard.
7. The method according to claim 5, characterized in that, Step S401 further includes: The calculation process includes: Step S4011: Set a counting variable and the maximum value of the motion; Step S4012: If the first motion feature vector and / or the first neighboring feature label and / or the image to be processed and / or the motion control command are out of phase and / or have opposite signs, count the counting variables; Step S4013: Specify the counting time for statistical analysis of the count variable, and combine the count variable with the counting time to convert it into a first oscillation frequency; Step S4014: Calculate the first oscillation frequency and the maximum value of motion, and output the first feedback error.
8. A vision servo control device for cutterhead changing in a tunnel boring machine, characterized in that, include: A processor, a memory for storing processor-executable instructions, wherein the processor is configured to implement the method of any one of claims 1 to 7 when executing the executable instructions.
9. A vision servo control non-volatile storage medium for cutterhead changing in a tunnel boring machine, characterized in that, The non-volatile storage medium includes stored computer program instructions that, when executed by a processor, implement the method described in any one of claims 1 to 7.