Communication Control Method for Marine Charging Robot Based on Visual Navigation

Abstract

The present invention relates to the technical field of marine robot control, and discloses a communication control method for a marine charging robot based on visual navigation, including the following steps: S101, collecting first-resolution image data, performing rough positioning processing on the target ship, and determining the initial position of the charging target area; S102, performing first feature stability analysis, dynamically generating a first region of interest, and performing precise positioning processing; S103, dividing the first-resolution image data into a first region, a second region, and a third region; S104, collecting the attitude information of the robot, performing first motion trend prediction processing, and generating a first correction instruction according to the prediction result of the first motion trend; S105, based on the master-slave communication protocol, the robot control unit makes adjustments according to the first correction instruction and performs fine-tuning according to the real-time position. The present invention realizes the autonomous docking charging of the marine charging robot with high precision, high reliability, and high efficiency under complex sea conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of marine robot control, and specifically relates to a communication control method for a marine charging robot based on visual navigation. Background Art

[0002] With the development of ship electrification, marine automatic charging technology has become the key to improving port operation efficiency and energy supply safety. However, traditional ship charging operations mainly rely on manual labor, which has problems such as high labor intensity, low safety, and insufficient efficiency. Especially in the complex marine environment, factors such as wave swaying, light changes, and communication delays make it difficult to ensure the accuracy and stability of charging docking. In the prior art, although charging robots based on visual navigation can partially replace manual operations, they still face the following challenges:

[0003] First, the adaptability to dynamic environments is insufficient. Sea waves and ship swaying can cause the images collected by visual sensors to be blurred or jittery, and the recognition rate of traditional feature point matching algorithms drops significantly under low light or high specular reflection conditions, making it difficult to achieve stable positioning. Second, the communication bandwidth is limited, and the transmission of high-resolution images has a high delay in the low-bandwidth marine environment, affecting the real-time response ability of the robot. In addition, the collaborative optimization of multi-sensors is insufficient. Existing systems lack in-depth fusion of data from visual sensors, inertial measurement units, lidar, etc., resulting in insufficient accuracy of motion prediction and correction instructions and affecting the final docking success rate. Summary of the Invention

[0004] The present invention provides a communication control method for a marine charging robot based on visual navigation, which solves the technical problems of low charging docking accuracy, large communication delay, and poor anti-interference ability in the complex marine environment in related technologies.

[0005] The present invention provides a communication control method for a marine charging robot based on visual navigation, including the following steps:

[0006] S101, collect first-resolution image data through the visual sensor of the end effector of the robot, perform rough positioning processing on the target ship, and determine the initial position of the charging target area;

[0007] S102, based on the initial position of the charging target area, perform a first feature stability analysis, dynamically generate a first region of interest, and extract second-resolution image data based on the first region of interest for precise positioning processing;

[0008] S103, divide the first-resolution image data into a first region, a second region, and a third region, where:

[0009] Apply a first compression parameter to the first region and give priority to transmission while maintaining the original resolution;

[0010] Apply the second compression parameter to the second region for medium-compression transmission;

[0011] Apply the third compression parameter to the third region for high-compression ratio transmission;

[0012] S104, Collect the attitude information of the robot, perform the first motion trend prediction process, and generate the first correction instruction according to the prediction result of the first motion trend;

[0013] S105, Based on the master-slave communication protocol, the master control unit sends the first correction instruction to the robot control unit. The robot control unit adjusts according to the first correction instruction and performs fine-tuning according to the real-time position.

[0014] Further, the rough positioning process includes:

[0015] Identify the preset optical positioning marks on the deck of the target ship, and the optical positioning marks are high-contrast concentric circles;

[0016] Calculate the initial position of the charging target area according to the geometric center coordinates of the optical positioning marks and the known installation positions.

[0017] Further, the steps of generating the first region of interest include:

[0018] S201, In the charging target area, identify and extract the feature points in the first-resolution image data, where the feature points include corner points and edge feature points;

[0019] S202, Continuously track the feature points, calculate the average displacement of each feature point between consecutive frames, and the proportion of the number of consecutive frames in the total number of frames, and perform weighted calculation on the average displacement and the proportion based on a preset weight to obtain the feature point stability feature;

[0020] S203, Divide the charging target area into N sub-regions according to the positions of the feature points, N is a positive integer, and the sub-regions are equal-size grid regions;

[0021] S204, Based on the feature point stability feature in each sub-region, calculate the stability score of the sub-region, and use the sub-regions with stability scores greater than the second preset threshold as the first region of interest.

[0022] Further, the steps of extracting the second-resolution image data based on the first region of interest and performing precise positioning processing include:

[0023] S301, Based on the metal reflection characteristics and wave interference characteristics of the ship's charging hole, perform high-fidelity image extraction on the first region of interest to obtain the second-resolution image data, where the high-fidelity image extraction is realized by bicubic interpolation and gamma correction;

[0024] S302. By using an edge feature extraction method to suppress dynamic noise, lock the preset logo contour of the charging hole hatch, and construct a positioning operation area;

[0025] S303. Dynamically adjust the positioning operation area according to the average displacement of the feature points.

[0026] Further, set the positioning operation area in the second-resolution image data as the first area, and apply the first compression parameter to give priority to transmission while maintaining the original resolution;

[0027] Set the part of the first region of interest except the first area in the second-resolution image data as the second area, and apply the second compression parameter for medium-compression transmission;

[0028] Set the other parts except the first region of interest in the first-resolution image data as the third area, and apply the third compression parameter for high-compression ratio transmission.

[0029] Further, the attitude information includes: the spatial position, speed, yaw angle, pitch angle, and roll angle of the robot.

[0030] Further, the first motion trend prediction uses the Kalman filter algorithm to calculate the future position and future speed of the robot.

[0031] Further, the steps of generating the first correction instruction include:

[0032] S401. Calculate the deviation distance by comparing the future position of the robot and the position of the current charging hole;

[0033] S402. Generate the first correction instruction according to the deviation distance. The first correction instruction includes the direction adjustment, speed adjustment, and angle adjustment of the end effector of the robot.

[0034] Further, the master-slave communication protocol includes:

[0035] The master control unit sends the first correction instruction;

[0036] The robot control unit receives the first correction instruction from the master control unit, executes the specified operation, and when the distance between the end effector of the robot and the charging target area is less than the first preset threshold, makes fine-tuning actions autonomously according to the position of the end effector of the robot and the position of the charging target area.

[0037] The beneficial effects of the present invention are as follows: By combining multi-level visual processing with preset optical markers and high-fidelity image extraction technology, the present invention significantly improves the accuracy and efficiency of charging hole recognition; by adopting feature point stability analysis and dynamic region adjustment mechanism, it effectively overcomes the interference of complex environments such as sea waves and reflections, ensuring that the robot can continuously lock the target under dynamic conditions.

[0038] Through the master-slave hierarchical communication protocol, the master control unit sends the first correction instruction, and the slave ship robot executes it and makes autonomous fine-tuning based on local sensor data, greatly reducing the communication bandwidth requirements. Combined with the image data hierarchical compression and transmission strategy, it prioritizes the real-time performance of core data under limited bandwidth, enabling the system to still operate stably in the low-bandwidth environment at sea. Brief Description of the Drawings

[0039] Figure 1 is a flowchart of the communication control method of the marine charging robot based on visual navigation of the present invention. Detailed Embodiments

[0040] Now, the subject matter described herein will be discussed with reference to exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the scope of protection of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described relative to some examples can also be combined in other examples.

[0041] As Figure 1 shown, the communication control method of the marine charging robot based on visual navigation includes the following steps:

[0042] S101, collecting first-resolution image data through the visual sensor of the robot end effector, performing rough positioning processing on the target ship, and determining the initial position of the charging target area;

[0043] S102, based on the initial position of the charging target area, performing the first feature stability analysis, dynamically generating the first region of interest, and extracting second-resolution image data based on the first region of interest for precise positioning processing;

[0044] S103, dividing the first-resolution image data into a first region, a second region, and a third region, where:

[0045] Applying the first compression parameter to the first region and giving priority to transmission while maintaining the original resolution;

[0046] Applying the second compression parameter to the second region and performing medium compression transmission;

[0047] Apply the third compression parameter to the third region for high-compression-rate transmission;

[0048] S104. Collect the pose information of the robot, perform the first motion trend prediction process, and generate the first correction instruction according to the prediction result of the first motion trend;

[0049] S105. Based on the master-slave communication protocol, the master control unit sends the first correction instruction to the robot control unit. The robot control unit makes adjustments according to the first correction instruction and performs fine-tuning according to the real-time position.

[0050] In an embodiment of the present invention, first-resolution image data is collected by the RGB camera of the robot end effector, and denoising and enhancement processing are performed on it to ensure the image quality.

[0051] In an embodiment of the present invention, the rough positioning process includes:

[0052] Identify the preset optical positioning marks on the deck of the target ship. The optical positioning marks are high-contrast concentric circles;

[0053] Calculate the initial position of the charging target area according to the geometric center coordinates of the optical positioning marks and the known installation positions;

[0054] Specifically, the optical positioning marks are made of retroreflective materials with a reflectivity greater than 80% and a black-and-white contrast greater than 90% to ensure recognition under low-light conditions.

[0055] In an embodiment of the present invention, the steps for generating the first region of interest include:

[0056] S201. In the charging target area, identify and extract the feature points in the first-resolution image data. The feature points include corner points and edge feature points. Specifically, the SIFT algorithm is used to extract the feature points in the first-resolution image data;

[0057] S202. Perform continuous frame tracking on the feature points. By calculating the average displacement of each feature point between consecutive frames, as well as the proportion of the number of consecutive frames in the total number of frames, and performing weighted calculation on the average displacement and the proportion based on a preset weight, the feature point stability feature is obtained. Specifically, the calculation formula for the feature point stability feature is: , where represents the feature point stability feature, which is used to measure the stability degree of the feature point in consecutive frames, represents the average displacement, represents the maximum allowable displacement. Preferably, is set to 4 pixels, represents the number of consecutive frames, and K represents the total number of frames. and represent a first weight coefficient and a second weight coefficient;

[0058] S203, divide the charging target area into N sub-areas according to the positions of the feature points, where N is a positive integer, and the sub-areas are equal-sized grid areas;

[0059] S204, calculate the stability score of each sub-area based on the feature point stability characteristics within the sub-area, and regard the sub-areas with stability scores greater than a second preset threshold as the first regions of interest. Specifically, the stability score of a sub-area is the weighted statistical result of the feature point stability characteristics of all feature points within the sub-area, which is used to measure whether the entire sub-area is stable.

[0060] In an embodiment of the present invention, if there are multiple sub-areas with stability scores greater than the second preset threshold, all spatially continuous and adjacent sub-areas are merged to form the first regions of interest. For the isolated sub-areas that meet the threshold conditions but are not connected to the first regions of interest, they are marked as abnormal areas and not included in the scope of the first regions of interest. In this way, the interference of false high-stability areas caused by waves, water surface reflection, etc. can be effectively excluded, and the stable recognition ability of the present invention in complex marine environments and the robustness of the generation of regions of interest can be improved.

[0061] In an embodiment of the present invention, the steps of extracting second-resolution image data based on the first regions of interest and performing precise positioning processing include:

[0062] S301, perform high-fidelity image extraction on the first regions of interest based on the metal reflection characteristics and wave interference characteristics of the ship's charging holes to obtain second-resolution image data. Among them, the high-fidelity image extraction is realized through bicubic interpolation and gamma correction. Specifically, since there are a large number of metal materials on the ship's deck, the charging holes are often accompanied by high-brightness reflection areas, and the interference of sea waves will cause image blurring and distortion. Therefore, in this embodiment, before extracting the second-resolution image data, the image quality of the first regions of interest is enhanced in combination with environmental characteristics, including:

[0063] Use the bicubic interpolation method to perform enlarged resampling on the image of the first regions of interest to enhance the detail texture while maintaining the edge smoothness;

[0064] Perform gamma correction operations to adjust the image brightness and contrast and improve the detail recognition of the reflection areas;

[0065] Through the above processing, second-resolution image data suitable for high-precision recognition can be obtained;

[0066] S302. By means of an edge feature extraction method that suppresses dynamic noise, lock the preset logo contour of the charging hole hatch and construct a positioning operation area. Specifically, due to the image noise brought by the wavy dynamic background, the edge recognition result is unstable. Therefore, in this embodiment, an edge detection method with the ability to suppress dynamic noise is preferably adopted, such as performing operations such as temporal filtering and inter-frame averaging before edge detection to enhance the coherence of real edges. The positioning operation area refers to an adjustable control range area set around the recognized charging hole logo contour in the image, and this area is used to cope with situations such as image disturbance and target micro-displacement;

[0067] Through the above processing, this embodiment can stably recognize the preset logo contour matching the charging hole hatch, such as an elliptical border with a specific shape;

[0068] S303. Dynamically adjust the positioning operation area according to the average displacement amount of feature points. Specifically, to improve the stability of the present invention in a dynamic environment such as wave interference, this embodiment dynamically adjusts the positioning operation area according to the average displacement amount of feature points to ensure that the charging hole is continuously within the positioning operation area. The specific steps include: preset a feature point drift threshold. When the average displacement amount of more than half of the feature points in the positioning operation area exceeds the feature point drift threshold, adjust the positioning operation area, including: taking the geometric center of the current positioning operation area as the reference point, and on the basis of the reference point, magnify the size of the positioning operation area according to a set ratio. If it is detected that the average displacement amount decreases in the subsequent frame image, automatically shrink the positioning operation area to a smaller size according to the ratio;

[0069] Through the above dynamic adjustment mechanism, the present invention can maintain the continuous recognition ability of the charging hole area under unstructured disturbances such as sea surface fluctuations, and at the same time improve the success rate and accuracy of the robot end docking action.

[0070] In an embodiment of the present invention, in a marine scenario, image data needs to be transmitted through a wireless link, such as WiFi, 4G / 5G. When transmitting high-resolution images, it will be limited by the broadband, resulting in a slow response speed of the ship system. To optimize the transmission efficiency of image data in a low-bandwidth marine environment and at the same time ensure the recognition accuracy of key areas, the system performs hierarchical processing on the image data sources according to the importance of the image areas, specifically including:

[0071] Set the positioning operation area in the second-resolution image data as the first area. Since it contains the charging hole logo contour and its key features, it is processed with the first compression parameter. The first compression parameter is a low compression ratio strategy, and the compression ratio is set to lossless compression; ensure that image details are not lost, and at the same time this area is set to have the highest transmission priority;

[0072] For the second region within the first region of interest in the second-resolution image data but not containing the positioning region, it is processed with the second compression parameter, and the second compression parameter is a medium compression ratio strategy with the compression ratio set to JPEG quality 70 to ensure a balance between image quality and transmission efficiency;

[0073] For the other regions in the first-resolution image data except the first region of interest, they are set as the third region and processed with the third compression parameter. The third compression parameter is a high compression ratio strategy with the compression ratio set to below JPEG quality 50 to minimize bandwidth occupancy and improve transmission efficiency;

[0074] Through this method of hierarchical processing of image data, the present invention can, in a low-bandwidth environment, prioritize ensuring high-resolution transmission of key regions such as the charging hole. At the same time, through high compression processing of the background region, it significantly improves data transmission efficiency and ensures that the ship system can still achieve precise charging docking operations under bandwidth-limited conditions.

[0075] In an embodiment of the present invention, the attitude information includes: the spatial position, speed, yaw angle, pitch angle, and roll angle of the robot.

[0076] Specifically, the robot acquires the spatial position and speed through the installed lidar and inertial measurement unit sensors. The accelerometer and gyroscope of the inertial measurement unit obtain its yaw angle, pitch angle, and roll angle; the spatial position of the robot is represented by and the speed is represented by and 、 and respectively represent the speeds on the x-axis, y-axis, and z-axis.

[0077] In an embodiment of the present invention, the first motion trend prediction adopts the Kalman filter algorithm to calculate the future position and future speed of the robot.

[0078] The Kalman filter algorithm is a state estimation algorithm used for motion prediction of dynamic systems. Based on the attitude information of the robot, the present invention predicts the future position and speed of the robot through the Kalman filter algorithm for subsequent path correction and adjustment of the motion trajectory of the robot's end effector to ensure that the robot can accurately dock with the charging hole and provide stable and reliable navigation and docking operations in the complex dynamic environment at sea.

[0079] In an embodiment of the present invention, the steps of generating the first correction instruction include:

[0080] S401, by comparing the future position of the robot and the position of the current charging hole, calculate the deviation distance, and the deviation distance is represented as , 、 and respectively represent the differences between the future positions of the robot and the current charging hole position in the x-direction, y-direction, and z-direction;

[0081] S402, generate a first correction instruction according to the deviation distance, and the first correction instruction includes the direction adjustment, speed adjustment, and angle adjustment of the end effector of the robot.

[0082] In an embodiment of the present invention, the master-slave communication protocol includes:

[0083] The master control unit sends a first correction instruction;

[0084] The robot control unit receives the first correction instruction from the master control unit, executes the specified operation, and when the distance between the end effector of the robot and the charging target area is less than the first preset threshold, makes fine-tuning actions autonomously according to the position of the end effector of the robot and the position of the charging target area;

[0085] Specifically, when the distance between the end effector of the robot and the charging target area is less than the first preset threshold, deviation detection is performed. When there is a position deviation, the robot control unit issues an adjustment instruction to make a fine displacement, such as moving 2 centimeters to the left or right.

[0086] In an embodiment of the present invention, S104 is used to generate a first correction instruction, and the first correction instruction is used to correct the motion trajectory of the robot to ensure that the robot can approach the target charging hole and complete the docking task. The first correction instruction is used as a large-range adjustment instruction, providing preliminary path correction and direction adjustment for the robot, ensuring that the robot will not deviate too far from the target when performing the docking task, and mainly used to guide the robot to approach the charging hole;

[0087] S105 transmits the first correction instruction from the master control unit to the robot control unit through the master-slave communication protocol. During this process, the robot control unit performs large-range adjustment according to the first correction instruction, and when the robot approaches the target, makes fine-tuning actions through real-time position feedback to ensure precise docking; the fine-tuning actions are finely adjusted through real-time feedback to ensure more accurate path correction of the robot.

[0088] It should be noted that the setting of the interval and threshold size is for the convenience of comparison. Among them, the size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameters and the quantified values. And the above formulas are all calculations of taking the numerical value after removing the dimension. The formulas are all obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0089] The embodiments of the present invention have been described above. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make many forms, all of which fall within the protection scope of this embodiment.

Claims

1. A communication control method for a marine charging robot based on visual navigation, characterized in that, It includes the following steps: S101, collect first-resolution image data through the vision sensor of the robot end effector, perform rough positioning processing on the target ship, and determine the initial position of the charging target area; S102, based on the initial position of the charging target area, perform first feature stability analysis, dynamically generate the first region of interest, and extract second-resolution image data based on the first region of interest for precise positioning processing; Among them, the steps for generating the first region of interest include: S201, within the charging target area, identify and extract feature points in the first-resolution image data, where the feature points include corner points and edge feature points; S202, perform continuous frame tracking on the feature points, calculate the average displacement of each feature point between consecutive frames, and the proportion of the number of consecutive frames in the total number of frames, and perform weighted calculation on the average displacement and the proportion based on a preset weight to obtain the feature point stability feature; S203, divide the charging target area into N sub-regions according to the positions of the feature points, N is a positive integer, and the sub-regions are equal-size grid regions; S204, based on the feature point stability feature within each sub-region, calculate the stability score of the sub-region, and use the sub-regions with stability scores greater than the second preset threshold as the first region of interest; The steps for precise positioning processing include: S301, based on the metal reflection characteristics and wave interference characteristics of the ship's charging hole, perform high-fidelity image extraction on the first region of interest to obtain second-resolution image data, where the high-fidelity image extraction is achieved through bicubic interpolation and gamma correction; S302, lock the preset marker contour of the charging hole hatch through an edge feature extraction method that suppresses dynamic noise, and construct a positioning operation area; S303, dynamically adjust the positioning operation area according to the average displacement of the feature points; S103, divide the first-resolution image data into a first region, a second region, and a third region, where: Apply the first compression parameter to the first region and give priority to transmission while maintaining the original resolution; Apply the second compression parameter to the second region for medium-compression transmission; Apply the third compression parameter to the third region for high-compression ratio transmission; S104, collect the pose information of the robot, perform first motion trend prediction processing, and generate a first correction instruction according to the prediction result of the first motion trend; S105, based on the master-slave communication protocol, the master control unit sends the first correction instruction to the robot control unit, and the robot control unit makes adjustments according to the first correction instruction and performs fine-tuning according to the real-time position.

2. The communication control method of the marine charging robot based on visual navigation according to claim 1, characterized in that, The rough positioning processing includes: Identify the preset optical positioning marker on the deck of the target ship, and the optical positioning marker is a high-contrast concentric circle; Calculate the initial position of the charging target area according to the geometric center coordinates of the optical positioning marker and the known installation position.

3. The communication control method of the marine charging robot based on visual navigation according to claim 1, characterized in that Set the positioning operation area in the second-resolution image data as the first region, and apply the first compression parameter to give priority to transmission while maintaining the original resolution; Set the part of the first region of interest in the second-resolution image data except the first region as the second region, apply the second compression parameter, and perform medium-compression transmission; Set the other parts of the first-resolution image data except the first region of interest as the third region, apply the third compression parameter, and perform high-compression ratio transmission.

4. The communication control method of the marine charging robot based on visual navigation according to claim 1, characterized in that The pose information includes: the spatial position, speed, yaw angle, pitch angle, and roll angle of the robot.

5. The communication control method of the marine charging robot based on visual navigation according to claim 1, characterized in that, The first motion trend prediction uses the Kalman filter algorithm to calculate the future position and future speed of the robot.

6. The communication control method of the marine charging robot based on visual navigation according to claim 1, wherein, The steps of generating the first correction instruction include: S401. Calculate the deviation distance by comparing the future position of the robot with the position of the current charging hole; S402. Generate the first correction instruction according to the deviation distance. The first correction instruction includes the direction adjustment, speed adjustment, and angle adjustment of the end effector of the robot.

7. The communication control method of the marine charging robot based on visual navigation according to claim 1, wherein The master-slave communication protocol includes: The master control unit sends the first correction instruction; The robot control unit receives the first correction instruction from the master control unit, executes the specified operation, and when the distance between the end effector of the robot and the charging target area is less than the first preset threshold, makes fine-tuning actions autonomously according to the position of the end effector of the robot and the position of the charging target area.

Images