A deep-sea heavy-duty operation robot capable of automatically avoiding obstacles and an obstacle avoidance method

Abstract

The present invention discloses a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles and an obstacle avoidance method. The robot includes: a data acquisition module; an image processing module; a stepping control module; a calculation and judgment module. The data acquisition module acquires sonar images and optical images in the moving direction of the robot. The image processing module respectively performs edge detection on the obstacles in the sonar images and optical images to obtain the edge pixel coordinates of the obstacles. The image processing module also selects the pixel regions driven by stepping for the sonar images and optical images. The calculation and judgment module determines whether to trigger stepping control according to the edge pixel coordinates of the obstacles. The stepping control module generates an action parameter matrix for stepping control and automatically generates parameter weights according to the obstacle depth and edge pixel coordinates of the sonar images and / or optical images. The robot executes corresponding obstacle avoidance strategies according to the action parameter matrix carrying the parameter weights.

Description

Technical Field

[0001] The present invention relates to the technical field of robot obstacle avoidance, and particularly to a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles and an obstacle avoidance method. Background Art

[0002] At present, with the development of robot technology, the obstacle avoidance technology of robots is also constantly updated and optimized. However, the existing obstacle avoidance technologies mainly include the following parts: 1. Path planning and preset routing based on a rule-based map. For example, traditional warehouse robots execute through preset path routing rules during routing planning. Path planning and preset routing based on a rule-based map cannot adapt to the dynamic changes of the environment. For example, underwater robots face an extremely complex environment underwater, making the technical solutions of path planning and preset routing based on a rule-based map less effective. 2. Using sensors such as sonar to detect obstacles and execute corresponding obstacle avoidance strategies. However, there are many technical problems with traditional sonar systems. For example, sonar is greatly affected by environmental interference. The water temperature and salinity stratification (such as the isothermal layer) will change the sound wave propagation path, resulting in detection blind spots or misjudgments. The complex noise on the seabed may also produce reflected interference noise effects on sonar obstacle detection, and sonar has a poor effect on detecting the edges of obstacles. 3. Obstacle avoidance methods based on land image detection. However, the above image detection methods cannot be applied to the marine environment in some cases because the marine environment may have water light absorption, various plankton, soluble or insoluble water pollutants, and suspended sandy particles, etc., resulting in poor imaging effects of visible light images of the marine water body, especially underwater visible light images at medium and long distances. Therefore, there are also relatively large technical problems with the obstacle avoidance technical solutions based on pure visible light image detection. Summary of the Invention

[0003] One of the invention objects of the present invention is to provide a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles. The robot includes an automatic obstacle avoidance method. The robot and the automatic obstacle avoidance method construct a step control algorithm. The step control algorithm is used to control the actions of the robot in the moving direction, and the step control algorithm is implemented based on the planar position and distance of the obstacle image in the moving direction of the robot as step constraint conditions, so that the robot performs step control movement under the condition of meeting the obstacle avoidance conditions, and gradually controls the robot to deviate from the obstacle through step control, thereby achieving efficient automatic obstacle avoidance of the obstacle. A functional mapping relationship is established between the step control algorithm and the power of the rotation and movement driving devices of the corresponding robot, so that the robot can adaptively adjust its own step power in different obstacle space scenarios to meet different efficiency obstacle avoidance requirements.

[0004] Another object of the present invention is to provide a deep - sea heavy - duty operation robot capable of automatically avoiding obstacles. The robot includes an automatic obstacle - avoidance method. The robot and the automatic obstacle - avoidance method provide obstacle position detection by combining sonar images and optical images. The detection direction is the moving direction of the robot. For the obstacle image information obtained in the moving direction, image edge detection is performed. The distance is calculated using the pixel point positions of the obstacle image edge and the center point position of the image in the moving direction of the robot, and the obstacle - avoidance action of the step - by - step algorithm is performed in combination with the depth information of the obstacle. The image edge detection adaptively matches the sonar image and the optical image according to the integrity of the edge features, thereby improving the robustness of the robot's automatic obstacle avoidance and reducing the invalidity of the obstacle - avoidance action caused by environmental changes.

[0005] Another object of the present invention is to provide a deep - sea heavy - duty operation robot capable of automatically avoiding obstacles. The robot includes an automatic obstacle - avoidance method. The robot and the automatic obstacle - avoidance method preset an obstacle - avoidance step - drive pixel area on the forward image, calculate whether there are edge pixel points included in the step - drive pixel area based on an edge - detection algorithm including sonar images and optical images, and automatically generate a step - control matrix including corresponding drive devices according to the edge - pixel coordinates of the edge pixel points closest to the center - point pixel of the step - drive pixel area. The robot is driven according to the step - control matrix to perform corresponding obstacle - avoidance rotation actions to achieve the obstacle - avoidance effect.

[0006] In order to achieve at least one of the above - mentioned invention objects, the present invention further provides a deep - sea heavy - duty operation robot capable of automatically avoiding obstacles. The robot includes:

[0007] A data acquisition module;

[0008] An image - processing module;

[0009] A step - control module;

[0010] A calculation and judgment module;

[0011] The data acquisition module acquires sonar images and optical images in the moving direction of the robot. The image - processing module performs edge detection on the obstacles in the sonar images and optical images respectively to obtain the edge - pixel coordinates of the obstacles. The image - processing module also selects a step - drive pixel area for the sonar images and optical images. The calculation and judgment module judges whether to trigger step control according to the edge - pixel coordinates of the obstacles. The step - control module generates an action - parameter matrix for step control and automatically generates parameter weights according to the obstacle depth and edge - pixel coordinates of the sonar image and / or optical image. The robot executes corresponding obstacle - avoidance strategies according to the action - parameter matrix carrying parameter weights.

[0012] According to one preferred embodiment of the present invention, the method executed by the image processing module includes: after obtaining the optical image and sonar image in the moving direction of the robot, determining the center point coordinates of the images, and taking the area with the center point coordinates of the images as the coordinate origin and a radius of r as the stepping drive pixel area; the image processing module calls an edge detection operator to perform edge detection on the obstacles in the optical image and sonar image, and judges the continuity of the edge pixels of the optical image through a calculation and judgment module. If the edge pixels of the optical image are continuous, calculate the distances between the coordinate positions of all the edge pixels of the optical image and the center point coordinates, and judge whether the target optical image edge pixel point with the smallest distance is in the stepping drive pixel area. If so, generate an action parameter matrix for the stepping control according to the coordinates of the target optical image edge pixel point with the smallest distance, the position of the obstacle body in the coordinate system, and the obstacle depth data.

[0013] According to another preferred embodiment of the present invention, the method executed by the image processing module includes: if the current optical image edge pixels are discontinuous after edge detection, switch to detecting the obstacle edge features of the sonar image, calculate the average value of the coordinates of all the edge pixels of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculate the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judge whether the midpoint coordinates are in the stepping drive pixel area through the calculation and judgment module. If so, automatically generate an action parameter matrix for the stepping control according to the midpoint coordinates, the position of the obstacle body in the coordinate system, and the obstacle depth data.

[0014] According to another preferred embodiment of the present invention, the method executed by the stepping control module includes: taking the coordinates of the target optical image edge pixel point with the minimum value, the numerical values of different coordinate axes of the midpoint coordinates of the sonar image, and the position of the obstacle body in the coordinate system as the first input parameters of the deflection drive device in the corresponding direction of the robot, and taking the detected depth value of the obstacle image as the second input parameter of the deflection drive device in the corresponding direction of the robot, generating the parameter weights of the corresponding drive device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weights and the action parameters of the corresponding drive device to obtain an action parameter matrix carrying the parameter weights, automatically selecting the control time step, and performing obstacle avoidance stepping control on the robot according to the action parameter matrix carrying the parameter weights.

[0015] According to another preferred embodiment of the present invention, the method executed by the calculation and judgment module includes: selecting n edge detection regions, calculating whether each pixel point of the edge in the edge detection regions is connected to the surrounding pixels, and when the corresponding edge pixel point is only connected to one pixel point, determining the corresponding edge pixel point as a break point, calculating the number of break points of all edge pixel points in the edge detection regions, if the number of break points of all edge pixel points is greater than a preset break point threshold, determining that the pixels in the current edge detection region are discontinuous, and if at least 1 / 3 of the edge detection regions among the selected n edge detection regions are discontinuous, automatically switching to obtain the obstacle edge features of the sonar image.

[0016] According to another preferred embodiment of the present invention, after completing one obstacle avoidance step control of the robot, the data acquisition module re-acquires the sonar image and the optical image in the moving direction of the robot, and the image processing module re-uses the image processing model to perform edge detection on the sonar image and the optical image respectively, and performs depth detection of the obstacles in the images, and performs new step control according to the position relationship between the edge pixel coordinates of the re-detected obstacles, the position of the obstacle main body in the coordinate system, and the position of the step driving pixel region, and the depth of the obstacles until the edge pixel coordinates of the obstacles are not in the step driving pixel region.

[0017] According to another preferred embodiment of the present invention, the execution of the calculation and judgment module includes: setting a first depth threshold, if the image processing module does not detect the edge pixel coordinates of the obstacles in both the optical image and the sonar image, calculating the depth data of the corresponding obstacles in the optical image or the sonar image in real time, and when the depth data is less than the first depth threshold, the step control module takes the inverse of all the numerical values of the action parameter matrix of the current step control to obtain a step control strategy for reverse driving.

[0018] In order to achieve at least one of the above invention purposes, the present invention further provides a method for automatic obstacle avoidance of a deep-sea heavy-duty operation robot, and the method includes:

[0019] Obtaining the optical image and the sonar image in the moving direction of the robot, and performing obstacle recognition on the optical image and the sonar image, using an edge detection algorithm to perform edge detection on the obstacles in the optical image and the sonar image respectively to obtain the corresponding obstacle edge pixel coordinates;

[0020] Selecting a step driving pixel region for the optical image and the sonar image, and obtaining the depth information of the obstacles relative to the optical image and the sonar image;

[0021] Determine whether to trigger step control based on the coordinates of the obstacle edge pixel points, the depth information of the obstacle, and the step drive pixel area, and automatically generate an action parameter matrix for the robot step control according to the triggered step control included;

[0022] Automatically generate a weight parameter for the corresponding action parameter according to the coordinates of the obstacle edge pixel points, the depth information of the obstacle, and the step drive pixel area, and perform weighted processing on the weight parameter and the corresponding action parameter;

[0023] Use the action parameter matrix of the step control carrying the weight parameter as an obstacle avoidance strategy to perform automatic obstacle avoidance control of the machine.

[0024] According to one preferred embodiment of the present invention, the method for selecting the step drive pixel area includes: after obtaining the optical image and sonar image in the moving direction of the robot, determining the center point coordinates of the image, and using the area with the center point coordinates of the image as the coordinate origin and a radius of r as the step drive pixel area.

[0025] According to one preferred embodiment of the present invention, the method for step control includes: calling an edge detection operator to perform edge detection on the obstacles in the optical image and sonar image, calculating and judging the continuity and integrity of the edge pixels of the optical image. If the edge pixels of the optical image are continuous and complete, calculate the distance between the coordinate positions of all the edge pixels of the optical image and the center point coordinates, and judge whether the target optical image edge pixel point with the smallest distance is in the step drive pixel area. If so, generate the action parameter matrix for the step control according to the coordinates of the target optical image edge pixel point with the smallest distance, the position of the obstacle body in the coordinate system, and the obstacle depth data.

[0026] According to another preferred embodiment of the present invention, the method for step control further includes: if the obstacle edge pixels of the current optical image after edge detection are not continuous, switch to detect the obstacle edge features of the sonar image, calculate the average value of the coordinates of all the edge pixels of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculate the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judge whether the midpoint coordinates are within the step drive pixel area. If so, automatically generate the action parameter matrix for the step control according to the midpoint coordinates and the position of the obstacle body in the coordinate system.

[0027] According to another preferred embodiment of the present invention, the method for detecting the continuity of the obstacle edge pixels of the optical image includes:

[0028] Select n edge detection regions, calculate whether each pixel of the edge and its surrounding pixels are connected in the edge detection regions. When a corresponding edge pixel is only connected to one pixel, the corresponding edge pixel is determined as a break point. Calculate the number of break points of all edge pixels in the edge detection regions. If the number of break points of all edge pixels is greater than a preset break point threshold, it is determined that the pixels in the current edge detection region are discontinuous. If at least 1 / 3 of the edge detection regions among the selected n edge detection regions are discontinuous, it is determined that the obstacle edge pixels of the current optical image are overall discontinuous, and automatically switch to obtain the obstacle edge features of the sonar image.

[0029] According to another preferred embodiment of the present invention, the automatic obstacle avoidance method includes: using the different coordinate axis values of the target optical image edge pixel point coordinates or the sonar image point coordinates of the minimum value as the first input parameter of the corresponding direction deflection driving device of the robot, and using the corresponding detected obstacle image depth value as the second input parameter of the corresponding direction deflection driving device of the robot, generating the parameter weights of the corresponding driving device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weights and the action parameters of the corresponding driving device to obtain an action parameter matrix carrying parameter weights, automatically selecting a control time step, and performing obstacle avoidance step control of the robot according to the action parameter matrix carrying parameter weights.

[0030] The present invention further provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement any one of the above-mentioned obstacle avoidance methods for a deep-sea heavy-duty operation robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Shown is a module schematic diagram of a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to the present invention.

[0032] Figure 2 Shown is a schematic flow chart of an automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to the present invention.

[0033] Figure 3 Shown is a schematic structural diagram of a step pixel driving region according to the present invention; wherein, 1 - pixel plane of the image, 2 - obstacle edge, P3 - step driving pixel region. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and other obvious variations can be conceived by those skilled in the art. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalent solutions, and other technical solutions without departing from the spirit and scope of the present invention.

[0035] It can be understood that the term "a" should be construed as "at least one" or "one or more". That is, in one embodiment, the number of an element can be one, while in other embodiments, the number of the element can be multiple. The term "a" should not be construed as a limitation on the number.

[0036] Please refer to Figures 1 to 3, the present invention discloses a deep - sea heavy - duty operation robot capable of automatically avoiding obstacles and a method for the robot to automatically avoid obstacles. The robot mainly includes the following core modules: a data acquisition module; an image processing module; a step control module and a calculation and judgment module. Among them, the data acquisition module includes a visible - light sensor, a sonar transceiver device and an image conversion module. The image sensor is used to collect visible light in the moving direction of the robot and convert it into a visible - light image through the image conversion module. The sonar transceiver device is used to collect sonar information in the moving direction of the robot and convert the sonar information into a sonar image through the image conversion module. The image processing module can call relevant target recognition algorithms to identify target obstacles in the visible - light image, and the image processing module can also call an edge - detection algorithm to detect the edge pixel points of the target obstacle. Since the target - detection algorithm and the edge - detection algorithm are both mature technologies in visible - light image processing, the present invention will not elaborate on which detection algorithm to choose. The present invention needs to use the image processing module to set pixel coordinate systems on the visible - light image and the sonar image respectively. Each pixel point in the pixel coordinate system represents a coordinate value. Use the pixel coordinate system to calculate the coordinate values of the edge pixel points of the target obstacle, and establish a step - drive pixel area on the visible - light image and the sonar image according to the pixel coordinate system. The step - drive pixel area is used to detect whether the edge pixel points of the corresponding obstacle are in the forward direction of the robot, and the position of the edge pixel points of the obstacle in the step - drive pixel area can be used as key reference data for the robot's obstacle - avoidance processing to construct an action - parameter matrix related to the robot, so that the robot can deflect along a specific direction of the edge pixel points, thereby autonomously completing the obstacle - avoidance action. It should be noted that due to the complex marine environment, visible - light images may have inapplicable problems in special marine environments. For example, in a relatively turbid seawater environment, the visible range of visible - light images is usually very short, and there may be problems such as the inability to detect edge pixels of the corresponding obstacle target or many breakpoints in the edge pixel points, resulting in discontinuity. And sonar has stronger anti - interference ability for water - body images than visible light. Therefore, it is necessary to automatically switch the obstacle images in a specific water - body environment and optimize the coordinate values of relevant edge pixel points at the same time. Thus, the present invention can be adapted to automatic obstacle avoidance in different marine environments.

[0037] Specifically, the present invention uses the image processing module to perform obstacle recognition on the visible - light image and the sonar image and perform edge detection respectively to obtain the following coordinate values A of the edge pixel points of the target obstacle n =(x n , y n ) and B n =(x n , y n ), where An Denote the edge pixel points of the visible light image as B n Denote the edge pixel points of the sonar image as n, representing the identification sequence of the corresponding edge pixel points. Let x n and y n be the pixel coordinates on the visible light image or sonar image respectively. It should be noted that in the present invention, the central pixel point of the visible light image or sonar image is taken as the origin O of the pixel coordinates, the preset horizontal direction of the image is taken as the x-axis, the direction perpendicular to the x-axis in the plane of the image is taken as the y-axis, and each pixel point is taken as the unit coordinate value to obtain the edge pixel coordinates A n =(x n , y n ) and B n =(x n , y n ). Further, in the present invention, a step drive pixel region P3 is constructed with the origin O as the center and a radius r, where P3 = (x j , y j ), and r 2 ≥(x j 2 +y j 2 ). When the optical image of the target obstacle has edge pixel coordinates A n ∈P3, the image processing module automatically generates an action parameter matrix E = [e1, e2, e3, e4, e5, e6... e n for step control, where e represents the action parameters of the corresponding drive devices of the robot. The action parameters include action power and direction, and different subscripts represent different drive devices. The devices represented by the action parameters may include but are not limited to propellers, pump jets, vector thrusters, steering gears, servo motors, rotary encoders, harmonic reducers, etc., which are execution devices involved in the actions of the robot. According to the configuration of the robot action devices, at least one core parameter of each configured device needs to be used as the above action parameters. The present invention only gives examples of the above action devices and will not elaborate on this in detail.

[0038] It should be noted that since the installation positions of different action devices are different and the action results of the corresponding action devices are different, the present invention needs to distinguish the installation positions and action capabilities of the above-mentioned different action devices in advance, and further distinguish the main action functions of the action parameters of the corresponding action devices. For example, the installation position of the above-mentioned propeller is on the left side of the robot body, so that the core action parameters of the propeller, such as the propeller power and the forward rotation speed, can boost the robot to realize the function of rotating to the right. It should be noted that since the action devices, performances, and positions installed on different robots are different, the action parameter matrices generated by different robots are also different. The present invention only gives examples. In addition, due to the interaction of forces, the actions of the robot underwater are not only obstacle avoidance functions. For example, the robot is also operating while avoiding obstacles. Therefore, the present invention needs to obtain the action parameters of all the action devices of the robot and comprehensively consider the influence of the action parameters of all the action devices on the robot's heading. The above influence can be obtained through weighted processing of different action parameters.

[0039] It is worth mentioning that in the present invention, it is necessary to detect the continuity of the edge pixel points of the target obstacle image in the visible light image and set relevant thresholds for the continuity of the edge pixel points of the obstacle in the visible light image. When the continuity of the edge pixel points of the obstacle in the visible light image does not meet the threshold requirements, it automatically switches to the edge pixel points of the target obstacle in the sonar image to calculate the relevant obstacle avoidance action parameter matrix. The method for judging the continuity of the edge pixel points of the obstacle in the visible light image includes: dividing the edge pixel points of the target obstacle image in the visible light image into n edge detection areas with the same area size, and calculating whether there are connected pixel points for each edge pixel point in each detection area among the n edge detection areas. The connected pixel points refer to whether there are edge pixel points above, below, left, and right of this pixel point (4-connected), or whether there are edge pixel points above, below, left, right, and at the four vertices (8-connected). If there is only one connected pixel point for this pixel point, it means that this pixel point is an end point, that is, this pixel point is a break point of the edge pixel point. The present invention needs to calculate the number or proportion value of the break points in the edge detection area. When the number or proportion value of the break points in the edge detection area is greater than a certain threshold, this detection area is determined as a discontinuous area, and further calculate the number of the areas determined as discontinuous areas among all the edge detection areas. When the number of the discontinuous areas is greater than one-third of the total number of the edge detection areas, it can be determined that the edge pixel points of the current target obstacle are discontinuous in the visible light image, so the edge pixel points detected by the visible light image are unavailable. Then it switches to the sonar image to obtain the edge pixel point coordinates of the corresponding obstacle.

[0040] When the edge pixel points of the target obstacle image in the visible light image are determined to be continuous, the following operations are further performed: Calculate the distances between the coordinate positions of all the edge pixels of the optical image and the center point coordinate, and determine whether the target optical image edge pixel point with the minimum distance is within the stepping drive pixel area. If so, generate the action parameter matrix for the stepping control according to the coordinate of the target optical image edge pixel point with the minimum distance, the position of the obstacle main body in the coordinate system, and the obstacle depth data. For example, the specific method is as follows: Define the target optical image edge pixel point with the minimum distance from the current distance of the target obstacle to the center point coordinate as a m =(x m , y m ), and the current coordinate of the target obstacle main body is P i =(x i , y i ). The coordinate of the target obstacle main body can be obtained by calculating, including but not limited to, the center point coordinate of the obstacle body. At this time, the coordinate vector i from the coordinate of the target obstacle main body P m to the edge pixel point a of the optical image with the minimum distance can be calculated. If the depth of the current target obstacle image is h, further perform the following weight parameter conversion on the vector feature of the coordinate vector : , where λ represents the normalization coefficient, , , and and respectively represent the weight coefficients in the x-axis direction and y-axis direction in the optical image pixel coordinate system, which are used for weighted processing of the action parameters of the corresponding action device. It should be noted that the weight coefficients in the above different directions represent the directionality, action intensity of the corresponding action parameters, and the numerical correlation between the corresponding coordinate vector and the depth information h. When the coordinate vector is positive in the x-axis direction, it means that the action parameter of the relevant action device needs to perform a rightward action. When the coordinate vector is negative in the x-axis direction, it means that the action parameter of the relevant action device needs to perform a leftward action. When the value of h is lower, it means that the obstacle is closer to the robot, and the weight coefficient value in the x-axis direction is higher. After multiplying by the corresponding action parameter, a larger deflection needs to be performed at a specific time step. The larger deflection can be reflected in the drive power and rotation angle of the corresponding action device. The present invention will not elaborate on this in detail. Similarly, in the y-axis direction, when the coordinate vector is positive in the y-axis direction, it means that the action parameter of the relevant action device needs to perform an upward action. When the coordinate vector When the value in the y-axis direction is negative, it indicates that the action parameters for driving the relevant action devices need to perform downward actions. The depth information h is also the core parameter for the directionality and action intensity of the corresponding action parameters.

[0041] It should be noted that the above action parameter matrix E = [e1, e2, e3, e4, e5, e6... e n and the weighted product of the weight parameters , need to be classified and processed according to the action direction of the action parameters. Some of them may be incorporated into the attitude resolution algorithms including pitch angle, roll angle, and heading angle. The above attitude resolution algorithms can be calculated by devices such as accelerometers, so as to obtain the action parameter matrix carrying parameter weights.

[0042] In another preferred embodiment of the present invention, the present invention can also borrow an AI model to generate and process the above action parameter matrix carrying parameter weights. For example, it can borrow regression prediction models including but not limited to CNN or DNN models to predict the corresponding action parameter matrix E, and input the prediction results into the corresponding action devices of the robot to execute the obstacle avoidance strategy. The present invention will not elaborate on this in detail.

[0043] Furthermore, if the obstacle edge pixels of the current optical image after edge detection are discontinuous, then switch to detecting the obstacle edge features of the sonar image, calculate the average value of the coordinates of all edge pixels of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculate the midpoint coordinate of the straight-line connection between the average value coordinate point and the center point of the image, and determine whether the midpoint coordinate is within the stepping drive pixel area. If so, automatically generate the action parameter matrix for the stepping control according to the midpoint coordinate and the position of the obstacle main body in the coordinate system. Taking the edge display coordinate B n = (x n , y n ) of the sonar image as an example, define the midpoint coordinate of the straight-line connection between the average value coordinate point of the edge pixels of the sonar image and the center point of the image as B s = (x s , y s ), and calculate the target obstacle main body coordinate as P i = (x i , y i ), further calculate the vector from the target obstacle main body coordinate P i = (x i , y i ) to the midpoint coordinate B s = (x s , y s ) , further calculation yields the following weight parameters and , where λ represents the normalization coefficient, and respectively represent the weight coefficients in the x-axis direction and y-axis direction in the pixel coordinate system of the optical image. h is the depth of the obstacle in the sonar image, and the depth of the obstacle in the sonar image can be directly detected by the sonar. The action parameter matrix with the corresponding carried parameter weights is obtained according to the same calculation method as that of the visible light image.

[0044] In another preferred embodiment of the present invention, if the edge pixel points of the obstacle cannot be detected, the following obstacle avoidance rule is executed: The calculation and judgment module executes the following: Set a first depth threshold. If the image processing module does not detect the edge pixel point coordinates of the obstacle in both the optical image and the sonar image, the depth data of the corresponding obstacle in the optical image or the sonar image is calculated in real time. When the depth data is less than the first depth threshold, the stepping control module takes the inverse of all the numerical values of the current action parameter matrix of the stepping control to obtain a stepping control strategy for reverse driving. In this embodiment, considering that a particularly huge obstacle may cover the entire image interface, only the depth information of the obstacle can be considered as the action parameter for reverse driving, so as to drive away from or deflect in the direction of the obstacle until the edge pixel points of the obstacle are found as a new obstacle avoidance strategy.

[0045] Embodiments disclosed by the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. Embodiments disclosed by the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. And in the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless segments, wire segments, optical cables, RF, etc., or any suitable combination of the above.

[0046] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0047] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the said principles, the embodiments of the present invention may have any variations or modifications.

Claims

1. An automatic obstacle avoidance deep-sea heavy-duty operation robot, characterized in that, The robot includes: A data acquisition module; An image processing module; A stepping control module; A calculation and judgment module; Wherein the data acquisition module acquires sonar images and optical images in the moving direction of the robot; the image processing module respectively performs edge detection on the obstacles in the sonar images and optical images to obtain the edge pixel coordinates of the obstacles, and the image processing module also selects the pixel area for stepping drive for the sonar images and optical images; the calculation and judgment module judges whether to trigger stepping control according to the edge pixel coordinates of the obstacles; the stepping control module generates an action parameter matrix for stepping control, and automatically generates parameter weights according to the obstacle depth and edge pixel coordinates of the sonar image and / or optical image, and the robot executes the corresponding obstacle avoidance strategy according to the action parameter matrix carrying the parameter weights; The method executed by the image processing module includes: after acquiring the optical image and sonar image in the moving direction of the robot, determining the center point coordinates of the image, and taking the area with the center point coordinates of the image as the coordinate origin and a radius of r as the pixel area for stepping drive; the image processing module calls an edge detection operator to perform edge detection on the obstacles in the optical image and sonar image, and judges the continuity of the edge pixels of the optical image through the calculation and judgment module. If the edge pixels of the optical image are continuous, calculate the distance between the coordinate positions of all the edge pixels of the optical image and the center point coordinates, and judge whether the target optical image edge pixel point with the minimum distance is in the pixel area for stepping drive. If so, generate the action parameter matrix for stepping control according to the coordinates of the target optical image edge pixel point with the minimum distance, the position of the obstacle body in the coordinate system, and the obstacle depth data.

2. The deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 1, wherein, The method executed by the image processing module includes: if the edge pixels of the current optical image after edge detection are not continuous, switch to detecting the edge features of the obstacles in the sonar image, calculate the average value of the coordinates of all the edge pixels of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculate the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judge whether the midpoint coordinates are in the pixel area for stepping drive through the calculation and judgment module. If so, automatically generate the action parameter matrix for stepping control according to the midpoint coordinates, the position of the obstacle body in the coordinate system, and the obstacle depth data.

3. The deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 2, wherein, The method for the step control module to execute includes: using the numerical values of different coordinate axes of the target optical image edge pixel coordinates or the sonar image midpoint coordinates of the minimum value as the first input parameter of the robot's corresponding direction deflection driving device, using the detected depth value of the obstacle image as the second input parameter of the robot's corresponding direction deflection driving device, generating the parameter weights of the corresponding driving device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weights and the action parameters of the corresponding driving device to obtain an action parameter matrix carrying the parameter weights, automatically selecting the control time step, and performing obstacle avoidance step control on the robot according to the action parameter matrix carrying the parameter weights.

4. A deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 1, characterized in that, The method for the calculation and judgment module to execute includes: selecting n edge detection regions, calculating whether each pixel point on the edge in the edge detection region is connected to the surrounding pixels. When the corresponding edge pixel point is only connected to one pixel point, the corresponding edge pixel point is determined as a break point, calculating the number of break points of all edge pixel points in the edge detection region. If the number of break points of all edge pixel points is greater than the preset break point threshold, it is determined that the pixels in the current edge detection region are discontinuous. If at least 1 / 3 of the edge detection regions among the selected n edge detection regions are discontinuous, the obstacle edge feature of the sonar image is automatically switched to be obtained; the calculation and judgment module execution includes: setting a first depth threshold. If the image processing module does not detect the edge pixel coordinates of the obstacle in both the optical image and the sonar image, the depth data of the corresponding obstacle in the optical image or the sonar image is calculated in real time. When the depth data is less than the first depth threshold, the step control module performs an inversion process on all the numerical values of the action parameter matrix of the current step control to obtain a step control strategy for reverse driving.

5. An automatic obstacle avoidance method for a deep - sea heavy - duty operation robot, characterized in that, The method includes: Obtaining the optical image and the sonar image in the moving direction of the robot, performing obstacle recognition on the optical image and the sonar image, and using the edge detection algorithm to perform edge detection on the obstacles in the optical image and the sonar image respectively to obtain the corresponding obstacle edge pixel coordinates; Selecting the step driving pixel regions for the optical image and the sonar image, and obtaining the depth information of the obstacle relative to the optical image and the sonar image; Judging whether to trigger step control according to the obstacle edge pixel coordinates, the depth information of the obstacle, and the step driving pixel regions, and automatically generating an action parameter matrix for the robot step control according to the triggered step control; Automatically generating the weight parameters of the corresponding action parameters according to the obstacle edge pixel coordinates, the depth information of the obstacle, and the step driving pixel regions, and performing weighted processing on the weight parameters and the corresponding action parameters; Using the action parameter matrix of the step control carrying the weight parameters as an obstacle avoidance strategy to perform automatic obstacle avoidance control on the machine; The method for selecting the stepping drive pixel area includes: after obtaining the optical image and sonar image in the moving direction of the robot, determining the center point coordinates of the image, and taking the area with the center point coordinates of the image as the coordinate origin and a radius of r as the stepping drive pixel area; The method for stepping control includes: calling an edge detection operator to perform edge detection on the obstacles in the optical image and sonar image, calculating and judging the continuity and integrity of the edge pixels of the optical image. If the edge pixels of the optical image are continuous and complete, then calculate the distances between the coordinate positions of all the edge pixels of the optical image and the center point coordinates, and judge whether the target optical image edge pixel point with the smallest distance is in the stepping drive pixel area. If so, generate the action parameter matrix for the stepping control according to the coordinates of the target optical image edge pixel point with the smallest distance and the obstacle depth data.

6. The automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 5, characterized in that The method for stepping control further includes: if the obstacle edge pixels of the current optical image after edge detection are not continuous, then switch to detect the obstacle edge features of the sonar image, calculate the average value of the coordinates of all the edge pixels of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculate the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judge whether the midpoint coordinates are within the stepping drive pixel area. If so, automatically generate the action parameter matrix for the stepping control according to the midpoint coordinates.

7. An automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 5, characterized in that, The method for detecting the continuity of the obstacle edge pixels of the optical image includes: Select n edge detection regions, calculate whether each pixel point on the edge is connected to the surrounding pixels in the edge detection region. When the corresponding edge pixel point is only connected to one pixel point, the corresponding edge pixel point is determined as a break point. Calculate the number of break points of all the edge pixel points in the edge detection region. If the number of break points of all the edge pixel points is greater than the preset break point threshold, then it is determined that the pixels in the current edge detection region are not continuous. If at least 1 / 3 of the edge detection regions among the selected n edge detection regions are not continuous, then it is determined that the obstacle edge pixels of the current optical image are not continuous as a whole, and automatically switch to obtain the obstacle edge features of the sonar image.

8. An automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 5, characterized in that The automatic obstacle avoidance method includes: using the different axis values of the target optical image edge pixel point coordinates or the sonar image midpoint coordinates with the minimum value as the first input parameter of the deflection drive device in the corresponding direction of the robot, and using the corresponding detected depth value of the obstacle image as the second input parameter of the deflection drive device in the corresponding direction of the robot. Generate the parameter weights of the corresponding drive device according to the first input parameter and the second input parameter, perform weighted calculation on the parameter weights and the action parameters of the corresponding drive device to obtain the action parameter matrix carrying the parameter weights, automatically select the control time step, and perform obstacle avoidance stepping control on the robot according to the action parameter matrix carrying the parameter weights.

Images