Deep sea heavy work robot capable of automatically avoiding obstacles and obstacle avoiding method

Abstract

The invention discloses a deep sea heavy work robot capable of automatically avoiding obstacles and an obstacle avoidance method. The robot comprises a data acquisition module; an image processing module; a stepping control module; a calculation judgment module; wherein the data acquisition module acquires a sonar image and an optical image in the moving direction of the robot; the image processing module performs edge detection on obstacles in the sonar image and the optical image to obtain edge pixel coordinates of the obstacles, and the image processing module also performs step driving pixel area selection on the sonar image and the optical image; the calculation and judgment module judges whether stepping control is triggered or not according to the edge pixel coordinates of the obstacle; the stepping control module generates an action parameter matrix of stepping control and automatically generates parameter weights according to the obstacle depth and edge pixel coordinates of the sonar image and / or the optical image, and the robot executes a corresponding obstacle avoidance strategy according to the action parameter matrix carrying the parameter weights.

Description

Technical Field

[0001] The present invention relates to the technical field of robot obstacle avoidance, and in particular to a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles and an obstacle avoidance method. Background Art

[0002] At present, with the development of robot technology, the obstacle avoidance technology of robots is also constantly updated and optimized. However, the existing obstacle avoidance technology mainly includes the following parts: 1. Path planning and preset routing based on rule maps. For example, traditional warehouse robots will execute through preset path routing rules when planning routes. Path planning and preset routing based on rule maps cannot adapt to the dynamic changes of the environment. For example, underwater robots face extremely complex environments underwater, which makes the path planning and preset routing technical solutions based on rule maps less effective. 2. Use sensors such as sonar to detect obstacles and execute corresponding obstacle avoidance strategies. However, there are many technical problems in traditional sonar systems. For example, sonar is greatly affected by environmental interference, water temperature and salinity stratification (such as isothermal layer) will change the propagation path of sound waves, resulting in detection blind spots or misjudgments, and complex seabed noise may also produce reflection interference noise on sonar obstacle detection, and sonar has poor detection effect on obstacle edges. 3. Obstacle avoidance method based on land image detection. However, the above image detection method is not applicable to the marine environment in some cases, because the marine environment may have water light absorption, various plankton, soluble or insoluble pollutants, and suspended sand particles, which result in poor imaging effects of visible light images of marine water bodies, especially medium and long-distance underwater visible light images; therefore, obstacle avoidance technology solutions based on simple visible light image detection also have major technical problems. Summary of the invention

[0003] One of the inventive purposes of the present invention is to provide a deep-sea heavy-duty operation robot that can automatically avoid obstacles, the robot includes an automatic obstacle avoidance method, wherein the robot and the automatic obstacle avoidance method construct a stepping control algorithm, wherein the stepping control algorithm is used to control the movement of the robot in the moving direction, and the stepping control algorithm is implemented based on the planar position and distance of the obstacle image of the obstacle in the moving direction of the robot as stepping constraints, so that the robot can perform step-controlled movement of the robot under the conditions of meeting the obstacle avoidance, and the robot is gradually controlled to deviate from the obstacle through stepping control, so as to automatically avoid the obstacle efficiently; wherein a functional mapping relationship is established between the stepping control algorithm and the power of the corresponding robot's rotation and movement drive device, so that the robot can adaptively adjust its own stepping power in different obstacle space scenarios to cope with different efficiency obstacle avoidance requirements.

[0004] Another invention object of the present invention is to provide a deep-sea heavy-duty operation robot that can automatically avoid obstacles, the robot includes an automatic obstacle avoidance method, wherein the robot and the automatic obstacle avoidance method provide obstacle position detection combined with sonar images and optical images, wherein the detection direction is the movement direction of the robot, and the obstacle image information obtained in the movement direction includes image edge detection, using the pixel point position of the obstacle image edge and the center point position of the image in the robot movement direction to calculate the distance, and combining the depth information of the obstacle to perform the obstacle avoidance action of the step algorithm. The image edge detection will perform adaptive matching of the sonar image and the optical image according to the completeness of the edge features, thereby improving the robustness of the robot's automatic obstacle avoidance and reducing the invalidity of the obstacle avoidance action due to environmental changes.

[0005] Another inventive object of the present invention is to provide a deep-sea heavy-duty operation robot that can automatically avoid obstacles, the robot including an automatic obstacle avoidance method, wherein the robot and the automatic obstacle avoidance method preset an obstacle avoidance stepping drive pixel area on the forward image, and calculate whether there are edge pixels contained in the stepping drive pixel area based on an edge detection algorithm including sonar images and optical images, and automatically generate a stepping control matrix including a corresponding driving device based on the edge pixel point coordinate value of the pixel closest to the center point of the stepping drive pixel area, and drive the robot to perform a corresponding obstacle avoidance rotation action according to the stepping control matrix to achieve an obstacle avoidance effect.

[0006] In order to achieve at least one of the above-mentioned invention objects, the present invention further provides a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles, the robot comprising: Data acquisition module; Image processing module; Stepper control module; Calculation and judgment module; The data acquisition module collects sonar images and optical images in the direction of robot movement; the image processing module performs edge detection on obstacles in the sonar image and optical image respectively to obtain edge pixel coordinates of obstacles, wherein the image processing module also selects step-drive pixel areas for the sonar image and optical image; the calculation and judgment module determines whether to trigger step control according to the edge pixel coordinates of the obstacle; the step control module generates an action parameter matrix for step control, and automatically generates parameter weights according to the obstacle depth and edge pixel coordinates of the sonar image and / or optical image, and the robot executes a corresponding obstacle avoidance strategy according to the action parameter matrix carrying the parameter weights.

[0007] According to one of the preferred embodiments of the present invention, the execution method of the image processing module includes: after obtaining the optical image and sonar image in the moving direction of the robot, determining the coordinates of the center point of the image, and taking the coordinates of the center point of the image as the coordinate origin and the area with r as the radius as the stepping drive pixel area; the image processing module calls the edge detection operator to perform edge detection on the obstacles in the optical image and the sonar image, and judges the continuity of the edge pixels of the optical image through the calculation and judgment module. If the edge pixels of the optical image are continuous, the distance between the coordinate positions of all the edge pixels of the optical image and the coordinates of the center point is calculated, and it is judged whether the target optical image edge pixel point with the smallest distance is in the stepping drive pixel area. If so, the action parameter matrix of the stepping control is generated according to the coordinates of the target optical image edge pixel point with the smallest distance, the coordinate system position of the obstacle body and the obstacle depth data.

[0008] According to another preferred embodiment of the present invention, the image processing module execution method includes: if the edge pixels of the current optical image after edge detection are discontinuous, switching to detect the obstacle edge features of the sonar image, and calculating the average value of all edge pixel coordinates of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculating the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judging whether the midpoint coordinates are within the stepping drive pixel area through the calculation and judgment module, and if so, automatically generating the stepping control action parameter matrix according to the midpoint coordinates, the coordinate system position of the obstacle body and the obstacle depth data.

[0009] According to another preferred embodiment of the present invention, the execution method of the stepping control module includes: using the minimum value of the edge pixel point coordinates of the target optical image, the different coordinate axis values ​​of the midpoint coordinates of the sonar image, and the coordinate system position of the obstacle body as the first input parameter of the corresponding direction deflection driving device of the robot, and using the corresponding detected obstacle image depth value as the second input parameter of the corresponding direction deflection driving device of the robot, and generating the parameter weight of the corresponding driving device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weight and the action parameter of the corresponding driving device to obtain an action parameter matrix carrying the parameter weight, automatically selecting the control time step, and performing obstacle avoidance stepping control corresponding to the robot according to the action parameter matrix carrying the parameter weight.

[0010] According to another preferred embodiment of the present invention, the calculation and judgment module execution method includes: selecting n edge detection areas, calculating whether each edge pixel point is connected to the surrounding pixels in the edge detection area, when the corresponding edge pixel point is only connected to one pixel point, the corresponding edge pixel point is determined to be a breakpoint, and the number of breakpoints of all edge pixels in the edge detection area is calculated. If the number of breakpoints of all edge pixels is greater than a preset breakpoint threshold, it is determined that the pixels in the current edge detection area are discontinuous. If at least 1 / 3 of the edge detection areas among the selected n edge detection areas are discontinuous, it is automatically switched to obtain the obstacle edge features of the sonar image.

[0011] According to another preferred embodiment of the present invention, after completing the obstacle avoidance stepping control of the robot once, the data acquisition module re-collects the sonar image and the optical image in the moving direction of the robot, and the image processing module re-uses the image processing model to perform edge detection on the sonar image and the optical image respectively, and performs depth detection of obstacles in the image, and performs new stepping control according to the edge pixel coordinates of the re-detected obstacle, the position relationship between the coordinate system position of the obstacle body and the stepping drive pixel area, and the depth of the obstacle, until the edge pixel coordinates of the obstacle are no longer in the stepping drive pixel area.

[0012] According to another preferred embodiment of the present invention, the calculation and judgment module executes the following steps: setting a first depth threshold; if the image processing module does not detect the edge pixel coordinates of the obstacle in the optical image and the sonar image, then calculating the depth data of the corresponding obstacle in the optical image or the sonar image in real time; when the depth data is less than the first depth threshold, then the step control module inverts all values ​​of the action parameter matrix of the current step control to obtain a reverse-driven step control strategy.

[0013] In order to achieve at least one of the above-mentioned invention objects, the present invention further provides an automatic obstacle avoidance method for a deep-sea heavy-duty operation robot, the method comprising: Obtaining an optical image and a sonar image in the direction of movement of the robot, and performing obstacle recognition on the optical image and the sonar image, and performing edge detection on the obstacles in the optical image and the sonar image respectively using an edge detection algorithm to obtain corresponding obstacle edge pixel point coordinates; The optical image and the sonar image are selected by stepping and driving a pixel area, and the depth information of the obstacle relative to the optical image and the sonar image is obtained; Determining whether to trigger step control according to the pixel coordinates of the obstacle edge, the depth information of the obstacle and the stepping drive pixel area, and automatically generating an action parameter matrix of the robot step control according to the triggered step control; Automatically generate weight parameters corresponding to action parameters based on the coordinates of the obstacle edge pixel points, the depth information of the obstacle, and the step-driven pixel area, and perform weighted processing on the weight parameters and the corresponding action parameters; Use the action parameter matrix of step control carrying the weight parameters as an obstacle avoidance strategy to perform automatic obstacle avoidance control of the machine.

[0014] According to one preferred embodiment of the present invention, the method for selecting the step-driven pixel area includes: after obtaining the optical image and sonar image in the moving direction of the robot, determining the center point coordinates of the image, and taking the area with the center point coordinates of the image as the coordinate origin and a radius of r as the step-driven pixel area.

[0015] According to one preferred embodiment of the present invention, the method of step control includes: calling an edge detection operator to perform edge detection on the obstacles in the optical image and sonar image, calculating and judging the continuity and integrity of the edge pixels of the optical image. If the edge pixels of the optical image are continuous and complete, calculate the distances between the coordinate positions of all the edge pixels of the optical image and the center point coordinates, and judge whether the target optical image edge pixel point with the smallest distance is in the step-driven pixel area. If so, generate the action parameter matrix of the step control according to the coordinates of the target optical image edge pixel point with the smallest distance, the position of the obstacle body in the coordinate system, and the obstacle depth data.

[0016] According to another preferred embodiment of the present invention, the method of step control further includes: if the obstacle edge pixels of the current optical image after edge detection are discontinuous, switch to detecting the obstacle edge features of the sonar image, calculate the average value of the coordinates of all the edge pixels of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculate the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judge whether the midpoint coordinates are within the step-driven pixel area. If so, automatically generate the action parameter matrix of the step control according to the midpoint coordinates and the position of the obstacle body in the coordinate system.

[0017] According to another preferred embodiment of the present invention, the method for detecting the continuity of the obstacle edge pixels of the optical image includes: Select n edge detection areas, calculate whether each edge pixel is connected to surrounding pixels in the edge detection area, and when the corresponding edge pixel is only connected to one pixel, determine the corresponding edge pixel as a breakpoint, calculate the number of breakpoints of all edge pixels in the edge detection area, and if the number of breakpoints of all edge pixels is greater than a preset breakpoint threshold, determine that the pixels in the current edge detection area are discontinuous, and if at least 1 / 3 of the edge detection areas among the selected n edge detection areas are discontinuous, determine that the obstacle edge pixels in the current optical image are discontinuous as a whole, and automatically switch to obtain the obstacle edge features of the sonar image.

[0018] According to another preferred embodiment of the present invention, the automatic obstacle avoidance method includes: using different coordinate axis values ​​of the minimum value of the target optical image edge pixel point coordinates or the sonar image midpoint coordinates as the first input parameter of the robot's corresponding direction deflection driving device, and using the corresponding detected obstacle image depth value as the second input parameter of the robot's corresponding direction deflection driving device, and generating parameter weights of the corresponding driving device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weights and the action parameters of the corresponding driving device to obtain an action parameter matrix carrying the parameter weights, automatically selecting the control time step, and performing obstacle avoidance stepping control corresponding to the robot according to the action parameter matrix carrying the parameter weights.

[0019] The present invention further provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement any of the above-mentioned deep-sea heavy-duty operation robot obstacle avoidance methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Shown is a module schematic diagram of a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to the present invention.

[0021] Figure 2 Shown is a schematic flow chart of an automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to the present invention.

[0022] Figure 3 The structure diagram of the stepping pixel driving area of ​​the present invention is shown; wherein 1 is the pixel plane of the image, 2 is the edge of the obstacle, P 3 - Stepping drive pixel area. DETAILED DESCRIPTION

[0023] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0024] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0025] Please combine Figures 1 to 3The present invention discloses a deep-sea heavy-duty operation robot capable of automatically avoiding obstacles and a robot automatic obstacle avoidance method. The robot mainly includes the following core modules: a data acquisition module; an image processing module; a stepping control module and a calculation and judgment module, wherein the data acquisition module includes a visible light sensor, a sonar transceiver and an image conversion module. The image sensor is used to collect visible light in the direction of movement of the robot and convert it into a visible light image through the image conversion module. The sonar transceiver is used to collect sonar information in the direction of movement of the robot and convert the sonar information into a sonar image through the image conversion module. The image processing module can call a related target recognition algorithm to identify target obstacles in the visible light image, and the image processing module can also call an edge detection algorithm to detect edge pixel points of the target obstacle. Since both the target detection algorithm and the edge detection algorithm are mature technologies in visible light image processing, the present invention will not elaborate on which detection algorithm to choose. The present invention needs to use the image processing module to set a pixel coordinate system on the visible light image and the sonar image respectively, and each pixel point in the pixel coordinate system represents a coordinate value. The pixel coordinate system is used to calculate the coordinate values ​​of the edge pixel points of the target obstacle, and the step-driven pixel area on the visible light image and the sonar image is established according to the pixel coordinate system. The step-driven pixel area is used to detect whether the edge pixel points of the corresponding obstacle are in the direction of the robot's forward movement, and the position of the edge pixel points of the obstacle in the step-driven pixel area can be used as the key reference data for the robot's obstacle avoidance processing to construct the robot-related action parameter matrix, so that the robot can deflect along the specific direction of the edge pixel points, so that the obstacle avoidance action can be completed autonomously. It should be noted that due to the complexity of the marine environment, the visible light image may be inapplicable in special marine environments. For example, in a relatively turbid seawater environment, the visible distance of the visible light image is usually very short, and the corresponding obstacle target may have edge pixels that cannot be detected or there are many discontinuous breakpoints in the edge pixel points. The sonar has a stronger anti-interference ability for water images than visible light. Therefore, it is necessary to automatically switch the obstacle image in a specific water environment. And at the same time, the coordinates of the relevant edge pixel points are optimized. Thus, the present invention can be adapted to automatic obstacle avoidance in different marine environments.

[0026] Specifically, the present invention uses the image processing module to perform obstacle recognition and edge detection on the visible light image and the sonar image to obtain the following edge pixel coordinates A of the target obstacle: n =(x n ,y n ) and B n =(x n ,y n ), where An Represents the edge pixel of the visible light image, B n represents the edge pixel of the sonar image, and n represents the corresponding edge pixel identification sequence. n and n The pixel coordinates on the visible light image or sonar image are respectively. It should be noted that the present invention uses the central pixel of the visible light image or sonar image as the origin O of the pixel coordinates, the horizontal direction preset by the image as the x-axis, the plane preset perpendicular to the x-axis direction of the image as the y-axis, and each pixel as the unit coordinate value to obtain the edge pixel coordinates A of the target obstacle. n =(x n ,y n ) and B n =(x n ,y n ). Further, in the present invention, the origin O is used as the center of the circle and the step-by-step driving pixel area P is constructed with a radius r. 3 , where P 3 =(x j ,y j ), r 2 ≥(x j 2 +y j 2 ), when the optical image of the target obstacle has edge pixel coordinates A n ∈P 3 , the image processing module automatically generates the step control action parameter matrix E = [e 1 ,e 2 ,e 3 ,e 4 ,e 5 ,e 6 ...e n ], where e represents the action parameters of the robot corresponding to the driving device, and the action parameters include action power and direction, and different subscripts represent different driving devices. The device represented by the action parameters may include but is not limited to propeller thrusters, pump-jet thrusters, vector thrusters, steering gears, servo motors, rotary encoders, harmonic reducers, etc., which are execution devices involved in the robot action. According to the configuration of the robot action device, at least one core parameter of each configured device needs to be used as the above-mentioned action parameter. The present invention only illustrates the above-mentioned action device by example, and the present invention will not elaborate on this.

[0027] It should be noted that due to the different installation positions of different action devices, the action results of the corresponding action devices are different. Therefore, the present invention needs to distinguish the installation positions and action capabilities of the above different action devices in advance, and further distinguish the main action functions of the action parameters of the corresponding action devices. For example, the installation position of the above propeller is on the left side of the robot body, so that the core action parameters of the propeller, such as the propeller power and the forward rotation speed, can boost the robot to achieve the function of rotating to the right. It should be noted that due to the different action devices, performances and positions installed on different robots, the action parameter matrices generated by different robots are also different. The present invention only gives examples. In addition, due to the interaction of forces, the actions of the robot underwater are not only obstacle avoidance functions. For example, the robot is also working while avoiding obstacles. Therefore, the present invention needs to obtain the action parameters of all the action devices of the robot and comprehensively consider the influence of the action parameters of all the action devices on the robot's heading. The above influence can be obtained by weighting different action parameters.

[0028] It is worth mentioning that in the present invention, it is necessary to detect the continuity of the edge pixel points of the target obstacle image in the visible light image and set the relevant thresholds for the continuity of the edge pixel points of the obstacle in the visible light image. When the continuity of the edge pixel points of the obstacle in the visible light image does not meet the threshold requirements, it automatically switches to the edge pixel points of the target obstacle in the sonar image to calculate the relevant obstacle avoidance action parameter matrix. The method for judging the continuity of the edge pixel points of the obstacle in the visible light image includes: dividing the edge pixel points of the target obstacle image in the visible light image into n edge detection areas with the same area size, and calculating whether there are connected pixel points for each edge pixel point in each detection area in the n edge detection areas. The connected pixel points refer to whether there are edge pixel points above, below, left and right of this pixel point (4-connected), or whether there are edge pixel points above, below, left, right and at the four vertices (8-connected). If there is only one connected pixel point for this pixel point, it means that this pixel point is an end point, that is, this pixel point is a break point of the edge pixel point. The present invention needs to calculate the number or ratio value of the break points in the edge detection area. When the number or ratio value of the break points in the edge detection area is greater than a certain threshold, this detection area is determined as a discontinuous area, and further calculate the number of the areas determined as discontinuous areas in all the edge detection areas. When the number of the discontinuous areas is greater than one-third of the total number of the edge detection areas, it can be determined that the edge pixel points of the current target obstacle are discontinuous in the visible light image. Therefore, the edge pixel points detected by the visible light image are unavailable. Then it switches to the sonar image to obtain the edge pixel point coordinates of the corresponding obstacle.

[0029] When the edge pixels of the target obstacle image in the visible light image are determined to be continuous, the following operation is further performed: the distance between the coordinate positions of all optical image edge pixels and the center point coordinates is calculated, and it is determined whether the target optical image edge pixel with the smallest distance is in the stepping drive pixel area. If so, the stepping control action parameter matrix is ​​generated according to the target optical image edge pixel coordinates with the smallest distance, the coordinate system position of the obstacle body and the obstacle depth data. For example, the specific method is to define the target optical image edge pixel with the smallest distance from the current distance of the target obstacle to the center point coordinates as a m =(x m ,y m ), the current target obstacle main body coordinate is P i =(x i ,y i ), wherein the coordinates of the target obstacle body can be obtained by calculating the coordinates of the center point of the obstacle body, including but not limited to, at this time, the coordinates of the target obstacle body P can be calculated. i To the minimum distance target optical image edge pixel a m The coordinate vector of , if the depth of the current target obstacle image is h, further convert the coordinate vector The vector features are transformed with the following weight parameters: , , where λ represents the normalization coefficient, and They represent the weight coefficients in the x-axis direction and the y-axis direction in the optical image pixel coordinate system, respectively, and are used for weighted processing of the action parameters of the corresponding action device. It is worth noting that the weight coefficients in the above different directions represent the directionality, action intensity and corresponding coordinate vector of the corresponding action parameters. and the depth information h in numerical terms. When the x-axis direction is positive, it means that the action parameters of the relevant action device need to be driven to perform a rightward action. When the x-axis direction is a negative value, it means that the action parameters of the relevant action device need to be driven to perform a leftward action. When the value of h is lower, it means that the obstacle is closer to the robot, and the corresponding x-axis weight coefficient value is higher. After multiplying by the corresponding action parameter, a larger deflection needs to be performed at a specific time step. The larger deflection can be reflected in the driving power and rotation angle of the corresponding action device, and the present invention will not go into details. Similarly, in the y-axis direction, when the coordinate vector When the y-axis direction is positive, it means that the action parameters of the related action device need to be driven to perform an upward action. When the y-axis direction is negative, it means that the action parameters of the relevant action device need to be driven to perform a downward action. The depth information h is also used as the core parameter of the directionality and action intensity of the corresponding action parameters.

[0030] It should be noted that the above action parameter matrix E=[e 1 ,e 2 ,e 3 ,e 4 ,e 5 ,e 6 ...e n ] and weight parameters , The weighted product needs to be classified according to the direction of action of the action parameters, some of which may include attitude solution algorithms including pitch angle, roll angle and heading angle. The above attitude solution algorithm can be calculated by devices such as accelerometers to obtain the action parameter matrix carrying parameter weights.

[0031] In another preferred embodiment of the present invention, the present invention can also use an AI model to generate and process the above-mentioned action parameter matrix carrying parameter weights, for example, including but not limited to a CNN or DNN model as a regression prediction model to predict the corresponding action parameter matrix E, and input the prediction result into the action device corresponding to the robot to execute the obstacle avoidance strategy. The present invention will not go into details.

[0032] Furthermore, if the obstacle edge pixels of the current optical image after edge detection are discontinuous, the obstacle edge features of the sonar image are switched to be detected, and the average value of the edge pixel coordinates of the sonar image is calculated to obtain the average value coordinate point of the edge pixels of the sonar image, and the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image are calculated, and it is determined whether the midpoint coordinates are within the stepping drive pixel area. If so, the stepping control action parameter matrix is ​​automatically generated based on the coordinate system position including the midpoint coordinates and the obstacle body. n =(x n ,y n ), the coordinates of the midpoint connecting the average coordinate point of the edge pixels of the sonar image and the straight line of the center point of the image are defined as B s =(x s ,y s ), and calculate the main coordinates of the target obstacle as P i =(x i ,y i ), further calculate the main coordinates of the target obstacle P i =(x i ,y i) to the midpoint coordinate B s =(x s ,y s ) , further calculations are performed to obtain the following weight parameters and , λ represents the normalization coefficient, and They respectively represent the weight coefficients in the x-axis direction and the y-axis direction in the optical image pixel coordinate system, and h is the obstacle depth in the sonar image, wherein the obstacle depth of the sonar image can be directly obtained by sonar detection, and the corresponding action parameter matrix carrying parameter weights is obtained according to the same calculation method as the visible light image.

[0033] In another preferred embodiment of the present invention, if the edge pixel points of the obstacle cannot be detected, the following obstacle avoidance rules are executed: the calculation and judgment module executes including: setting a first depth threshold, if the image processing module does not detect the edge pixel point coordinates of the obstacle in the optical image and the sonar image, then the depth data of the corresponding obstacle in the optical image or the sonar image is calculated in real time, and when the depth data is less than the first depth threshold, the step control module inverts all the values ​​of the action parameter matrix of the current step control to obtain a step control strategy for reverse drive. In this embodiment, considering that a particularly large obstacle may cover the entire image interface, only the obstacle depth information can be considered as the action parameter of the reverse drive, so that the direction of the obstacle is moved away from or deflected until the obstacle edge pixel point is found as a new obstacle avoidance strategy.

[0034] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0035] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0036] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Without departing from the said principles, the embodiments of the present invention may have any variations or modifications.

Claims

1. A deep-sea heavy-duty operation robot capable of automatically avoiding obstacles, characterized in that: The robot comprises: Data acquisition module; Image processing module; Stepper control module; Calculation and judgment module; The data acquisition module collects sonar images and optical images in the direction of robot movement; the image processing module performs edge detection on obstacles in the sonar image and optical image respectively to obtain edge pixel coordinates of obstacles, wherein the image processing module also selects step-drive pixel areas for the sonar image and optical image; the calculation and judgment module determines whether to trigger step control according to the edge pixel coordinates of the obstacle; the step control module generates an action parameter matrix for step control, and automatically generates parameter weights according to the obstacle depth and edge pixel coordinates of the sonar image and / or optical image, and the robot executes a corresponding obstacle avoidance strategy according to the action parameter matrix carrying the parameter weights.

2. The deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 1, characterized in that: The execution method of the image processing module includes: after acquiring the optical image and the sonar image in the moving direction of the robot, determining the coordinates of the center point of the image, and taking the coordinates of the center point of the image as the coordinate origin and the area with r as the radius as the stepping driving pixel area; the image processing module calls the edge detection operator to perform edge detection on the obstacles in the optical image and the sonar image, and judges the continuity of the edge pixels of the optical image through the calculation and judgment module. If the edge pixels of the optical image are continuous, the distance between the coordinate positions of all the edge pixels of the optical image and the coordinates of the center point is calculated, and it is judged whether the target optical image edge pixel point with the minimum distance is in the stepping driving pixel area. If so, the action parameter matrix of the stepping control is generated according to the coordinates of the target optical image edge pixel point with the minimum distance, the coordinate system position of the obstacle body and the obstacle depth data.

3. The deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 2, characterized in that: The image processing module execution method includes: if the edge pixels of the current optical image after edge detection are discontinuous, switching to detect the obstacle edge features of the sonar image, and calculating the average value of all edge pixel coordinates of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculating the midpoint coordinates of the straight line connecting the average value coordinate point and the center point of the image, and judging whether the midpoint coordinates are within the stepping drive pixel area through the calculation and judgment module, and if so, automatically generating the stepping control action parameter matrix according to the midpoint coordinates, the coordinate system position of the obstacle body and the obstacle depth data.

4. The deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 3, characterized in that: The execution method of the step control module includes: using different coordinate axis values ​​of the minimum target optical image edge pixel point coordinates or the sonar image midpoint coordinates as the first input parameter of the corresponding direction deflection driving device of the robot, and using the corresponding detected obstacle image depth value as the second input parameter of the corresponding direction deflection driving device of the robot, and generating the parameter weight of the corresponding driving device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weight and the action parameter of the corresponding driving device to obtain an action parameter matrix carrying the parameter weight, automatically selecting the control time step, and performing obstacle avoidance step control corresponding to the robot according to the action parameter matrix carrying the parameter weight.

5. The deep-sea heavy-duty operation robot capable of automatically avoiding obstacles according to claim 1, characterized in that: The method for executing the calculation and judgment module includes: selecting n edge detection areas, calculating whether each edge pixel point is connected to surrounding pixels in the edge detection area, when the corresponding edge pixel point is only connected to one pixel point, determining the corresponding edge pixel point as a breakpoint, calculating the number of breakpoints of all edge pixel points in the edge detection area, if the number of breakpoints of all edge pixel points is greater than a preset breakpoint threshold, determining that the pixels in the current edge detection area are discontinuous, if at least 1 / 3 of the edge detection areas in the selected n edge detection areas are discontinuous, automatically switching to obtain the obstacle edge features of the sonar image; the calculation and judgment module execution includes: setting a first depth threshold, if the image processing module does not detect the edge pixel coordinates of the obstacle in both the optical image and the sonar image, then calculating the depth data of the corresponding obstacle in the optical image or the sonar image in real time, when the depth data is less than the first depth threshold, the step control module inverts all values ​​of the action parameter matrix of the current step control to obtain a step control strategy of reverse drive.

6. A method for automatic obstacle avoidance of a deep-sea heavy-duty operation robot, characterized in that: The method comprises: Obtaining an optical image and a sonar image in the direction of movement of the robot, and performing obstacle recognition on the optical image and the sonar image, and performing edge detection on the obstacles in the optical image and the sonar image respectively using an edge detection algorithm to obtain corresponding obstacle edge pixel point coordinates; The optical image and the sonar image are selected by stepping and driving a pixel area, and the depth information of the obstacle relative to the optical image and the sonar image is obtained; Determining whether to trigger step control according to the pixel coordinates of the obstacle edge, the depth information of the obstacle and the stepping drive pixel area, and automatically generating an action parameter matrix of the robot step control according to the triggered step control; Automatically generate a weight parameter corresponding to the action parameter according to the pixel coordinates of the obstacle edge, the depth information of the obstacle and the step-driven pixel area, and perform weighted processing on the weight parameter and the corresponding action parameter; The action parameter matrix of the step control carrying the weight parameters is used as an obstacle avoidance strategy to perform automatic obstacle avoidance control of the machine.

7. The automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 6, characterized in that: The stepping drive pixel area selection method includes: after obtaining the optical image and sonar image in the moving direction of the robot, determining the center point coordinates of the image, and taking the center point coordinates of the image as the coordinate origin and the area with r as the radius as the stepping drive pixel area; the stepping control method includes: calling the edge detection operator to perform edge detection on obstacles in the optical image and the sonar image, calculating and judging the continuity and integrity of the edge pixels of the optical image, if the edge pixels of the optical image are continuous and complete, calculating the distance between the coordinate positions of all the edge pixels of the optical image and the center point coordinates, and judging whether the target optical image edge pixel point with the smallest distance is in the stepping drive pixel area, and if so, generating the stepping control action parameter matrix according to the target optical image edge pixel point coordinates with the smallest distance and the obstacle depth data.

8. The automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 7, characterized in that: The step control method further includes: if the obstacle edge pixels of the current optical image after edge detection are discontinuous, switching to detect the obstacle edge features of the sonar image, and calculating the average value of all edge pixel coordinates of the sonar image to obtain the average value coordinate point of the edge pixels of the sonar image, calculating the midpoint coordinates of a straight line connecting the average value coordinate point and the center point of the image, and determining whether the midpoint coordinates are within the step drive pixel area, and if so, automatically generating the step control action parameter matrix based on the midpoint coordinates.

9. The automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 7, characterized in that: The obstacle edge pixel continuity detection method of the optical image comprises: Select n edge detection areas, calculate whether each edge pixel is connected to surrounding pixels in the edge detection area, and when the corresponding edge pixel is only connected to one pixel, determine the corresponding edge pixel as a breakpoint, calculate the number of breakpoints of all edge pixels in the edge detection area, and if the number of breakpoints of all edge pixels is greater than a preset breakpoint threshold, determine that the pixels in the current edge detection area are discontinuous, and if at least 1 / 3 of the edge detection areas among the selected n edge detection areas are discontinuous, determine that the obstacle edge pixels in the current optical image are discontinuous as a whole, and automatically switch to obtain the obstacle edge features of the sonar image.

10. The automatic obstacle avoidance method for a deep-sea heavy-duty operation robot according to claim 7, characterized in that: The automatic obstacle avoidance method includes: using different coordinate axis values ​​of the minimum value of the edge pixel point coordinates of the target optical image or the midpoint coordinates of the sonar image as the first input parameter of the corresponding direction deflection driving device of the robot, and using the corresponding detected obstacle image depth value as the second input parameter of the corresponding direction deflection driving device of the robot, and generating parameter weights of the corresponding driving device according to the first input parameter and the second input parameter, performing weighted calculation on the parameter weights and the action parameters of the corresponding driving device to obtain an action parameter matrix carrying the parameter weights, automatically selecting a control time step, and performing obstacle avoidance stepping control of the corresponding robot according to the action parameter matrix carrying the parameter weights.

Images