Visual detection-based expressway patrol robot operation obstacle avoidance system and method
Through the visual inspection and rotation adjustment system, combined with distance detection and control center operation, the obstacle blocking or impact problem of patrol robots when running along the steel guardrail track is solved, achieving continuity and efficiency of robot obstacle avoidance.
Patent Information
- Application Number
- CN202510389574.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-04
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When the patrol robot runs along the steel guardrail track, it is easily blocked or impacted by obstacles in front, resulting in equipment damage or interruption of tasks.
The visual detection module is used to identify obstacles, combine the rotation adjustment system and the distance detection module, set the warning distance threshold through the sensor, and notify the control center to perform remote control, realizing the robot's attitude conversion and obstacle avoidance.
Effectively avoid obstacle collisions, ensure the continuity and efficiency of patrol tasks, improve system response capabilities by dynamically adjusting the limit distance threshold, and avoid interrupting tasks due to temporary obstacle avoidance.
Smart Images
Figure CN120255513A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of inspection robot attitude control, and specifically to a running obstacle avoidance system and method for a highway inspection robot based on visual detection. Background Art
[0002] Highway inspection refers to the regular or irregular inspection and monitoring of highways and their related facilities through manual or automated equipment, real-time monitoring of traffic flow, detection of violations (such as speeding, illegal parking, etc.), to ensure road safety, smoothness, and intact facilities, and to promptly discover and handle various potential problems.
[0003] Currently, as an automated patrol device, the highway inspection robot combines the steel guardrail track on the highway with robot technology, enabling the patrol robot to use the steel guardrail as a track and automatically move forward along the steel guardrail, eliminating the need for active control by personnel and achieving the purpose of remotely monitoring the traffic conditions of highway sections.
[0004] However, when the inspection robot runs along the steel guardrail as a track, its main body (camera end) faces the inside of the highway and occupies a small part of the space volume on the road. If there are fallen objects, stones, vehicle failures, or other obstacles approaching the steel track on the side of the highway on its forward path, the inspection robot is likely to be blocked or even damaged by impact if it still moves forward along the steel guardrail.
[0005] Therefore, the present invention provides a running obstacle avoidance system and method for a highway inspection robot based on visual detection. Summary of the Invention
[0006] Aiming at the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a running obstacle avoidance system and method for a highway inspection robot based on visual detection, so that when an obstacle appears around the steel guardrail, the inspection robot can immediately respond and avoid obstacles, thereby preventing accidental impact or shutdown.
[0007] To achieve the above purpose, the present invention provides the following technical solution: A running obstacle avoidance system for a highway inspection robot based on visual detection, comprising:
[0008] A visual detection module, which is used to take images of the forward direction through the camera of the inspection robot, and perform real-time processing on the images collected by the camera to identify obstacles appearing around the steel guardrail;
[0009] An attitude conversion module, which is used to configure a rotation adjustment system on the inspection robot that can rotate the robot body and face it towards the outside of the highway. After the robot body rotates and faces the outside of the road, it can avoid obstacles on the road during the process of moving forward along the steel guardrail;
[0010] A distance detection module, which is used to detect the distance between the obstacle and the inspection robot by using a sensor, set a warning distance threshold, compare the distance between the obstacle and the inspection robot with the warning distance threshold, and make corresponding responses according to the comparison result;
[0011] A decision-making judgment module, which is used to notify the control center personnel to remotely control the inspection robot. The control center personnel can decide whether to enable the rotation adjustment system to change the attitude of the robot to avoid obstacles according to the images collected by the vision detection module.
[0012] In some embodiments, the rotation adjustment system includes a rotating platform that rotates the robot body. It is driven by a motor to rotate. The main body part of the robot is installed on the platform, so that this part can rotate around a central axis. Under the action of the rotating platform, the camera end of the robot body can face the highway or face away from the highway.
[0013] In some embodiments, the specific response after comparing the distance between the obstacle and the inspection robot with the warning distance threshold is: if the distance between the obstacle and the inspection robot is greater than the warning distance threshold, maintain real-time image monitoring of the obstacle and notify the control center personnel to prepare for controlling the posture change of the inspection robot; if the distance between the obstacle and the inspection robot is less than or equal to the warning distance threshold, the decision-making judgment module will be executed.
[0014] In some embodiments, the control center personnel determine whether the obstacles existing on the path of the inspection robot's forward movement need to be avoided through image information. If the obstacle moves away from the robot's movement path or the threat posed by the obstacle to the robot is very low, the control center personnel do not perform additional operations on the posture of the inspection robot, and the inspection robot continues to perform normal movement and shooting of the highway; if the obstacle is fixed and obstructs the robot's movement path, the control center personnel need to use the rotation adjustment system to move the robot body to the outside of the highway, so as to achieve attitude change and obstacle avoidance, and resume the position of the robot body to continue shooting after obstacle avoidance.
[0015] In some embodiments, a sudden obstacle distance threshold is set, and the sudden obstacle distance threshold should be less than the warning distance threshold. The distance when the visual detection module recognizes an obstacle is compared with the sudden obstacle distance threshold, and corresponding actions are taken according to the comparison result: If the distance when the visual detection module recognizes an obstacle is greater than or equal to the sudden obstacle distance threshold, no additional operation is performed; if the distance when the visual detection module recognizes an obstacle is less than the sudden obstacle distance threshold, the system controls the patrol robot to execute a speed reduction and move forward.
[0016] In some embodiments, a limit distance threshold less than the sudden obstacle distance threshold is set. When the distance when the visual detection module recognizes an obstacle is less than the sudden obstacle distance threshold, the distance when the obstacle is recognized is further compared with the limit distance threshold: If the distance when the obstacle is recognized is greater than the limit distance threshold, the speed reduction and forward movement will continue without additional adjustment actions; if the distance when the obstacle is recognized is less than or equal to the limit distance threshold, the system will control the patrol robot to stop moving forward on the steel guardrail and wait for instructions from the control center personnel.
[0017] In some embodiments, when the distance when the visual detection module recognizes an obstacle is less than or equal to the limit distance threshold, the system executes an avoidance recognition strategy;
[0018] The avoidance recognition strategy includes determining whether the obstacle is a moving object based on the recognized obstacle image information. When the system determines that the obstacle is not a moving object, no additional operation is performed; when the system determines that the obstacle is a moving object, the system will control the patrol robot to move backward.
[0019] In some embodiments, a dynamic adjustment threshold is set. When the distance between the recognized obstacle and the robot is between the sudden obstacle distance threshold and the limit distance threshold, the distance between the recognized obstacle and the robot is compared with the dynamic adjustment threshold: If the distance between the recognized obstacle and the robot is greater than or equal to the dynamic adjustment threshold, the robot will normally execute a speed reduction and move forward; if the distance between the recognized obstacle and the robot is less than the dynamic adjustment threshold, the system will determine whether the obstacle is a moving object based on the recognized obstacle image information. If the obstacle is not a moving object, the robot will normally execute a speed reduction and move forward. If the obstacle is a moving object, the dynamic adjustment threshold will be regarded as the limit distance threshold.
[0020] The present invention also provides the following technical solution: A method for obstacle avoidance during the operation of a highway patrol robot based on visual detection, the method comprising the following steps:
[0021] The camera of the patrol robot takes images of its forward direction, and the images collected by the camera are processed in real time to recognize obstacles appearing around the steel guardrail;
[0022] Configure a rotation adjustment system on the patrol robot that can rotate the robot body and face it towards the outside of the highway. After the robot body rotates and faces the outside of the road, it can avoid obstacles on the road during the process of moving forward along the steel guardrail.
[0023] Use sensors to detect the distance between the obstacle and the patrol robot, set a warning distance threshold, compare the distance between the obstacle and the patrol robot with the warning distance threshold, and make corresponding responses according to the comparison results.
[0024] Notify the control center personnel to remotely control the patrol robot. The control center personnel can decide whether to enable the rotation adjustment system to change the posture of the robot to avoid obstacles based on the images collected by the visual detection module.
[0025] The present invention further provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the above-mentioned highway patrol robot operation obstacle avoidance system based on visual detection.
[0026] The technical solution provided by the present invention has the following beneficial effects compared with the prior art:
[0027] First, the present invention uses visual detection to identify obstacles on the traveling path of the patrol robot, and notifies the control center personnel to remotely operate and control the obstacle avoidance posture of the patrol robot, so as to be able to avoid obstacles on the road, ensure that the robot will resume normal patrol tasks after avoiding obstacles, and avoid interrupting tasks due to temporary obstacle avoidance.
[0028] Second, through the design of the sudden obstacle distance threshold, when the system recognizes that the obstacle is relatively close, the system will reduce the traveling speed of the robot, providing time for the control center personnel to judge and control the posture of the robot in advance. And through the design of the limit distance threshold, when the system recognizes that the distance between the obstacle and the robot is extremely close under the influence of complex environmental and weather conditions, the system will immediately stop to prevent collisions.
[0029] Third, the present invention can dynamically adjust the limit distance threshold according to the distance between the obstacle and the robot, the moving characteristics of the obstacle, and the visibility conditions. This dynamic adjustment improves the response ability of the system, enabling the robot to recognize and respond to potential collision risks earlier. Especially when sudden obstacles appear, it can quickly make obstacle avoidance responses. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a logical schematic diagram of a highway patrol robot operation obstacle avoidance system based on visual detection according to the present invention;
[0031] Figure 2 This is a schematic flowchart of an obstacle avoidance method for the operation of a highway inspection robot based on visual detection according to the present invention. Specific embodiments
[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0033] It can be understood that the term "one" should be understood as "at least one" or "one or more". That is, in one embodiment, the number of one element can be one, and in other embodiments, the number of this element can be multiple. The term "one" cannot be understood as a limitation on the quantity.
[0034] The present invention provides an obstacle avoidance system for the operation of a highway inspection robot based on visual detection, as Figure 1 shown, including:
[0035] A visual detection module, which is used to take images of the forward direction through the camera of the inspection robot, and perform real-time processing on the images collected by the camera, extract the obstacle features in the images through edge detection or a deep learning model, and identify the obstacles appearing around the steel guardrail;
[0036] An attitude conversion module, which is used to configure a rotation adjustment system on the inspection robot that can rotate the robot body (camera end) and face the outside of the highway, so that after the robot body rotates and faces the outside of the highway, it can avoid the obstacles existing on the highway during the process of moving forward along the steel guardrail;
[0037] A distance detection module, which is used to detect the distance between the obstacle and the inspection robot by using a sensor when the visual detection module identifies an obstacle ahead, set a warning distance threshold, compare the distance between the obstacle and the inspection robot with the warning distance threshold, and make corresponding responses according to the comparison results;
[0038] A decision-making and judgment module, which is used to notify the control center personnel to remotely control the inspection robot, and the control center personnel can decide whether to enable the rotation adjustment system to rotate the robot body to the outside of the highway to avoid the obstacles on the highway according to the images collected by the visual detection module.
[0039] Specifically, the rotation adjustment system is a common rotating platform for rotating the robot body. It is driven by a motor to rotate. The main body part of the robot is installed on the platform, enabling this part to rotate around a central axis. The axis center should be located at the connection part where the robot touches the steel guardrail. Under the action of the rotating platform, the camera end of the robot body can face the highway or face away from the highway. During normal operation, the inspection robot body should face the highway, and the body is located inside the road, so that the camera end can collect images of the highway conditions. When it is necessary to avoid obstacles on the road, under the action of the rotating platform, the robot body can be rotated to the side of the steel guardrail away from the highway. When the body rotates to the outside of the road, although the camera end will face away from the highway at this time and cannot normally collect road condition images, it can avoid obstacles on the road.
[0040] The specific reaction after comparing the distance between the obstacle and the inspection robot with the warning distance threshold is as follows: If the distance between the obstacle and the inspection robot is greater than the warning distance threshold, it indicates that when the visual detection module can identify an obstacle on the road ahead, the inspection robot is far from the identified obstacle, and the image may not be clear enough to immediately provide sufficient details for effective obstacle avoidance decision-making. Keep real-time image monitoring of the obstacle and notify the control center personnel to prepare for controlling the posture change of the inspection robot; If the distance between the obstacle and the inspection robot is less than or equal to the warning distance threshold, the decision-making judgment module will be executed. After the decision-making judgment module, the control center personnel can observe the obstacle image information collected by the robot. Determine whether the obstacle existing on the forward path of the inspection robot needs to be avoided through the image information. If the obstacle moves away from the robot's moving path or the threat posed by the obstacle to the robot is very low (such as branches, small dropped items, etc.), then the control center personnel do not perform additional operations on the posture of the inspection robot, and the inspection robot continues to execute normal travel and shooting of the road; If the obstacle is fixed and obstructs the robot's travel path, the control center personnel need to use the rotation adjustment system to move the robot body to the outside of the road. At this time, the inspection robot will lose the function of shooting the road conditions, but it can avoid obstacles on the road and resume the position of the robot body to continue shooting after obstacle avoidance.
[0041] Furthermore, when the visual detection module identifies obstacles on the route of the inspection robot's advancement, image acquisition through the camera is easily affected by the external environment. For example, rain, snow, or fog may cause rain or snow to accumulate on the camera lens, or reduce visibility in a haze environment. At this time, when the inspection robot is operating normally on the highway, when the visual detection module identifies an obstacle around the front steel guardrail, the inspection robot may already be about to come into contact with the obstacle, and the reaction time left for the control center personnel is insufficient. Therefore, it is necessary to set a sudden obstacle distance threshold, which should be less than the warning distance threshold. For example, the sudden obstacle distance threshold can be set to 60%-80% of the warning distance threshold. Compare the distance when the visual detection module identifies an obstacle with the sudden obstacle distance threshold, and make corresponding responses according to the comparison results. Specifically, if the distance when the visual detection module identifies an obstacle is greater than or equal to the sudden obstacle distance threshold, it indicates that when the system notifies the control center personnel that they need to control the posture of the inspection robot, the inspection robot is still at a sufficient distance from the obstacle, giving the control center personnel enough time to react and determine whether obstacle avoidance is needed, and no additional operation is required. If the distance when the visual detection module identifies an obstacle is less than the sudden obstacle distance threshold, it indicates that when the system notifies the control center personnel that they need to control the posture of the inspection robot, the inspection robot is at a short distance from the obstacle, leaving insufficient time for the control center personnel to react and judge. At this time, the inspection robot advances at a speed reduced to half of the normal traveling speed to leave enough reaction time for the control center personnel.
[0042] Meanwhile, extreme situations should also be taken into account, such as extremely low visibility caused by severe smog, or an obstacle suddenly appearing in front of the inspection robot. To prevent the inspection robot from hitting the suddenly appearing obstacle, an extreme distance threshold smaller than the sudden obstacle distance threshold should be set. When the distance when the vision detection module recognizes an obstacle is less than the sudden obstacle distance threshold, the distance when the obstacle is recognized is compared with the extreme distance threshold, and corresponding actions are taken according to the comparison result: If the distance when the obstacle is recognized is greater than the extreme distance threshold, it indicates that there is still a certain distance between the robot and the obstacle when the obstacle is recognized, and the robot will continue to execute decelerated movement without additional adjustment actions; If the distance when the obstacle is recognized is less than or equal to the extreme distance threshold, it indicates that the distance between the robot and the obstacle is extremely close when the obstacle is recognized, and the system will control the inspection robot to stop moving on the steel guardrail and wait for instructions from the control center personnel. For example, the warning distance threshold is set at 100 meters, the sudden obstacle distance threshold is set at 60 meters, and the extreme distance threshold is set at 10 meters. Under normal conditions, the camera on the inspection robot can observe and recognize obstacles around the steel guardrail 100 meters away, and notify the control center personnel to make judgments and responses after the distance between the robot and the obstacle enters within 100 meters. In special cases, when the distance between the robot and the obstacle is 50 meters when the vision detection module recognizes the obstacle, the robot will execute decelerated movement to leave reaction time for the control center personnel. When the distance between the robot and the obstacle is 8 meters when the vision detection module recognizes the obstacle, which is less than the extreme distance threshold, the inspection robot will immediately stop to avoid hitting the obstacle when continuing to move forward.
[0043] On the other hand, when the distance when the visual detection module recognizes an obstacle is less than or equal to the limit distance threshold, it means that the camera of the inspection robot suddenly captures and recognizes an obstacle in front. This situation may indicate that the obstacle is moving, such as an animal on the road, goods spilled from a vehicle, or an object blown by the wind. Such obstacles will not only block the normal path of the inspection robot along the steel guardrail, but may even move towards the robot and cause a collision with the robot. To avoid the occurrence of the above situation, when the distance when the visual detection module recognizes an obstacle is less than or equal to the limit distance threshold, an avoidance recognition strategy should also be executed. The avoidance recognition strategy includes judging whether the obstacle is a moving object based on the recognized obstacle image information. This can be achieved by comparing consecutive frames of images. When the position of the obstacle changes significantly, the system can judge that the obstacle is a moving object, or by combining a target tracking algorithm, the system can track the movement trajectory of the obstacle and judge whether it is a moving object. When the system judges that the obstacle is not a moving object, no additional operation is performed; when the system judges that the obstacle is a moving object, after stopping the inspection robot from moving forward, the inspection robot will be controlled to move backward. By moving in the reverse direction, the inspection robot can be kept away from the obstacle, avoiding the phenomenon that the moving obstacle may collide with the robot.
[0044] As another preferred embodiment of the present invention, the limit distance threshold can be dynamically adjusted. When the vision detection module recognizes an obstacle around the steel guardrail ahead, if the distance between the obstacle and the robot is between the sudden obstacle distance threshold and the limit distance threshold at this time, the robot will execute a decelerated forward movement to reserve time for the control center personnel to react. However, the closer the recognized distance between the robot and the obstacle is to the limit distance threshold, the closer the robot is to the obstacle. In the case where the recognized distance between the robot and the obstacle is very close to the limit distance threshold, even if the robot executes a decelerated forward movement, the robot may still contact and collide with the obstacle before the control center personnel can react. Therefore, a dynamic adjustment threshold needs to be set. This threshold can be taken as half of the sudden obstacle distance threshold. When the recognized distance between the obstacle and the robot is between the sudden obstacle distance threshold and the limit distance threshold, the recognized distance between the obstacle and the robot is compared with the dynamic adjustment threshold, and corresponding reactions are made according to the comparison results. Specifically: if the recognized distance between the obstacle and the robot is greater than or equal to the dynamic adjustment threshold, it indicates that the distance between the obstacle and the robot is sufficient, and the robot will normally execute a decelerated forward movement; if the recognized distance between the obstacle and the robot is less than the dynamic adjustment threshold, it indicates that the distance between the obstacle and the robot is short and there is a risk of collision. The system will determine whether the obstacle is a moving object based on the recognized obstacle image information. If the obstacle is not a moving object, the robot will normally execute a decelerated forward movement. If the obstacle is a moving object, the dynamic adjustment threshold will be regarded as the limit distance threshold, and the system will determine that the recognized distance of this obstacle has entered the limit distance threshold, and perform an operation to make the robot move backward. Regarding whether the limit distance threshold needs to be dynamically adjusted, the reference of determining whether the obstacle is a moving object is added. This is because when the vision detection module recognizes an obstacle around the steel guardrail ahead, if the distance is too close (below the dynamic adjustment threshold), the time window for the robot to collide with the obstacle becomes very short. At this time, the time reserved for the control center personnel to remotely control by executing a decelerated forward movement is relatively tight. If the obstacle is also a moving object, it is easy to cause a collision problem. Therefore, the system will dynamically adjust the limit distance threshold to increase the sensitivity of controlling the robot to move backward.
[0045] Generally speaking, the present invention aims to design an obstacle avoidance system for the operation of a highway inspection robot based on visual detection. Aiming at the problem that the inspection robot is easily blocked and collided by obstacles when moving along the steel guardrail, the present invention uses visual detection means to identify the obstacles on the traveling path of the inspection robot, and notifies the control center personnel to remotely operate and control the obstacle avoidance posture of the inspection robot, so as to be able to avoid the obstacles on the road, ensure that the robot will resume normal inspection tasks after avoiding the obstacles, avoid interrupting the tasks due to temporary obstacle avoidance, and ensure the continuity and efficiency of the inspection tasks. Through the design of the sudden obstacle distance threshold, when an obstacle is identified as being relatively close, the system will reduce the traveling speed of the robot, providing the control center personnel with time to judge and control the robot's posture in advance. And through the design of the extreme distance threshold, when the system identifies an obstacle at a very close distance to the robot due to the influence of complex environmental and weather conditions, the system will immediately stop the machine to prevent collisions. The system can judge whether the obstacle is a dynamic object (such as an animal, an item blown by the wind, etc.) through image analysis and target tracking algorithms. For moving obstacles, the robot will automatically execute reverse traveling to avoid collisions with the obstacles. At the same time, the system can dynamically adjust the extreme distance threshold according to the distance between the obstacle and the robot, the moving characteristics of the obstacle, and the visibility conditions. This dynamic adjustment improves the response ability of the system, enabling the robot to identify and respond to potential collision risks earlier. Especially when a sudden obstacle appears, it can quickly make an obstacle avoidance reaction.
[0046] The present invention provides a method for obstacle avoidance in the operation of a highway inspection robot based on visual detection, as Figure 2 shown, the method includes the following steps:
[0047] Step 1, use the camera of the inspection robot to take images of its forward direction, and perform real-time processing on the images collected by the camera to identify the obstacles that appear around the steel guardrail;
[0048] Step 2, configure a rotation adjustment system on the inspection robot that can rotate the robot body and face it towards the outside of the highway. After the robot body rotates and faces towards the outside of the highway, it can avoid the obstacles existing on the road during the process of moving along the steel guardrail;
[0049] Step 3, use a sensor to detect the distance between the obstacle and the inspection robot, set a warning distance threshold, compare the distance between the obstacle and the inspection robot with the warning distance threshold, and make corresponding reactions according to the comparison results;
[0050] Step 4, notify the control center personnel to remotely control the inspection robot. The control center personnel can decide whether to enable the rotation adjustment system to change the posture of the robot for obstacle avoidance according to the images collected by the visual detection module.
[0051] In the embodiments disclosed by the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. The embodiments disclosed by the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit, the above-mentioned functions defined in the methods of the present application are executed. It should be noted that the above-mentioned computer-readable medium in the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program codes. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device. The program codes contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless segments, wire segments, optical cables, RF, etc., or any suitable combination of the above.
[0052] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.
[0053] Those skilled in the art should understand that the above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application.
Claims
1. An obstacle avoidance system for the operation of a highway patrol robot based on visual detection, characterized in that, Including: A visual detection module, which is used to take images of the forward direction through the camera of the inspection robot, and perform real-time processing on the images collected by the camera to identify obstacles appearing around the steel guardrail; An attitude conversion module, which is used to configure a rotation adjustment system on the inspection robot that can rotate the robot body and face it towards the outside of the highway. After the robot body rotates and faces the outside of the highway, it can avoid obstacles existing on the highway during the process of moving forward along the steel guardrail; A distance detection module, which is used to detect the distance between the obstacle and the inspection robot by using a sensor, set a warning distance threshold, compare the distance between the obstacle and the inspection robot with the warning distance threshold, and make corresponding responses according to the comparison result; A decision-making judgment module, which is used to notify the control center personnel to remotely control the inspection robot. The control center personnel can decide whether to enable the rotation adjustment system to change the attitude of the robot to avoid obstacles according to the images collected by the visual detection module.
2. The operation obstacle avoidance system of a highway patrol robot based on visual detection according to claim 1, characterized in that, The rotation adjustment system includes a rotating platform that rotates the robot body. It is driven by a motor to rotate. The main body part of the robot is installed on the platform, so that this part can rotate around a central axis. Under the action of the rotating platform, the camera end of the robot body can face the highway or face away from the highway.
3. The operation obstacle avoidance system of a highway patrol robot based on visual detection according to claim 1, wherein, The specific response after comparing the distance between the obstacle and the inspection robot with the warning distance threshold is as follows: If the distance between the obstacle and the inspection robot is greater than the warning distance threshold, maintain real-time image monitoring of the obstacle and notify the control center personnel to prepare for controlling the posture change of the inspection robot; If the distance between the obstacle and the inspection robot is less than or equal to the warning distance threshold, the decision-making judgment module will be executed.
4. The operation obstacle avoidance system of a highway patrol robot based on visual detection according to claim 3, characterized in that, The control center personnel determine through the image information whether the obstacles existing on the forward path of the inspection robot need to be avoided. If the obstacle moves away from the robot's moving path or the threat posed by the obstacle to the robot is very low, the control center personnel do not perform additional operations on the posture of the inspection robot, and the inspection robot continues to perform normal travel and shooting of the highway; If the obstacle is fixed and obstructs the robot's travel path, the control center personnel need to use the rotation adjustment system to move the robot body to the outside of the highway, so as to achieve attitude change and obstacle avoidance, and resume the position of the robot body to continue shooting after obstacle avoidance.
5. The operation obstacle avoidance system for a highway patrol robot based on visual detection according to claim 4, characterized in that, Set a sudden obstacle distance threshold, and the sudden obstacle distance threshold should be less than the warning distance threshold. Compare the distance when the visual detection module identifies the obstacle with the sudden obstacle distance threshold, and make corresponding responses according to the comparison result: If the distance when the visual detection module identifies the obstacle is greater than or equal to the sudden obstacle distance threshold, no additional operation is performed; If the distance when the visual detection module identifies the obstacle is less than the sudden obstacle distance threshold, the system controls the inspection robot to perform decelerated travel.
6. The obstacle avoidance system for the operation of a highway inspection robot based on visual detection according to claim 5, wherein, Set a limit distance threshold smaller than the sudden obstacle distance threshold. When the distance when the visual detection module recognizes an obstacle is less than the sudden obstacle distance threshold, then compare the distance when the obstacle is recognized with the limit distance threshold: If the distance when the obstacle is recognized is greater than the limit distance threshold, continue to execute decelerated travel without making additional adjustment actions; If the distance when the obstacle is recognized is less than or equal to the limit distance threshold, the system will control the inspection robot to stop moving forward on the steel guardrail and wait for instructions from the control center personnel.
7. The operation obstacle avoidance system of a highway patrol robot based on visual detection according to claim 6, characterized in that, When the distance when the visual detection module recognizes an obstacle is less than or equal to the limit distance threshold, the system executes an avoidance recognition strategy; The avoidance recognition strategy includes judging whether the obstacle is a moving object according to the recognized obstacle image information. When the system judges that the obstacle is not a moving object, no additional operation is performed; When the system judges that the obstacle is a moving object, it will control the inspection robot to move backward.
8. The operation obstacle avoidance system of a highway patrol robot based on visual detection according to claim 7, characterized in that, Set a dynamic adjustment threshold. When the distance between the recognized obstacle and the robot is between the sudden obstacle distance threshold and the limit distance threshold, compare the distance between the recognized obstacle and the robot with the dynamic adjustment threshold: If the distance between the recognized obstacle and the robot is greater than or equal to the dynamic adjustment threshold, the robot will normally execute decelerated forward movement; If the distance between the recognized obstacle and the robot is less than the dynamic adjustment threshold, the system will judge whether the obstacle is a moving object according to the recognized obstacle image information. If the obstacle is not a moving object, the robot will normally execute decelerated forward movement. If the obstacle is a moving object, the dynamic adjustment threshold will be regarded as the limit distance threshold.
9. A method for an expressway patrol robot based on visual detection to avoid obstacles during operation, characterized in that, According to the visual detection-based highway inspection robot operation obstacle avoidance system according to any one of claims 1-8, the method includes the following steps: Use the camera of the inspection robot to take images of its forward direction, and perform real-time processing on the images collected by the camera to identify obstacles appearing around the steel guardrail; Configure a rotation adjustment system on the inspection robot that can rotate the robot body and face it towards the outside of the highway. After the robot body rotates and faces towards the outside of the highway, it can avoid obstacles existing on the highway during the process of moving forward along the steel guardrail; Use a sensor to detect the distance between the obstacle and the inspection robot, set a warning distance threshold, compare the distance between the obstacle and the inspection robot with the warning distance threshold, and make corresponding responses according to the comparison results; Notify the control center personnel to remotely control the inspection robot. The control center personnel can decide whether to enable the rotation adjustment system to change the posture of the robot for obstacle avoidance according to the images collected by the visual detection module.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a visual detection-based highway inspection robot operation obstacle avoidance system according to any one of claims 1-8 above.