Safety inspection robot driving method and device based on structured light recognition, electronic equipment and program product

Abstract

The application discloses a safety inspection robot driving method and device based on structured light recognition, electronic equipment and program product. The safety inspection robot driving method comprises the following steps: projecting structured light to a wave-shaped guardrail to form a reflected light bar on the surface of the wave-shaped guardrail; collecting a guardrail surface image containing the reflected light bar; processing the guardrail surface image to determine a region of interest of the reflected light bar in the guardrail surface image; extracting a light bar feature point of the reflected light bar in the region of interest; determining an anchor feature based on the geometric distribution of the light bar feature point; determining a steering vector for steering the inspection robot to a preset orientation according to the deviation of the anchor feature from a preset position; determining a differential speed control amount of a driving mechanism of the inspection robot in real time according to the steering vector; and driving the driving mechanism according to the differential speed control amount, so that the inspection robot travels along the wave-shaped guardrail. The safety inspection robot driving method can enable the inspection robot to stably and reliably travel autonomously along the wave-shaped guardrail.

Description

Technical Field

[0001] This invention relates to the field of inspection robot technology, and in particular to a driving method, device, electronic equipment and program product for a security inspection robot based on structured light recognition. Background Technology

[0002] Protective structures such as wave-shaped guardrails, which extend continuously along railway lines, often require long-distance, continuous inspection and monitoring. To reduce the burden of manual inspection, inspection robots capable of moving along the guardrails are typically used to assist in related safety operations. These robots move along the paths defined by the guardrails and perform corresponding inspection tasks during their movement. Generally, these inspection robots need to acquire certain information while moving along the guardrails to maintain their relative position to the guardrails.

[0003] However, the environment in which guardrails are located is highly variable. The surface morphology of the guardrails themselves and the external conditions of the site may fluctuate due to various factors. There is still room for improvement in how inspection robots maintain stable movement relative to the guardrails during long-distance travel. There is a desire to provide a more stable and reliable intelligent inspection robot drive solution that can move along guardrails.

[0004] The background description is provided for the purpose of understanding the relevant technologies in this field and is not intended as an admission of prior art. Summary of the Invention

[0005] The present invention aims to provide a solution that at least partially solves the above-mentioned problems, so as to enable the inspection robot to move autonomously along the guardrail stably and reliably.

[0006] In a first aspect, embodiments of the present invention provide a driving method for a security inspection robot based on structured light recognition, the driving method comprising:

[0007] Structured light is projected onto the wave-shaped guardrail to form reflective light stripes on the surface of the guardrail;

[0008] Acquire an image of the guardrail surface containing the reflected light stripe;

[0009] The image of the guardrail is processed to determine the region of interest of the reflected light stripe in the image of the guardrail.

[0010] Extract the light stripe feature points of the reflected light stripe within the region of interest;

[0011] Anchoring features are determined based on the geometric distribution of the light stripe feature points;

[0012] The amount of steering that causes the inspection robot to turn toward the preset orientation is determined based on the deviation of the anchoring feature from the preset position.

[0013] The differential control amount of the drive mechanism of the inspection robot is determined in real time based on the steering amount.

[0014] The drive mechanism is driven according to the differential control value so that the inspection robot moves along the wave-shaped guardrail.

[0015] In some embodiments, the wave-shaped guardrail is a multi-wave-shaped guardrail, and the reflected light strip is a multi-periodic light strip formed on the surface of the multi-wave-shaped guardrail;

[0016] Extracting the light stripe feature points of the reflected light stripe within the region of interest includes:

[0017] Identify multiple peak feature points of the multi-period light stripe within the region of interest; and / or

[0018] Identify multiple trough feature points of the multi-period light stripe within the region of interest;

[0019] The determination of anchoring features based on the geometric distribution of the light stripe feature points includes:

[0020] A single anchor point is determined based on the geometric relationship between the multiple peak feature points and / or multiple trough feature points.

[0021] In some embodiments, determining a single anchor point based on the geometric relationship between the plurality of peak feature points and / or the plurality of trough feature points includes:

[0022] The centerline of the multi-period light stripe is determined based on the multiple peak feature points and / or multiple trough feature points;

[0023] The anchor point is determined based on the position where the centerline of the multi-period light stripe intersects with a preset reference line in the guardrail surface image; or

[0024] Determining a single anchor point based on the geometric relationship between the plurality of peak feature points and / or the plurality of trough feature points includes:

[0025] The geometric midpoint of the plurality of wave crest feature points and / or the plurality of wave trough feature points is determined as the anchor point.

[0026] In some embodiments, acquiring the image of the guardrail surface including the reflected light stripe includes:

[0027] The guardrail surface image is acquired in the form of consecutive frames;

[0028] The step of processing the guardrail surface image to determine the region of interest of the reflected light stripe in the guardrail surface image includes:

[0029] For the guardrail surface image of the initial frame, the region of interest of the initial frame is determined according to the preset projection direction of the structured light;

[0030] For the guardrail surface image that is not the initial frame, the region of interest for the current frame is determined based on the position of the anchoring feature determined by the guardrail surface image of the previous frame.

[0031] In some embodiments, determining the turning amount to orient the inspection robot toward a preset orientation based on the deviation of the anchoring feature relative to a preset position includes:

[0032] Determine the lateral pixel deviation of the anchoring feature relative to a preset position in the image coordinate system of the guardrail surface image;

[0033] The lateral pixel deviation is used as a control error input feedback controller to determine the steering amount.

[0034] In some embodiments, the inspection robot includes at least two walking parts and at least two drive mechanisms that independently drive the at least two walking parts;

[0035] The step of determining the differential control amount of the drive mechanism of the inspection robot in real time based on the steering amount includes:

[0036] The multi-channel independent speed control quantity is obtained from the decomposition of the steering amount, and the multi-channel independent speed control quantity corresponds one-to-one with the at least two drive mechanisms.

[0037] In some embodiments, the driving method further includes:

[0038] Identify whether the reflected light strip in the guardrail surface image has a given light strip anomaly, the light strip anomaly including light strip breakage at a given location and / or light strip distortion at a given location compared to a preset contour;

[0039] If the light stripe is detected to be abnormal, the position of the guardrail rivet is determined according to the position of the abnormal light stripe in the guardrail surface image.

[0040] The speed adjustment curve of the inspection robot's drive mechanism is generated based on the position of the guardrail rivets. The speed adjustment curve is used to drive the inspection robot to travel through the area where the guardrail rivets are located.

[0041] In some embodiments, the inspection robot includes an adjustable suspension mechanism;

[0042] The driving method further includes:

[0043] Determine the pixel spacing between adjacent peaks of the multi-period light stripe in the guardrail surface image;

[0044] Based on the difference between the pixel pitches, the adjustable suspension mechanism adjusts the height of the inspection robot relative to the surface of the wave-shaped guardrail so that the inspection robot maintains a preset vertical posture on the inclined surface of the wave-shaped guardrail.

[0045] In some embodiments, the inspection robot includes a magnetic holding mechanism for adsorbing and holding the inspection robot on the wave-shaped guardrail and an additional holding mechanism for holding the inspection robot on the wave-shaped guardrail in an abnormal manner.

[0046] The driving method further includes:

[0047] The adsorption and holding force of the wave-shaped guardrail is monitored in real time to obtain the change value of the adsorption and holding force;

[0048] When the change in the adsorption and holding force exceeds a preset threshold, the additional holding mechanism is triggered to hold the inspection robot on the wave-shaped guardrail.

[0049] In a second aspect, embodiments of the present invention provide a driving device for a security inspection robot based on structured light recognition, comprising:

[0050] A structured light projection module is configured to project structured light onto a wave-shaped guardrail to form a reflected light strip on the surface of the wave-shaped guardrail.

[0051] The image acquisition module is configured to acquire an image of the guardrail surface containing the reflected light strips;

[0052] The region of interest determination module is configured to process the guardrail surface image and determine the region of interest of the reflected light stripe in the guardrail surface image;

[0053] The feature point determination module is configured to extract the light stripe feature points of the reflected light stripe within the region of interest.

[0054] Anchoring feature determination module, configured to determine anchoring features based on the geometric distribution of the light stripe feature points;

[0055] The steering determination module is configured to determine the steering amount that causes the inspection robot to turn toward a preset orientation based on the deviation of the anchoring feature relative to a preset position.

[0056] The differential speed determination module is configured to determine the differential speed control amount of the drive mechanism of the inspection robot in real time based on the steering amount.

[0057] The drive module is configured to drive the drive mechanism according to the differential control amount, so that the inspection robot moves along the wave-shaped guardrail.

[0058] In a third aspect, embodiments of this disclosure provide an electronic device that may include: a processor and a memory storing a computer program, the processor being configured to implement the method as described in the first aspect when the computer program is executed.

[0059] In a fourth aspect, embodiments of this disclosure provide a program product including a computer program, wherein the computer program, when executed by a processor, implements the method as described in the first aspect.

[0060] The structured light recognition-based safety inspection robot driving method provided in this invention can actively project structured light onto a wave-shaped guardrail to form reflected light stripes on its surface. It acquires an image of the guardrail surface containing the reflected light stripes and extracts light stripe feature points within the region of interest corresponding to the light stripes. Based on the geometric distribution of these feature points, it determines anchoring features. Then, based on the deviation of the anchoring features from a preset position, it determines the turning amount that will cause the inspection robot to move towards a preset orientation. Based on this, it determines the differential speed control amount of the driving mechanism in real time to drive the inspection robot. This scheme enhances the recognizability of the wave-shaped guardrail's geometry in the image through actively projected structured light and uses the deviation of the anchoring features from the preset position to perform closed-loop control of the inspection robot's drive. It solves the technical problem of stable autonomous movement of the inspection robot over long distances on the complex undulating surface of the wave-shaped guardrail, thereby significantly improving the stability and reliability of the inspection robot's autonomous movement along the wave-shaped guardrail.

[0061] In a further embodiment of the present invention, the lateral pixel deviation of the anchoring feature relative to the guardrail surface image at a preset position in the image coordinate system is used as the control error input feedback controller to obtain the steering amount. This scheme realizes a stable deflection adjustment mechanism and further improves the stability of the inspection robot's travel orientation.

[0062] In a further embodiment of the present invention, the presence of light stripe breaks and / or light stripe distortions compared to a preset contour in the reflected light stripe in the guardrail surface image is identified. Then, when a light stripe anomaly is identified, the position of the guardrail rivet is determined according to its corresponding position, and a speed adjustment curve of the drive mechanism is generated accordingly. This solution solves the technical problem that the rivets on the wave-shaped guardrail and the guardrail joints cause body bumps and affect the smooth movement of the inspection robot. It enables the inspection robot to pass smoothly through the area where the guardrail rivets and guardrail joints are located, thereby improving the stability of the inspection robot's movement.

[0063] In a further embodiment of the present invention, the pixel spacing between multiple pairs of adjacent peaks or multiple pairs of adjacent troughs in the guardrail surface image of the multi-period light stripe is determined, and the adjustment amount of the adjustable suspension mechanism on the height of the aircraft body relative to the surface of the wave-shaped guardrail is adjusted according to the difference in the pixel spacing. This solution reduces the difference by adjusting the height of the aircraft body according to the difference in the pixel spacing between adjacent peaks or multiple pairs of adjacent troughs, thereby enabling the aircraft body to maintain a preset posture with the surface of the wave-shaped guardrail, solving the technical problem that the aircraft body is prone to posture deviation on the surface of the wave-shaped guardrail, thus affecting the stability of data acquisition and travel.

[0064] In some embodiments of the present invention, a magnetic holding mechanism is also used to adhere the inspection robot to the wave-shaped guardrail. The magnetic holding force is monitored in real time, and when the change in the magnetic holding force exceeds a preset threshold, an additional holding mechanism is triggered to abnormally hold the inspection robot to the wave-shaped guardrail. This solution, by introducing an additional holding mechanism to supplement the holding when the magnetic holding force changes abnormally, solves the technical problem that relying solely on a single holding method poses a risk of the inspection robot detaching from the wave-shaped guardrail when the magnetic holding force fluctuates.

[0065] Other optional features and technical effects of the embodiments of the present invention are partly described below and partly apparent from reading this document. Attached Figure Description

[0066] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The elements shown are not limited to the scale shown in the drawings, and the same or similar reference numerals in the drawings denote the same or similar elements, wherein:

[0067] Figure 1 A schematic side view of the multi-wave guardrail is shown;

[0068] Figure 2 An exemplary flowchart of a security inspection robot driving method based on structured light recognition according to an embodiment of the present invention is shown;

[0069] Figure 3 An exemplary flowchart of a security inspection robot driving method based on structured light recognition according to an embodiment of the present invention is shown;

[0070] Figure 4 An exemplary flowchart of a security inspection robot driving method based on structured light recognition according to an embodiment of the present invention is shown;

[0071] Figure 5 An exemplary flowchart of a security inspection robot driving method based on structured light recognition according to an embodiment of the present invention is shown;

[0072] Figure 6A schematic diagram is shown illustrating the projection of structured light and the acquisition of images of a guardrail surface using a method according to an embodiment of the present invention;

[0073] Figure 7 A schematic image of a guardrail surface including reflective light stripes is shown;

[0074] Figure 8 A schematic diagram of a region of interest determined using a structured light recognition-based security inspection robot driving method according to an embodiment of the present invention is shown.

[0075] Figure 9 A schematic diagram is shown showing light stripe feature points determined using a structured light recognition-based security inspection robot driving method according to an embodiment of the present invention;

[0076] Figure 10 An exemplary block diagram of a security inspection robot drive device based on structured light recognition according to an embodiment of the present invention is shown; and

[0077] Figure 11 An exemplary structural diagram of an electronic device capable of implementing the method according to an embodiment of the present invention is shown.

[0078] List of reference numerals in the attached diagram:

[0079] 110. Structured light projector; 120. Image acquisition device; 130. Reflected light stripe; 131. Peak; 132. Peak feature point; 133. Trough; 134. Trough feature point; 135. Light stripe ridge line; 141. Surface highlight; 142. Background interference; 143. Specular reflection of light stripe; 144. Partial high reflectivity of light stripe; L. Wave-shaped guardrail; P. Guardrail post; Q. Guardrail surface point; q. Imaging point; S1. Centerline; S0. Preset reference line; E. Anchor point; S210~S280, S310~S320, S410~S430, S510~S520: Method steps. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0081] In this embodiment of the invention, the inspection robot includes a mobile platform capable of autonomous or semi-autonomous movement along a predetermined object and performing tasks such as inspection, detection, and data collection. This inspection robot can also be referred to as a safety inspection robot. In this embodiment of the invention, the wave-shaped guardrail includes a guardrail whose surface has an undulating shape along its extension direction. In this embodiment of the invention, the longitudinal extension direction of the guardrail (also referred to as the extension direction of the guardrail) refers to the direction in which the guardrail extends continuously as a whole when it is continuously laid along a route. For explanation, this direction is generally consistent with the direction of the road or path along which the guardrail follows, and is distinct from the height direction of the guardrail, the lateral direction of the guardrail panel, and the local direction of the wave undulations on the surface of the multi-wave guardrail. In some embodiments, the wave-shaped guardrail may include a multi-wave guardrail, i.e., its surface has, for example, but not limited to, two, three, four, or other numbers of undulation cycles, i.e., multiple wave structures, and thus multiple peaks and troughs. The surface of the wave-shaped guardrail may also be inclined relative to the horizontal plane.

[0082] In this embodiment of the invention, the structured light includes projected light with predetermined spatial distribution characteristics. In this embodiment of the invention, the reflected light strip includes a collectable strip-shaped light distribution formed on the surface of the wave-shaped guardrail after the structured light illuminates it. For explanation, when the wave-shaped guardrail is a multi-waveform guardrail, the reflected light strip may include a multi-period light strip with multiple undulation periods along its extension direction. The peak feature points and trough feature points may refer to the feature points in the multi-period light strip corresponding to the protruding and concave portions of the wave undulations, respectively.

[0083] In this embodiment of the invention, the Region of Interest (ROI) includes a local region delineated in the acquired image for subsequent processing. In this embodiment of the invention, the light stripe feature points include points extracted from the reflected light stripe that characterize the position and / or shape of the reflected light stripe; the anchoring features include reference features determined based on the geometric distribution of the light stripe feature points, used to characterize the positional and / or orientational relationship of the inspection robot relative to the wave-shaped guardrail.

[0084] In this embodiment of the invention, the steering amount may include an amount used to adjust the travel direction of the inspection robot towards a preset orientation, and the differential control amount may include a control amount used to create a speed difference between the drive mechanisms that drive different walking parts respectively, thereby achieving steering or tracking. In this embodiment of the invention, the preset orientation, preset position, and preset reference line may respectively include a pre-set orientation and position as a control target or judgment benchmark, and a reference line used to cooperate with the determined features to determine the anchor point. However, it is understood that the specific forms of the above terms do not constitute a limitation of the invention.

[0085] As mentioned earlier, to reduce the burden of manual inspections, safety inspection robots capable of traveling along corrugated guardrails are typically used to assist in long-distance, continuous inspections. For this, the safety inspection robot needs to move long distances continuously along the path defined by the guardrail while maintaining its relative position. Therefore, to ensure the safety inspection robot can travel stably along the corrugated guardrail, its trajectory needs to be continuously adjusted during travel to maintain an appropriate relative position to the guardrail.

[0086] In response, this invention recognizes that wave-shaped guardrails have undulating wave-like surfaces with relatively complex surface morphology. However, there is currently a lack of control schemes for long-distance travel of inspection robots. Known technologies simply drive the walking wheels to move the inspection robot, and such simple solutions are insufficient to cope with the complex environment faced by inspection robots during long-distance travel, and cannot guarantee the stable long-distance operation of the inspection robot.

[0087] In response, this invention further recognizes that the texture of the corrugated guardrail surface itself is not significant in some situations, and the lighting conditions in actual working environments are often less than ideal. Furthermore, when these conditions overlap, relying solely on the information naturally presented by the corrugated guardrail surface itself is insufficient to reliably obtain its geometric features, making it impossible to simply use traditional visual processing methods to obtain sufficient evidence to support adjustments in travel orientation. As an explanation, this invention further recognizes that when the texture of the corrugated guardrail surface is weak or the lighting conditions are unfavorable, the geometric features extracted from the surface information are not stable enough. This is because visual processing methods rely on the texture or brightness information presented by the corrugated guardrail surface itself, and this information is itself insufficient or unstable in the aforementioned situations, causing fluctuations in the obtained geometric features. For example, the corrugated guardrail surface may also have protrusions such as rivets in certain areas, and variations in shape such as tilting in different road sections. Explained, but not limited to, these local protrusions and shape variations may also affect the stability and posture maintenance of the mobile platform, thus increasing the difficulty of obtaining visual information.

[0088] To address this issue, this invention provides a driving scheme for an inspection robot that effectively alleviates or overcomes the aforementioned problems. The scheme actively projects structured light onto the wave-shaped guardrail to form significant reflected light stripes on its surface, transforming the relatively complex and less prominent geometric features of the guardrail surface into more easily identifiable geometric shapes of reflected light stripes. Furthermore, the processing is focused on the region of interest where the reflected light stripes are located. Stable anchoring features are determined by the geometric distribution of the light stripe feature points, and the steering amount is determined in a closed-loop manner within the pixel domain. The steering amount is then used to determine the differential speed control amount of the driving mechanism in real time to drive the inspection robot, thereby ensuring the stability and reliability of the inspection robot's movement along the wave-shaped guardrail.

[0089] The following describes in detail the inspection robot driving method and exemplary inspection robot driving device according to embodiments of the present invention with reference to the accompanying drawings.

[0090] In some embodiments, reference Figure 1 This image shows a schematic side view of a wave-shaped guardrail for use with a structured light recognition-based security inspection robot driving method according to an embodiment of the present invention. In this embodiment, the wave-shaped guardrail L is arranged along its extending direction and supported by guardrail posts P, and the surface of the wave-shaped guardrail L has an undulating wave shape along its vertical extending direction. In some embodiments, the wave-shaped guardrail L includes a multi-wave guardrail with multiple wave structures. Figure 1 In the illustrated embodiment, the wave-shaped guardrail L comprises three wave structures and may be referred to as a three-wave guardrail. In some embodiments, the inspection robot can abut and remain on the surface of the wave-shaped guardrail L in a unilateral manner, travel along the longitudinal direction of the wave-shaped guardrail L, and perform tasks such as road inspection, environmental detection, and information collection during its travel.

[0091] In some embodiments, the inspection robot to which the driving method of the present invention can be applied may include, in addition to the structured light projector and image acquisition device, at least two other components, such as two traveling sections, at least two driving mechanisms that independently drive the at least two traveling sections, a magnetic holding mechanism, an additional holding mechanism, and an adjustable suspension mechanism. In this embodiment, each traveling section (e.g., each traveling section) may include a traveling section extending along the direction of the corrugated guardrail (e.g., the direction of road extension). In some embodiments, each driving mechanism may correspond one-to-one with each traveling section, and each driving mechanism may include its own independent and independently controllable actuator, thereby enabling the application of mutually independent speed control amounts to the at least two traveling sections, creating a speed difference between the at least two traveling sections to achieve multi-channel independent differential speed control. It is understood that, compared to the case of driving by a single driving channel, the above-mentioned multi-channel independent differential speed arrangement can provide more flexible control freedom for the inspection robot's steering and attitude adjustments when traveling along the corrugated guardrail.

[0092] In some embodiments, the magnetic holding mechanism can be configured to magnetically hold the inspection robot to the surface of the corrugated guardrail. In some embodiments, an additional holding mechanism can be configured to be triggered when the holding state of the inspection robot relative to the corrugated guardrail becomes abnormal, so as to apply additional holding action to the inspection robot. In some embodiments, the adjustable suspension mechanism can be configured to adjust the height of the robot body relative to the surface of the corrugated guardrail so as to maintain an appropriate posture of the robot body relative to the corrugated guardrail when the robot body is tilted or otherwise changes shape relative to the surface of the corrugated guardrail.

[0093] In some embodiments, reference Figure 6 The diagram illustrates the projection of structured light and the acquisition of images of a guardrail surface using a method according to an embodiment of the present invention. In this embodiment, a structured light projector 110 is configured to project structured light onto the guardrail surface of a wave-shaped guardrail L, the structured light being reflected on the guardrail surface of the wave-shaped guardrail L to form reflected light stripes 130, and an image acquisition device 120 is configured to acquire images of the guardrail surface including the reflected light stripes 130.

[0094] In some embodiments, the structured light projector 110 and the image acquisition device 120 can be positioned in a preset relative relationship, such that the reflected light strip 130 formed on the surface of the wave-shaped guardrail L falls within the field of view of the image acquisition device 120. In some embodiments, continuing to refer to... Figure 6 For a point Q on the surface of the wave-shaped guardrail, its coordinates in the camera coordinate system of the image acquisition device 120 can be denoted as Q(x). c , y c , z c After imaging, it corresponds to the imaging point q on the image plane, and its image coordinates can be denoted as q(x, y). The projection relationship between the two is determined by the camera coordinate system of the image acquisition device 120 (in the image plane). Figure 6 China and Israel O c With X as the origin c Y c Z c The coordinate system shown for the coordinate axes), the image coordinate system (in... Figure 6 The pixel coordinate system (with O0 as the origin and U and V as the coordinate axes, and / or the image physical coordinate system (with O1 as the origin and X and Y as the coordinate axes) and parameters such as focal length f are determined. Thus, the geometry of the surface of the wave-shaped guardrail L is mapped to the geometry of the reflected light strip 130 in the guardrail surface image, providing a geometric basis for subsequently determining the lateral pixel deviation of the anchoring feature relative to the preset position within the image coordinate system.

[0095] In some embodiments, the structured light projector 110 may be configured to project structured light in the form of a light plane onto the wave-shaped guardrail L, such that the light plane intersects with the surface of the wave-shaped guardrail L, thereby forming the reflected light strip 130 on the surface of the wave-shaped guardrail L. For explanation, the reflected light strip 130 may thus exhibit a corresponding geometric shape along the undulations of the surface of the wave-shaped guardrail L.

[0096] In some embodiments, the structured light projector 110 may include a laser and optical elements for shaping the laser beam emitted by the laser, the optical elements being configured to spread (fan) the laser beam into a light plane in a predetermined direction. In some embodiments, the optical elements may include, but are not limited to, line-generating elements such as cylindrical lenses and Powell prisms. In another alternative embodiment, the structured light projector 110 may also include a laser and a scanning mechanism for scanning the laser beam along a predetermined direction, which forms an equivalent light plane by reciprocating the laser beam along the predetermined direction. The present invention does not limit the specific manner in which the structured light projector 110 forms the light plane; any form that can form a reflective light strip 130 that can be collected on the surface of the wave-shaped guardrail L falls within the protection scope of the present invention.

[0097] In some embodiments, the structured light projector 110 may include a near-infrared laser projector. By way of explanation and not limitation, using a near-infrared laser projector makes the projected structured light less susceptible to interference from the visible light environment, thereby enabling the formation of reflective light stripes with good contrast on the surface of the wave-shaped guardrail under different lighting conditions. Accordingly, the image acquisition device 120 may employ an image sensor sensitive to the near-infrared band to acquire an image of the guardrail surface containing the reflective light stripes. In an optional embodiment, the image acquisition device 120 may also be equipped with a filter matching the wavelength of the structured light to filter out ambient light and highlight the reflective light stripes.

[0098] In other embodiments, the image acquisition device 120 for acquiring images of the guardrail surface may further include a binocular stereo vision system consisting of a camera and a structured light projector. In this embodiment, the image acquisition device 120 can acquire images of the guardrail surface including the reflected light strips 130 while simultaneously obtaining three-dimensional information (e.g., three-dimensional point cloud) of the surface of the wave-shaped guardrail L, which falls within the protection scope of this invention.

[0099] In some embodiments, reference Figure 7 A schematic photograph of a guardrail surface image including reflective light stripes 130 is shown. In this embodiment, the reflective light stripes 130 in the guardrail surface image appear as bright stripes undulating along the extension direction of the wavy guardrail L. In this embodiment, in addition to the reflective light stripes 130 themselves, there are various interfering factors in the guardrail surface image, such as... Figure 7The waveform guardrail L shown in the image exhibits surface highlights 141, background interference 142, specular reflection of the light stripe 143, and high reflectivity of the light stripe portion 144. To explain, surface highlights 141 and background interference 142 create bright pixels in areas outside the reflected light stripe, while specular reflection of the light stripe 143 and high reflectivity of the light stripe portion 144 cause uneven brightness distribution within the reflected light stripe itself. It is understandable that directly extracting features of the reflected light stripe 130 from the entire guardrail surface image is easily affected by the aforementioned interference. Therefore, this embodiment of the invention focuses processing on the region of interest where the reflected light stripe 130 is located, extracts the light stripe feature points of the reflected light stripe 130 within the region of interest, and then determines the anchoring features, thereby reducing the impact of the aforementioned interference on the anchoring results, which will be described in detail below.

[0100] It is understood that, in the embodiments of the present invention, the inspection robot that can be driven by the driving method of the present invention can adopt a variety of structural forms, as long as it has a structure light projector 110 for projecting structure light onto the wave-shaped guardrail, an image acquisition device 120 for acquiring images of the guardrail surface, and a walking part required for traveling along the wave-shaped guardrail and a driving mechanism that independently drives each walking part. The present invention does not limit this without contradicting the spirit of the embodiments of the present invention.

[0101] In some embodiments, reference Figure 2 The inspection robot driving method according to the first aspect of the present invention may include the following steps S210 to S280. In some embodiments, the above steps S210 to S280 may be executed cyclically as the inspection robot travels along the wave-shaped guardrail.

[0102] S210: Project structured light onto the corrugated guardrail to form a reflective light strip on the surface of the guardrail.

[0103] In some embodiments, structured light can be projected onto the wave-shaped guardrail using a structured light projector. In some embodiments, the structured light projector may include, but is not limited to, an infrared laser projector. In one specific embodiment, through the aforementioned... Figure 6 The structured light projector projects laser light onto a wave-shaped guardrail with two waveforms. In the above embodiment, a detailed description of the structured light projector can be found in the foregoing description of... Figure 6 The specific details are not repeated here.

[0104] In some embodiments, actively projected structured light can be reflected on the surface of the wave-shaped guardrail to form reflected light stripes. As an explanation, and not a limitation, since the surface of the wave-shaped guardrail is undulating along its extension direction, the reflected light stripes formed by the structured light will exhibit corresponding geometric shapes according to the undulations of the guardrail surface. Therefore, the information represented by the geometric shape of the reflected light stripes can be used to determine the orientation of the wave-shaped guardrail. Furthermore, the method of the above embodiments of the present invention, by actively projecting structured light, further achieves the stable presentation of the geometric shape of the wave-shaped guardrail even under conditions where the guardrail surface has weak texture or unfavorable lighting.

[0105] S220: Acquire images of the guardrail surface containing reflected light stripes.

[0106] In some embodiments, an image acquisition device can continuously acquire images of the guardrail surface. In this embodiment, the guardrail surface image may include reflected light stripes formed by actively projected structured light. In a specific embodiment, through the aforementioned... Figure 6 The camera in the middle is used to capture images of the guardrail surface containing reflected light stripes. In one specific embodiment, refer to... Figure 7 The image shown is a captured image of the guardrail.

[0107] In some embodiments, where a structured light sensor capable of outputting a 3D point cloud is used, the acquired 3D point cloud can be projected onto an imaging plane to obtain a guardrail surface image. The 3D point cloud can then be fitted to a plane using algorithms such as, but not limited to, RANSAC, to extract the 3D features of the wave-shaped guardrail. In this embodiment, the extracted 3D features can serve as a substitute or supplement for subsequently determining anchoring features. However, it is understood that in other embodiments, other methods can be reasonably employed to acquire guardrail surface images containing reflected light stripes, all of which fall within the protection scope of this invention.

[0108] S230: Process the guardrail surface image to determine the region of interest in the reflected light stripe within the guardrail surface image.

[0109] In some embodiments, the acquired guardrail surface image may be processed, such as but not limited to image recognition and image segmentation, to determine a region of interest including the reflected light stripes. For explanation, the determined region of interest can be used for subsequent processing such as light stripe feature point extraction, thereby avoiding, for example, the aforementioned... Figure 7 The interference from background areas such as highlights and background noise on the surface of the guardrail ensures the processing accuracy of subsequent processing such as light stripe feature point extraction.

[0110] In some embodiments, a pre-trained machine learning model can be used to determine and segment the region of interest (ROI) containing reflected light stripes from the guardrail surface image. In some embodiments, object detection models, including but not limited to YOLO and Faster R-CNN, can be used to detect and determine the bounding box of the ROI, thereby determining the ROI containing the reflected light stripes. However, it is understood that in other embodiments, the ROI can also be determined by a multimodal model with object localization capabilities combined with preset reflected light stripe description information. This invention does not limit the specific means of determining the ROI.

[0111] In one specific embodiment, in conjunction with reference to the reference Figure 7 and Figure 8 ,in, Figure 8 It shows the Figure 7 The diagram shows the region of interest obtained by processing the guardrail surface image. In this region of interest, the reflected light stripe 130 is highlighted, while the surface highlights 141 and background interference 142 of the wave-shaped guardrail L are excluded from the region of interest. This image can be used for subsequent processing such as light stripe feature point extraction.

[0112] S240: Extract the light stripe feature points of the reflected light stripe within the region of interest.

[0113] In some embodiments, light stripe feature points can be extracted within the aforementioned defined region of interest. In these embodiments, the light stripe feature points include feature points capable of characterizing the position and / or shape of the reflected light stripe.

[0114] In some embodiments, the aforementioned light stripe feature points can be extracted from the reflected light stripes within the region of interest using methods such as, but not limited to, grayscale centroid extraction, gradient extremum extraction, or centerline extraction. By way of explanation and not limitation, limiting the extraction process to the region of interest reduces the computational load required for extraction, decreases background interference, and improves extraction accuracy.

[0115] In an optional embodiment, the step of extracting the feature points of the reflected light stripe within the region of interest may include: determining the ridge line of the reflected light stripe; and determining the feature points of the light stripe based on the ridge line. In one example, the ridge line of the reflected light stripe is determined based on the gray-level centroid. Specifically, the brightness distribution of the reflected light stripe at each position is determined sequentially along the direction perpendicular to the extension of the reflected light stripe, and the brightness weighted centroid (i.e., gray-level centroid) of the cross-section of the light stripe at that position is determined as the center point of the reflected light stripe at that position. Subsequently, the center points at each position along the extension direction of the reflected light stripe are connected sequentially to obtain a ridge line with convergent width that characterizes the position and shape of the reflected light stripe. In an optional embodiment, the ridge line is a single-pixel ridge line, that is, the line width of the ridge line is one pixel. As a specific example, refer to the reference... Figure 8 and Figure 9 , Figure 9 It shows the Figure 8 The reflected light stripe ridge 135 obtained by processing the region of interest shown can be used as a basis for extracting light stripe feature points.

[0116] In some embodiments, the wave-shaped guardrail includes a multi-wave-shaped guardrail. In a specific example, referring to the reference... Figures 7 to 9 The wave-shaped guardrail is a triple-wave guardrail. In some embodiments, the reflected light strip 130 includes a multi-period light strip formed on the surface of the multi-wave guardrail. In this embodiment, the steps for extracting the light strip feature points of the multi-period light strip will be described in detail below.

[0117] S250: Anchoring features are determined based on the geometric distribution of light stripe feature points.

[0118] In some embodiments, anchoring features can be determined based on the geometric distribution of the extracted light stripe feature points. In this embodiment, the anchoring features can be used to characterize the positional and orientational relationship of the inspection robot relative to the wave-shaped guardrail. In some embodiments, the anchoring features may include anchoring points, which can be used as positioning references.

[0119] As an explanation, and not a limitation, since the image acquisition device is fixedly mounted on the inspection robot, the imaging position of the reflected light strip and the anchoring features (anchor points) determined therefrom in the guardrail surface image depends on the inspection robot's position and orientation relative to the wave-shaped guardrail. Therefore, when the inspection robot maintains the expected position and orientation, the anchor point is located at a preset position in the guardrail surface image; when the inspection robot shifts laterally or changes orientation relative to the wave-shaped guardrail, the anchor point deviates from the preset position accordingly. Furthermore, the difference between the inspection robot's current position and orientation and the expected position and orientation can be determined based on the anchor point and the preset position. This difference can serve as the basis for driving control of the inspection robot, which will be described in detail below.

[0120] As previously mentioned, the wave-shaped guardrail may include a multi-wave-shaped guardrail, the surface of which may have multiple undulation periods, i.e., multiple waveform structures, and thus multiple peaks and troughs. In some embodiments, the reflected light strip may include a multi-periodic light strip formed on the surface of the multi-wave-shaped guardrail.

[0121] In some embodiments, protruding portions (peaks) and recessed portions (troughs) corresponding to waveform undulations can be identified in the multi-period light stripe, and light stripe feature points are determined at these portions for further determination of anchoring features. In this embodiment, it is understood that peaks and troughs are used only to characterize the relative convex-concave relationship of the multi-period light stripe along its extension direction, rather than its absolute spatial orientation. The portions referred to as peaks and troughs can be interchanged under different viewing directions or reference datums.

[0122] In some embodiments, step S240 may include steps A1 and / or A2. A1: Identifying multiple peak feature points of the multi-period light stripe within the region of interest. A2: Identifying multiple trough feature points of the multi-period light stripe within the region of interest.

[0123] In some embodiments, the peak feature points and / or trough feature points can be determined by identifying the ridge lines of the multi-period light stripes. A detailed description of the ridge lines is provided in the above embodiments and will not be repeated here. However, it is understood that in other embodiments, the ridge lines can also be directly identified in the multi-period light stripes (e.g., Figure 8 The peak feature points and / or trough feature points are identified on the multi-period light stripe 130 shown. This embodiment of the invention does not limit this.

[0124] In some embodiments, the lateral deviation of the reflected light strip relative to the overall extension direction (or preset reference direction) of the reflected light strip can be detected along the extension direction of the reflected light strip (or optionally along the ridge line of the reflected light strip). The position where the deviation towards the convex side reaches a local maximum can be determined as the peak feature point, and the position where the deviation towards the concave side reaches a local minimum can be determined as the trough feature point.

[0125] In one specific embodiment, the first-order difference (i.e., slope) of the lateral coordinates of the light stripe ridge is calculated along its extension direction. The location where the lateral deviation changes from increasing to decreasing (slope crossing zero) is determined as a peak feature point, and the location where the lateral deviation changes from decreasing to increasing is determined as a trough feature point. In other embodiments, the local extreme values ​​of the curvature of the light stripe ridge can also be determined as the corresponding peak or trough feature points, which falls within the protection scope of this invention. In one specific embodiment, referring to... Figure 9 The figure shows four peak feature points and six trough feature points obtained from the identification of multi-period light stripes.

[0126] Accordingly, after identifying multiple peak feature points and / or multiple trough feature points of the multi-period light stripe, the above step S250 may further include the following step A3. A3: Determine a single anchor point based on the geometric relationship between the multiple peak feature points and / or multiple trough feature points.

[0127] In some embodiments, the overall trend of multi-period light stripes can be determined based on multiple peak feature points and / or multiple trough feature points, thereby determining the anchor point. Accordingly, in some embodiments, step A3 above may specifically include steps B1 and B2 below.

[0128] B1: Determine the centerline of the multi-period light stripe based on multiple peak feature points and / or multiple trough feature points.

[0129] In some embodiments, the centerline, distinct from the aforementioned ridge lines that characterize the specific undulations of the light stripes point by point, includes lines used to represent the overall orientation of multi-period light stripes. In some embodiments, the centerline is a straight line segment located within the region of interest.

[0130] In some embodiments, a centerline representing the overall attitude of multiple peaks can be obtained from multiple peak feature points. In other embodiments, a centerline representing the overall attitude of multiple troughs can also be obtained from multiple trough feature points. In optional embodiments, the centerline can be determined by combining multiple peak feature points and multiple trough feature points.

[0131] However, it is understood that this invention does not limit the specific method, form, or orientation of the centerline. In some embodiments, using... Figure 8 and Figure 9 Taking the rectangular region of interest shown as an example: In one embodiment, the generally horizontal centerline S1 can be determined based on the centerline obtained from four wave crest feature points; in another embodiment, the generally vertical centerline is determined by combining four wave crest feature points 132 and six wave trough feature points 134. However, it is understood that the above-mentioned horizontal / vertical orientation varies depending on the view's placement and the actual direction of the wave-shaped guardrail, and does not constitute a limitation of the present invention. Furthermore, the present invention does not impose specific restrictions on the specific orientation of the centerline determined based on different feature points. For example, in the embodiment where the centerline S1 is determined by combining four wave crest feature points 132 and six wave trough feature points 134, the centerline S1 can also be used as an intermediate result, and the perpendicular bisector of the centerline S1 can be further taken as the final determined centerline. This falls within the protection scope of the embodiments of the present invention.

[0132] B2: Determine the anchor point based on the position where the centerline of the multi-period light strip intersects with the preset reference line in the guardrail surface image.

[0133] In some embodiments, preset reference lines corresponding to different centerline determination methods may be pre-defined. In this embodiment, the intersection of the centerline and the preset reference line can be used as an anchor point. As an explanation and not a limitation, referring to the foregoing description, the final general orientation of the centerline will differ for different centerline determination methods, and this embodiment of the invention explicitly allows for such differences. Accordingly, the preset reference lines may also include preset reference lines with different general orientations, such as generally horizontal or vertical, which falls within the protection scope of this embodiment of the invention. In a specific embodiment, in conjunction with reference to... Figure 8 and Figure 9 , Figure 9 The example shows a preset reference line S0 and an anchor point E determined by the preset reference line S0 and the center line S1.

[0134] In alternative embodiments, the anchoring point can also be determined based on the overall geometric distribution of multiple peak feature points and / or multiple trough feature points. Accordingly, in some embodiments, step A3 above may include step B3 below.

[0135] B3: Determine the geometric midpoint of multiple peak feature points and / or multiple trough feature points as the anchor point.

[0136] In some embodiments, the geometric midpoint may include the point corresponding to the average of the position coordinates of multiple feature points in the image coordinate system of the guardrail surface image. In other words, the geometric midpoint may include the centroid of the point set composed of the aforementioned feature points. For illustrative purposes and not as a limitation, compared to determining the anchor point by intersecting the centerline of a multi-period light stripe with a preset reference line, using the geometric midpoint of multiple peaks and / or troughs as the anchor point eliminates the need to separately determine the centerline and preset reference line, simplifying the process and improving the speed of anchor point determination.

[0137] By way of explanation and not limitation, the method of the above embodiments of the present invention condenses multiple feature points dispersed on multiple periods into a single anchor point. This not only enables the stable determination of the anchor point when the reflected light stripe is a multi-period light stripe, but also facilitates the subsequent determination and correction of deviations in a uniform manner, and can suppress the influence of random errors of individual feature points on the anchoring results.

[0138] In some embodiments, the captured image of the guardrail surface reflecting the light stripe may include consecutive video frames. Accordingly, in some embodiments, step S220 may also include the step of capturing the guardrail surface image in the form of consecutive frames. In this embodiment, the image acquisition device may include a video camera or an industrial area scan camera, etc., capable of outputting images in the form of consecutive frames (video stream).

[0139] Furthermore, as mentioned above, steps S210 to S280 can be executed cyclically. In this regard, the embodiments of the present invention further recognize that, when the acquired images of the guardrail surface containing reflected light stripes are continuous frames, in addition to using a method for single-frame images, the temporal continuity between adjacent frames can be further utilized to determine the region of interest for each frame through frame-by-frame tracking.

[0140] Accordingly, in some embodiments, step S230 may include steps C1 and C2 as described below.

[0141] C1: For the guardrail surface image of the initial frame, determine the region of interest of the initial frame according to the preset projection direction of the structured light.

[0142] In some embodiments, the preset projection direction may include the pre-calibrated emission direction of the structured light projector relative to the image acquisition device. By way of explanation and not limitation, since the emission direction is pre-calibrated and relatively fixed, the area where the reflected light stripe appears in the guardrail surface image is correspondingly predictable, which can help estimate the area where the reflected light stripe may appear in the initial frame, thereby improving the efficiency of determining the region of interest.

[0143] C2: For guardrail surface images that are not in the initial frame, determine the region of interest for the current frame based on the position of the anchoring features determined by the guardrail surface image of the previous frame.

[0144] In some embodiments, the results determined in a previous frame (e.g., the previous frame), such as the position of anchoring features, can be used as a reference for determining the region of interest (ROI) in the current frame, thereby recursively determining the ROI of the current frame sequentially along consecutive frames. By way of explanation and not limitation, the acquisition interval between adjacent frames is often short, and the displacement of the inspection robot between adjacent frames is limited, resulting in relatively small changes in the position of the reflected light stripe between adjacent frames. Therefore, the position of the ROI determined in the previous frame can be used as a reference for determining the ROI of the current frame.

[0145] By way of explanation and not limitation, the method of the above embodiments of the present invention enables the processing between adjacent frames to no longer be isolated from each other, but to guide the processing of the current frame with the help of the positioning information obtained in the previous frame. On the one hand, it is not necessary to perform a full image search again for each frame, but the region of interest can be determined near the region of interest determined in the previous frame, thereby helping to reduce processing overhead and improve real-time performance. On the other hand, since the reference comes from the previously acquired frame, it can reflect the current actual direction of the wave guardrail, so the region of interest can dynamically follow the change of the wave guardrail direction, thereby improving the robustness of tracking and maintaining stable positioning of the reflected light strip even when the guardrail is bent or the robot's posture changes.

[0146] This invention further recognizes that while the method of determining the region of interest by tracking frame by frame based on previous frames improves efficiency, it also creates a dependency between the processing results of each frame: once a frame fails to correctly track the reflected light strip due to interference, its erroneous anchoring features will be passed down as a reference for determining the region of interest in subsequent frames, which may propagate and even accumulate along consecutive frames. Inspection robots often face multiple interference factors while moving along wave-shaped guardrails, such as, but not limited to: debris obscuring the reflected light strip, strong reflections from the surface of the wave-shaped guardrail in outdoor strong light conditions, and body vibrations caused when crossing guardrail rivets or joints. These factors may cause abnormal positional changes in the reflected light strip of a frame, or even the light strip feature points or anchoring features determined based on it, between adjacent frames, thus triggering the aforementioned erroneous tracking and its cross-frame propagation.

[0147] In some embodiments, the inspection robot driving method may further include a step of determining the positional jitter of anchor features or light stripe feature points in consecutive frames.

[0148] In some embodiments, the Euclidean distance between the anchor feature positions in the current frame and the previous frame can be determined as the position jitter. In this embodiment, the position jitter can be used to characterize the drastic degree of position change of anchor features or light stripe feature points between adjacent frames.

[0149] In some embodiments, when multiple light bar feature points are used, the displacement of each light bar feature point between adjacent frames can be calculated separately, and the average value, root mean square value, or maximum value can be taken as the position jitter. However, it is understood that in other embodiments, other methods can be reasonably used to calculate the position jitter, and the embodiments of the present invention do not limit this.

[0150] In some embodiments, tracking loss can be determined when the position jitter exceeds a preset tolerance. As an explanation, since the position change of anchor features between adjacent frames is usually small, if there is a sudden change in the position of anchor features that exceeds the preset tolerance, it often means that the reflected light strip has failed to be tracked correctly due to the aforementioned obstruction, strong reflection, or rivets, joints, etc., that is, tracking loss has occurred.

[0151] In some embodiments, the preset tolerance may include a pre-set fixed threshold. In optional embodiments, the preset tolerance may also include an adaptive threshold, such as taking a preset multiple of the statistical values ​​(e.g., mean or median) of the jitter in previous frames as the preset tolerance. This allows the lock-out determination to adapt to normal jitter levels under different travel speeds and different acquisition frame rates, reducing the probability of false positives.

[0152] S260: Determine the amount of steering that causes the inspection robot to turn toward the preset orientation based on the deviation of the anchoring feature from the preset position.

[0153] In some embodiments, the preset position may include the position of the anchoring feature (e.g., anchor point) in the image coordinate system of the guardrail surface image when the inspection robot is in a desired position and orientation relative to the wave-shaped guardrail. In one specific embodiment, the preset position is the center of the guardrail surface image. However, it is understood that those skilled in the art can reasonably select the preset position according to the specific structure of the inspection robot and the different usage environments. For example, the anchor point when the inspection robot maintains the desired lateral distance on the specific guardrail and is generally parallel to the direction of the wave-shaped guardrail can be selected as the preset position, and the present invention does not limit this.

[0154] In some embodiments, the steering amount to orient the inspection robot toward a preset orientation can be determined based on the deviation of the anchor feature relative to a preset position, more specifically, for example, but not limited to, the deviation of the anchor feature relative to the preset position in the image coordinate system of the guardrail surface image or the region of interest. By way of explanation and not limitation, the preset position can be understood as the control target for closed-loop correction: when the anchor feature is in the preset position, it can be considered that the inspection robot is in the desired pose; when the anchor feature deviates from the preset position, the aforementioned deviation can be considered as the offset of the inspection robot's pose relative to the desired pose, and the steering amount can be determined accordingly to pull the anchor feature back to the preset position, thereby causing the inspection robot to return to and remain in the preset orientation and desired pose.

[0155] In some embodiments, reference Figure 3 Step S260 may include the following steps S310 and S320.

[0156] S310: Determine the lateral pixel deviation of the anchoring feature relative to the preset position in the image coordinate system of the guardrail surface image.

[0157] In some embodiments, the lateral pixel deviation (pixel distance) of the anchoring feature relative to a preset position can be determined in the image coordinate system of the guardrail surface image.

[0158] S320: Uses lateral pixel deviation as a control error input feedback controller to determine the steering amount.

[0159] In some embodiments, lateral pixel deviation can be used as a control error for feedback control. In this embodiment, the control error can be used to characterize the direction and degree of deviation of the current position of the anchoring feature in the guardrail surface image relative to a preset position. In this embodiment, the feedback controller can output a steering amount based on the control error.

[0160] In some embodiments, the steering amount may include the direction and magnitude for characterizing the inspection robot's steering correction toward a preset orientation. In some embodiments, the steering amount may include yaw angle correction, desired angular velocity, travel curvature correction, or intermediate control amounts for determining the speed difference between the left and right traveling parts, etc., and the embodiments of the present invention are not limited thereto.

[0161] In some embodiments, the feedback controller may include a proportional-integral-derivative (PID) controller. For explanation, the feedback controller can perform proportional, integral, and derivative operations on the control error to obtain the steering input, thereby achieving better correction speed and stability. However, it is understood that the feedback controller may also include other types of controllers, and the embodiments of the present invention do not limit the specific form of the feedback controller.

[0162] In some embodiments, the drive mechanism can be driven according to the differential speed control value, so that the inspection robot moves along the wave-shaped guardrail, causing the lateral pixel deviation to converge to within a preset pixel threshold. In this embodiment, the preset pixel threshold can be reasonably determined according to specific accuracy requirements, and the embodiments of the present invention do not limit it in this regard.

[0163] By way of explanation and not limitation, the method of the above embodiments of the present invention uses the lateral pixel deviation of the anchor feature relative to a preset position in the image coordinate system as the control error input to the feedback controller, and the steering amount is obtained by the output of the feedback controller, so that the control error tends to decrease under closed-loop control, thereby enabling the inspection robot to continuously adjust towards the preset orientation, so that the inspection robot can travel stably over long distances.

[0164] S270: Determines the differential control amount of the drive mechanism in real time based on the steering input.

[0165] In some embodiments, the inspection robot may include one or more drive mechanisms. In this embodiment, the differential control amount of the drive mechanism can be determined in real time based on the steering input. In some embodiments, the differential control amount may include a control amount for creating speed differences between different drive mechanisms of the inspection robot or between different outputs of the same drive mechanism, thereby enabling the inspection robot to perform steering or tracking adjustments.

[0166] In some embodiments, the drive mechanism may include components for providing a driving force (e.g., driving torque) to a corresponding traveling part to drive the traveling part to move. In some embodiments, the traveling part may include components for contacting the surface of the corrugated guardrail and driving the inspection robot to travel along the corrugated guardrail. The traveling part is, for example, but not limited to, wheels or wheel sets, etc. The specific form of the traveling part is not limited in the embodiments of the present invention.

[0167] In some embodiments, the drive mechanism can transmit the power output by the drive to the corresponding traveling part through a transmission mechanism disposed between the drive and the traveling part.

[0168] In some embodiments, the inspection robot may include at least two traveling parts and at least two drive mechanisms that independently drive each traveling part. In this embodiment, step S280 may include: obtaining multi-channel independent speed control quantities based on the decomposition of the steering amount, wherein the multi-channel independent speed control quantities may correspond one-to-one with the at least two drive mechanisms.

[0169] In some embodiments, based on the basic travel speed of the inspection robot, a corresponding speed change can be determined for each channel (i.e., each traveling part corresponding to each drive mechanism) according to the steering amount, and this speed change is combined with the basic travel speed to obtain an independent speed control amount corresponding to each channel. For explanation, the direction (increase or decrease) and magnitude of each speed change can be determined by the steering amount, so that a speed difference is generated between the traveling parts located on different sides.

[0170] The present invention further recognizes that the method of determining the steering amount based on the deviation of the anchoring feature relative to the preset position is a feedback adjustment. When the direction of the wave guardrail changes (e.g., the guardrail bends), it often needs to be corrected only after the deviation occurs, which may cause tracking lag at the point of change of direction and thus affect the smoothness of the inspection robot's driving.

[0171] In some embodiments, the curvature change of the forward-facing wave-shaped guardrail can be determined based on the changes in the anchoring feature positions in consecutive frame images of the guardrail surface. A feedforward compensation amount is then determined accordingly, and the steering amount is superimposed with the feedforward compensation amount to determine the differential control amount. By way of explanation and not limitation, the method of the above embodiments estimates the forward curvature and determines the corresponding feedforward compensation amount; in other words, it pre-applies an adjustment amount adapted to the forward directional change before the deviation has fully occurred. This allows the feedback-derived steering amount to be superimposed with the feedforward compensation amount, enabling the inspection robot to adjust its travel direction accordingly before reaching the point of directional change, thereby reducing tracking lag and allowing the inspection robot to travel more smoothly and closely along the curved wave-shaped guardrail.

[0172] In another alternative embodiment, the load of the drive mechanism can also be monitored, and when the load exceeds a preset load threshold, the differential control amount can be adjusted so that the inspection robot can smoothly pass through the height difference of the wave-shaped guardrail.

[0173] S280: Drives the drive mechanism according to the differential speed control value, so that the inspection robot moves along the wave-shaped guardrail.

[0174] In some embodiments, the drive mechanism can be driven according to the determined differential control amount, so that the traveling part moves at the corresponding speed, thereby enabling the inspection robot to travel along the wave-shaped guardrail.

[0175] In some embodiments, as previously described, multi-channel independent speed control quantities can be obtained from the decomposition of the steering amount. In this embodiment, each of the aforementioned independent speed control quantities can be applied to its corresponding drive mechanism, causing each drive mechanism to independently drive its corresponding traveling part according to the applied speed control quantity. For explanation, this allows each traveling part to move at a corresponding speed and form the required speed difference, thereby enabling the inspection robot to turn or follow a line along the wave-shaped guardrail according to the steering amount.

[0176] The present invention further recognizes that the surface of corrugated guardrails is often not smooth and continuous, but may have local protrusions or height variations. These protrusions or height variations can cause the inspection robot to shake when it passes through these areas, thereby affecting its tracking stability and even causing it to fall off the corrugated guardrail. To explain, corrugated guardrails are connected to posts using guardrail rivets, and long-distance corrugated guardrails are made up of multiple corrugated guardrail segments spliced ​​together (adjacent segments are also fixed together using guardrail rivets), thus having guardrail rivets that protrude relative to the panel surface; at the same time, the joints (e.g., overlaps) of adjacent corrugated guardrail segments form stepped joints due to height differences; in addition, corrugated guardrails are susceptible to local deformation such as bending and unevenness due to external forces (e.g., collisions, compression).

[0177] In response, this invention recognizes that the aforementioned local protrusions, steps, or distortions can cause the reflected light strip to break or distort at the corresponding location. Therefore, embodiments of this invention specifically utilize the correspondence between the aforementioned local protrusions and abnormalities in the reflected light strip. By identifying abnormalities in the reflected light strip (e.g., light strip breakage, light strip distortion), the location of guardrail rivets and other components is determined, and the movement of the inspection robot is adjusted accordingly to ensure that the inspection robot smoothly passes through the areas containing guardrail rivets, joints, etc.

[0178] Accordingly, in some embodiments, reference is made to Figure 4 The inspection robot driving method may further include the following steps S410 to S430.

[0179] S410: Identify whether a given light stripe anomaly exists in the reflected light stripe in the guardrail surface image.

[0180] In some embodiments, light stripe anomalies include light stripe breakage at a given location and / or light stripe distortion at a given location relative to a predetermined contour. By way of explanation and not limitation, structures such as guardrail rivets may cause the reflected light stripe at that location to break, or cause the reflected light stripe to bulge, step, misalign, or otherwise distort relative to its normal contour.

[0181] S420: When an abnormality in the light stripe is detected, the position of the guardrail rivet is determined based on the position of the abnormality in the guardrail surface image.

[0182] In some embodiments, the position of the guardrail rivet can be determined based on the image coordinates corresponding to, but not limited to, the midpoint between the two ends of the broken light strip in the guardrail surface image, or the peak of the light strip distortion.

[0183] In some embodiments, the position of the guardrail rivet may include the distance of the guardrail rivet from the inspection robot along the direction of travel of the inspection robot. As explained above, the image acquisition device is fixedly mounted on the inspection robot, thereby allowing each position in the guardrail surface image to have a preset positional relationship with the inspection robot, for example, by reference... Figure 7 The distance of the abnormal light stripe from the inspection robot along the robot's direction of travel can be inferred from the distance of the abnormal light stripe from the image edge, such as the left edge. However, it is understood that the above method is only an example. In other embodiments, the location of the abnormal light stripe and the location of the guardrail rivets can be reasonably determined by other means. This embodiment of the invention does not limit this.

[0184] S430: Generates the speed adjustment curve of the inspection robot's drive mechanism based on the position of the guardrail rivets.

[0185] In some embodiments, the speed adjustment curve can be used to drive the inspection robot through the area where the guardrail rivets are located, thereby enabling the inspection robot to smoothly pass through the area where the guardrail rivets and guardrail joints are located.

[0186] In some embodiments, a speed adjustment curve can be used to control the inspection robot to reduce its speed when it reaches a preset distance from the guardrail rivet, maintain a preset low speed while passing through the area where the guardrail rivet is located, and gradually restore the speed to normal after passing through the area. In some embodiments, the speed adjustment curve may be trapezoidal (first uniformly decelerating, then maintaining a constant low speed, and finally uniformly accelerating), etc. As an explanation and not a limitation, the determined speed adjustment curve can control the inspection robot to reduce its speed when passing through local protrusions such as guardrail rivets and joints, reduce the impact and bumps on the robot body, and allow time for corresponding adjustments of the magnetic holding mechanism, adjustable suspension mechanism, etc., so that the inspection robot can pass through the aforementioned areas smoothly.

[0187] As previously described, the corrugated guardrail comprises a guardrail whose surface is undulating along its extension direction. The present invention recognizes that the traveling mechanism of the inspection robot therefore contacts not a flat surface, but a curved, inclined (guardrail) surface. Consequently, the multiple traveling mechanisms of the inspection robot, such as multiple wheels, are prone to misalignment upon contact with the inclined surface, leading to an overall tilt of the robot body relative to the corrugated guardrail, thus posing a risk of deviation from the travel path and detachment. As an example, referring to the references... Figures 7 to 9When the two walking mechanisms of the inspection robot come into contact with the two peaks 131 of the wave guardrail, there is a risk of sliding unfavorably towards the adjacent trough 133, which may cause the robot body, which was originally basically parallel to the guardrail surface, to tilt relative to the guardrail surface.

[0188] In some embodiments, the inspection robot may further include an adjustable suspension mechanism. In this embodiment, the adjustable suspension mechanism may include a suspension mechanism connected between the robot's body and its traveling section, used to support the body, and whose support state (e.g., support height) is adjustable. For explanation, by adjusting the support state of this suspension mechanism, the height and / or posture of the inspection robot's body relative to the corrugated guardrail surface can be changed.

[0189] Accordingly, in some embodiments, reference is made to Figure 5 The inspection robot driving method may further include the following steps S510 and S520.

[0190] S510: Determine the pixel spacing between adjacent peaks or adjacent troughs in the guardrail surface image for multi-period light stripes.

[0191] In some embodiments, pixel spacing between multiple pairs of adjacent peaks or multiple pairs of adjacent troughs can be identified within a region of interest containing multiple periodic light stripes. In an optional embodiment, the pixel spacing between adjacent peak feature points or adjacent trough feature points identified within the region of interest can be used as the pixel spacing between adjacent peaks. In this embodiment, the description of the region of interest and the peak and trough feature points can be found in the relevant sections of the foregoing embodiments, and will not be repeated here.

[0192] In an interpretive rather than limiting sense, the present invention recognizes that the actual spacing between adjacent peaks on a wave-shaped guardrail is approximately fixed. Therefore, when the aircraft is directly facing the surface of the wave-shaped guardrail (i.e., the aircraft is approximately parallel to the surface of the wave-shaped guardrail), the peaks on the reflected light strip are at approximately the same depth from the image acquisition device, and correspondingly, the pixel spacing between adjacent peaks is also approximately uniform. However, when the aircraft is tilted relative to the surface of the wave-shaped guardrail (i.e., the aircraft is not parallel to the surface of the wave-shaped guardrail), the structured light projected onto the wave-shaped guardrail is also deflected, causing the peaks on the reflected light strip to fall at different depths from the image acquisition device, with some peaks closer to the image acquisition device and others farther away. Here, the present invention recognizes that due to perspective imaging, the pixel spacing between adjacent peaks closer to the image acquisition device is usually larger, and the pixel spacing between adjacent peaks farther away is usually smaller, thus changing the pixel spacing between adjacent peaks from uniform to non-uniform. Therefore, the difference between the pixel spacing of multiple pairs of adjacent peaks, or the difference between the pixel spacing of adjacent peaks and a preset spacing, can be used to reflect the attitude relationship of the aircraft relative to the surface of the wave-shaped guardrail.

[0193] S520: Adjusts the height of the inspection robot relative to the corrugated guardrail surface by adjusting the adjustable suspension mechanism according to the difference between pixel pitches.

[0194] In some embodiments, the adjustable suspension mechanism can adjust the height of the inspection robot relative to the wave-shaped guardrail surface based on the difference in pixel spacing between adjacent peaks or troughs in the guardrail surface image, or the difference between the pixel spacing between adjacent peaks or troughs and a preset pixel spacing. In this embodiment, the preset pixel spacing may include, for example, the pixel spacing between adjacent peaks when the inspection robot is held in the expected position and orientation.

[0195] By way of explanation and not limitation, the method of the above embodiments of the present invention can determine whether the inspection robot is tilted relative to the surface of the wave-shaped guardrail and the degree of tilt based on the difference between the pixel spacing of multiple pairs of adjacent peaks or the pixel spacing of multiple pairs of adjacent valleys. Based on this, different adjustment amounts are determined for the suspension support applied to different parts of the body by the adjustable suspension mechanism to reduce the difference, so that the pixel spacing between each pair of adjacent peaks / adjacent valleys tends to be consistent, thereby correcting the tilt posture of the body relative to the surface of the wave-shaped guardrail. This allows the body to continuously maintain a preset posture on the surface of the wave-shaped guardrail, such as being parallel to the surface of the wave-shaped guardrail, which is beneficial for maintaining the stable adsorption of the magnetic holding mechanism on the wave-shaped guardrail, ensuring the stable imaging of the reflected light strip by the image acquisition device, and enabling the inspection robot to move smoothly.

[0196] In an optional embodiment, the tilt angle of the inspection robot can also be estimated based on the deflection angle of the light strip feature points or anchoring features in the guardrail surface image relative to a preset reference direction. This tilt angle can be used to trigger emergency hold or attitude adjustment.

[0197] In some embodiments, the inspection robot may further include a magnetic holding mechanism for adsorbing and holding the inspection robot on the wave-shaped guardrail, and an additional holding mechanism for holding the inspection robot on the wave-shaped guardrail in non-normal situations. In this embodiment, in other words, the inspection robot can be held on the wave-shaped guardrail without relying on the additional holding mechanism under normal operating conditions.

[0198] Accordingly, in some embodiments, the inspection robot driving method may further include the following steps D1 and D2.

[0199] D1: Real-time monitoring of the adsorption and retention force of the wave-shaped guardrail to obtain the change value of the adsorption and retention force.

[0200] In some embodiments, the adsorption holding force may include the force exerted by the magnetic holding mechanism to adsorb and hold the inspection robot on the surface of the wave-shaped guardrail. In some embodiments, the adsorption holding force can be monitored in real time by means including but not limited to a force sensor disposed between the magnetic holding mechanism and the robot body, or indirectly by means of output parameters (e.g., magnetic flux density) of a magnetic flux sensor. The present invention does not limit the specific means of directly or indirectly monitoring the adsorption holding force of the wave-shaped guardrail.

[0201] D2: When the change in adsorption and holding force exceeds the preset threshold, the additional holding mechanism is triggered to keep the inspection robot on the wave-shaped guardrail.

[0202] In some embodiments, the change in adsorption retention force may be the amount of change, the rate of change, or other quantities that can characterize the change in adsorption retention force over a period of time.

[0203] In some embodiments, if the change in the adsorption and holding force exceeds a preset threshold, the retention state of the inspection robot relative to the corrugated guardrail is considered abnormal. In this embodiment, the abnormality may include, but is not limited to, passing through a connection point, guardrail deformation, or external disturbance. This is explained as follows: the magnetic circuit will be weakened, leading to an increase in the gap and a sharp drop in the adsorption and holding force. To address this, an additional holding mechanism (e.g., an anchoring arm) can be triggered to apply additional holding action to the inspection robot, preventing it from detaching from the corrugated guardrail.

[0204] In a third aspect of the present invention, another method for driving a safety inspection robot is also provided. The method includes: acquiring an image of a guardrail in front of the inspection robot; processing the guardrail image to extract multiple guardrail extension lines extending longitudinally along the guardrail; determining a disappearance point based on the convergence position of the multiple guardrail extension lines in the guardrail image; determining a turning amount that causes the inspection robot to move towards a preset orientation based on the deviation of the disappearance point from a preset position; determining a differential speed control amount of the drive mechanism of the inspection robot in real time based on the turning amount; and driving the drive mechanism according to the differential speed control amount to make the inspection robot move along the guardrail.

[0205] In some embodiments, determining the disappearance point based on the convergence position of the plurality of guardrail extension lines in the guardrail image includes: finding the intersection points of the plurality of guardrail extension lines pairwise or extending them to find the intersection points to obtain a plurality of intersection points; determining the geometric center of the plurality of intersection points as the disappearance point; and / or, determining the disappearance point based on the convergence position of the plurality of guardrail extension lines in the guardrail image includes: determining a first representative extension line and a second representative extension line based on the plurality of guardrail extension lines; determining the intersection point obtained by finding the intersection point of the first representative extension line and the second representative extension line or extending them to find the intersection point as the disappearance point.

[0206] In some embodiments, acquiring images of the guardrail in front of the inspection robot includes: acquiring the guardrail images in the form of consecutive frames; determining the differential control amount of the drive mechanism of the inspection robot in real time based on the steering amount includes: determining the curvature change of the guardrail in front of the inspection robot based on the position change of the disappearance point in the consecutive frames of the guardrail images; determining the feedforward compensation amount based on the curvature change; and determining the differential control amount by superimposing the steering amount and the feedforward compensation amount.

[0207] In some embodiments, acquiring the image of the guardrail in front of the inspection robot includes: acquiring the guardrail image in the form of consecutive frames; the driving method further includes: determining the position jitter of the disappearing point in the consecutive frames of the guardrail image; when the position jitter exceeds a preset tolerance, determining that the disappearing point tracking is out of lock; in the case that the disappearing point tracking is out of lock, controlling the driving of the driving mechanism and / or adjusting the extraction processing of the guardrail extension line based on a preset loss-of-lock control mode.

[0208] In some embodiments, acquiring images of the guardrail in front of the inspection robot includes: acquiring the guardrail images in the form of consecutive frames; before determining the differential control amount, the driving method further includes: determining whether the disappearance point in the consecutive frames of the guardrail images continuously remains within a preset neighborhood of the preset position; if it remains within the preset neighborhood, switching the inspection robot to an autonomous movement mode so as to drive the drive mechanism according to the differential control amount in the autonomous movement mode; if it is not within the preset neighborhood, switching the inspection robot to a manual remote control mode so as to drive the drive mechanism according to external remote control commands in the manual remote control mode.

[0209] In some embodiments, determining the turning amount to orient the inspection robot toward a preset orientation based on the deviation of the disappearing point relative to a preset position includes: determining the lateral pixel deviation of the disappearing point relative to a preset position in the image coordinate system of the guardrail image; and using the lateral pixel deviation as a control error input feedback controller to determine the turning amount.

[0210] In some embodiments, the inspection robot includes an adjustable suspension mechanism, and the guardrail is a multi-waveform guardrail; the driving method further includes: determining the pixel spacing between multiple pairs of adjacent waveform lines in the guardrail image; and adjusting the height of the inspection robot relative to the guardrail surface by the adjustable suspension mechanism according to the difference between the pixel spacing between the multiple pairs of adjacent waveform lines, so that the inspection robot maintains a preset vertical posture on the inclined surface of the multi-waveform guardrail.

[0211] In the above embodiments of the present invention, the safety inspection robot driving method according to the third aspect of the present invention can be used in combination with the structured light recognition-based safety inspection robot driving method according to the first aspect of the present invention. For example, the inspection robot driving method according to the first aspect of the present invention can be used as the main method, and the safety inspection robot driving method according to the third aspect of the present invention can be used as an auxiliary method to more accurately determine the differential speed control amount, so that the inspection robot can move autonomously along the wave-shaped guardrail stably and reliably. However, it is understood that in other embodiments, the safety inspection robot driving method according to the third aspect of the present invention can be used as the main method, and the structured light recognition-based safety inspection robot driving method according to the first aspect of the present invention can be used as an auxiliary method to control the inspection robot to move autonomously along the wave-shaped guardrail, which falls within the protection scope of the embodiments of the present invention.

[0212] The methods and steps of the above embodiments of the present invention can be combined to obtain new embodiments without contradicting each other, which fall within the protection scope of the embodiments of the present invention.

[0213] Therefore, the inspection robot driving method according to the embodiments of the present invention can have at least the following technical effects.

[0214] The structured light recognition-based safety inspection robot driving method provided in this invention can actively project structured light onto a wave-shaped guardrail to form reflected light stripes on its surface. It acquires an image of the guardrail surface containing the reflected light stripes and extracts light stripe feature points within the region of interest corresponding to the light stripes. Based on the geometric distribution of these feature points, it determines anchoring features. Then, based on the deviation of the anchoring features from a preset position, it determines the turning amount that will cause the inspection robot to move towards a preset orientation. Based on this, it determines the differential speed control amount of the driving mechanism in real time to drive the inspection robot. This scheme enhances the recognizability of the wave-shaped guardrail's geometry in the image through actively projected structured light and uses the deviation of the anchoring features from the preset position to perform closed-loop control of the inspection robot's drive. It solves the technical problem of stable autonomous movement of the inspection robot over long distances on the complex undulating surface of the wave-shaped guardrail, thereby significantly improving the stability and reliability of the inspection robot's autonomous movement along the wave-shaped guardrail.

[0215] In a further embodiment of the present invention, the lateral pixel deviation of the anchoring feature relative to the guardrail surface image at a preset position in the image coordinate system is used as the control error input feedback controller to obtain the steering amount. This scheme realizes a stable deflection adjustment mechanism and further improves the stability of the inspection robot's travel orientation.

[0216] In a further embodiment of the present invention, the presence of light stripe breaks and / or light stripe distortions compared to a preset contour in the reflected light stripe in the guardrail surface image is identified. Then, when a light stripe anomaly is identified, the position of the guardrail rivet is determined according to its corresponding position, and a speed adjustment curve of the drive mechanism is generated accordingly. This solution solves the technical problem that the rivets on the wave-shaped guardrail and the guardrail joints cause body bumps and affect the smooth movement of the inspection robot. It enables the inspection robot to pass smoothly through the area where the guardrail rivets and guardrail joints are located, thereby improving the stability of the inspection robot's movement.

[0217] In a further embodiment of the present invention, the pixel spacing between multiple pairs of adjacent peaks or multiple pairs of adjacent troughs in the guardrail surface image of the multi-period light stripe is determined, and the adjustment amount of the adjustable suspension mechanism on the height of the aircraft body relative to the surface of the wave-shaped guardrail is adjusted according to the difference in the pixel spacing. This solution reduces the difference by adjusting the height of the aircraft body according to the difference in the pixel spacing between adjacent peaks or multiple pairs of adjacent troughs, thereby enabling the aircraft body to maintain a preset posture with the surface of the wave-shaped guardrail, solving the technical problem that the aircraft body is prone to posture deviation on the surface of the wave-shaped guardrail, thus affecting the stability of data acquisition and travel.

[0218] In some embodiments of the present invention, a magnetic holding mechanism is also used to adhere the inspection robot to the wave-shaped guardrail. The magnetic holding force is monitored in real time, and when the change in the magnetic holding force exceeds a preset threshold, an additional holding mechanism is triggered to abnormally hold the inspection robot to the wave-shaped guardrail. This solution, by introducing an additional holding mechanism to supplement the holding when the magnetic holding force changes abnormally, solves the technical problem that relying solely on a single holding method poses a risk of the inspection robot detaching from the wave-shaped guardrail when the magnetic holding force fluctuates.

[0219] In this embodiment of the invention, reference Figure 10 Furthermore, a safety inspection robot drive device 1000 based on structured light recognition is provided. According to the second aspect of the present invention, the inspection robot drive device 1000 may include a structured light projection module 1010, an image acquisition module 1010, a region of interest determination module 1020, a feature point determination module 1030, an anchoring feature determination module 1040, a steering determination module 1050, a differential speed determination module 1060, and a drive module 1070.

[0220] In some embodiments, the structured light projection module 1010 is configured to project structured light onto the wave-shaped guardrail to form a reflective light strip on the surface of the wave-shaped guardrail.

[0221] In some embodiments, the image acquisition module 1020 is configured to acquire an image of the guardrail surface containing the reflected light strip.

[0222] In some embodiments, the region of interest determination module 1030 is configured to process the guardrail surface image to determine the region of interest of the reflected light strip in the guardrail surface image.

[0223] In some embodiments, the feature point determination module 1040 is configured to extract the light stripe feature points of the reflected light stripe in the region of interest.

[0224] In some embodiments, the anchoring feature determination module 1050 is configured to determine anchoring features based on the geometric distribution of the light stripe feature points.

[0225] In some embodiments, the steering determination module 1060 is configured to determine a steering amount that causes the inspection robot to turn toward a preset orientation based on the deviation of the anchoring feature from a preset position.

[0226] In some embodiments, the differential speed determination module 1070 is configured to determine the differential speed control amount of the drive mechanism of the inspection robot in real time based on the steering amount.

[0227] In some embodiments, the drive module 1080 is configured to drive the drive mechanism according to the differential control amount, so that the inspection robot travels along the wave-shaped guardrail.

[0228] In other embodiments, an inspection robot drive device is also provided. According to the fourth aspect of the present invention, the inspection robot drive device may include an image acquisition module, an extension line extraction module, an anonymization point determination module, a steering determination module, a differential speed determination module, and a drive module.

[0229] In some embodiments, the image acquisition module is configured to acquire images of the guardrail in front of the inspection robot.

[0230] In some embodiments, the extension line extraction module is configured to process the guardrail image and extract multiple guardrail extension lines extending longitudinally along the guardrail from the guardrail image.

[0231] In some embodiments, the disappearance point determination module is configured to determine the disappearance point based on the convergence position of the plurality of guardrail extension lines in the guardrail image.

[0232] In some embodiments, the steering determination module is configured to determine a steering amount that causes the inspection robot to turn toward a preset orientation based on the deviation of the disappearance point from a preset position.

[0233] In some embodiments, the differential speed determination module is configured to determine the differential speed control amount of the drive mechanism of the inspection robot in real time based on the steering amount.

[0234] In some embodiments, the drive module is configured to drive the drive mechanism according to the differential control amount to cause the inspection robot to travel along the guardrail.

[0235] The inspection robot driving device and its components, modules, units, and features described in the embodiments of the present invention can be incorporated into the inspection robot driving method of the embodiments of the present invention in a non-contradictory manner, and will not be elaborated here. Conversely, the inspection robot driving method and its steps, sub-steps, and features described in the embodiments of the present invention can also be incorporated into the inspection robot driving device of the embodiments of the present invention in a non-contradictory manner.

[0236] In this embodiment of the invention, an electronic device is also provided, which includes a processor and a memory storing a computer program, the processor being configured to implement the method of any embodiment of the invention when running the computer program.

[0237] Figure 11 A schematic diagram is shown of a method or electronic device 1100 that can be used to implement embodiments of the present invention. In some embodiments, the number of electronic devices may be more or less than the number shown. In some embodiments, a single or multiple electronic devices may be used. Cloud-based or distributed electronic devices may also be used in some embodiments.

[0238] like Figure 11 As shown, the electronic device 1100 includes a processor 1101 and a memory 1102. The processor executes programs stored in the memory, which, when executed by a computer, can implement the methods, steps, or functions described in the above embodiments. The processor 1101 may include various types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. The processor 1101 and the memory 1102 are interconnected via a bus 1103. Input / output (I / O) interfaces may also be connected to the bus 1103. The systems, devices, modules, or units described in the above embodiments can be implemented by a computer or its associated components.

[0239] Although not shown, in this embodiment of the invention, a program product is provided, including a computer program that, when executed by a processor, implements the method of any embodiment of the invention.

[0240] Although not shown, in an embodiment of the invention, a storage medium is provided storing a computer program configured to be run to implement the method of any embodiment of the invention.

[0241] Storage media in embodiments of the present invention include articles that are permanent or non-permanent, removable or non-removable, and can store information by any method or technology. Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information that can be accessed by a computing device.

[0242] The methods, programs, systems, apparatuses, etc., in embodiments of the present invention can be executed or implemented in one or more networked computers, or practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be performed by remote processing devices connected via a communication network.

[0243] Those skilled in the art will understand that the embodiments described in this specification can be provided as methods, systems, or computer program products. Therefore, those skilled in the art will realize that the functional modules / units or controllers and related method steps described in the above embodiments can be implemented in software, hardware, or a combination of both.

[0244] Unless explicitly stated otherwise, the actions or steps of the methods and procedures described in the embodiments of the present invention do not necessarily have to be performed in a specific order and can still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0245] This document describes several embodiments of the present invention; however, for the sake of brevity, the descriptions of the embodiments are not exhaustive, and identical or similar features or parts between the embodiments may be omitted. In this document, "one embodiment," "some embodiments," "example," "specific example," or "some examples" refers to embodiments applicable to at least one, but not all, of the present invention. The above terms do not necessarily refer to the same embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of the different embodiments or examples.

[0246] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best mode for implementing the systems and methods. Those skilled in the art will understand that various changes can be made to the embodiments of the systems and methods described herein without departing from the spirit and scope of the invention as defined in the appended claims when implementing the systems and / or methods.

Claims

1. A driving method for a security inspection robot based on structured light recognition, characterized in that, The driving method includes: Structured light is projected onto the wave-shaped guardrail to form reflective light stripes on the surface of the guardrail; Acquire an image of the guardrail surface containing the reflected light stripe; The image of the guardrail is processed to determine the region of interest of the reflected light stripe in the image of the guardrail. Extract the light stripe feature points of the reflected light stripe within the region of interest; Anchoring features are determined based on the geometric distribution of the light stripe feature points; The amount of steering that causes the inspection robot to turn toward the preset orientation is determined based on the deviation of the anchoring feature from the preset position. The differential control amount of the drive mechanism of the inspection robot is determined in real time based on the steering amount. The drive mechanism is driven according to the differential control value so that the inspection robot moves along the wave-shaped guardrail.

2. The method according to claim 1, characterized in that, The wave-shaped guardrail is a multi-wave-shaped guardrail, and the reflected light strip is a multi-periodic light strip formed on the surface of the multi-wave-shaped guardrail. Extracting the light stripe feature points of the reflected light stripe within the region of interest includes: Identify multiple peak feature points of the multi-period light stripe within the region of interest; and / or Identify multiple trough feature points of the multi-period light stripe within the region of interest; The determination of anchoring features based on the geometric distribution of the light stripe feature points includes: A single anchor point is determined based on the geometric relationship between the multiple peak feature points and / or multiple trough feature points.

3. The method according to claim 2, characterized in that, Determining a single anchor point based on the geometric relationship between the plurality of peak feature points and / or the plurality of trough feature points includes: The centerline of the multi-period light stripe is determined based on the multiple peak feature points and / or multiple trough feature points; The anchor point is determined based on the position where the centerline of the multi-period light stripe intersects with a preset reference line in the guardrail surface image; or Determining a single anchor point based on the geometric relationship between the plurality of peak feature points and / or the plurality of trough feature points includes: The geometric midpoint of the plurality of wave crest feature points and / or the plurality of wave trough feature points is determined as the anchor point.

4. The method according to claim 1, characterized in that, The acquisition of the guardrail surface image containing the reflected light stripe includes: Images of the guardrail surface are acquired in consecutive frames. The step of processing the guardrail surface image to determine the region of interest of the reflected light stripe in the guardrail surface image includes: For the guardrail surface image of the initial frame, the region of interest of the initial frame is determined according to the preset projection direction of the structured light; For the guardrail surface image that is not the initial frame, the region of interest for the current frame is determined based on the position of the anchoring feature determined by the guardrail surface image of the previous frame.

5. The method according to claim 1, characterized in that, The step of determining the turning amount to orient the inspection robot toward a preset orientation based on the deviation of the anchoring feature from the preset position includes: Determine the lateral pixel deviation of the anchoring feature relative to a preset position in the image coordinate system of the guardrail surface image; The lateral pixel deviation is used as a control error input feedback controller to determine the steering amount.

6. The method according to any one of claims 1 to 5, characterized in that, The inspection robot includes at least two walking parts and at least two drive mechanisms that independently drive the at least two walking parts; The step of determining the differential control amount of the drive mechanism of the inspection robot in real time based on the steering amount includes: The multi-channel independent speed control quantity is obtained from the decomposition of the steering amount, and the multi-channel independent speed control quantity corresponds one-to-one with the at least two drive mechanisms.

7. The method according to any one of claims 1 to 5, characterized in that, The driving method further includes: Identify whether the reflected light strip in the guardrail surface image has a given light strip anomaly, the light strip anomaly including light strip breakage at a given location and / or light strip distortion at a given location compared to a preset contour; If the light stripe is detected to be abnormal, the position of the guardrail rivet is determined according to the position of the abnormal light stripe in the guardrail surface image. The speed adjustment curve of the inspection robot's drive mechanism is generated based on the position of the guardrail rivets. The speed adjustment curve is used to drive the inspection robot to travel through the area where the guardrail rivets are located.

8. A drive device for a security inspection robot based on structured light recognition, characterized in that, include: A structured light projection module is configured to project structured light onto a wave-shaped guardrail to form a reflected light strip on the surface of the wave-shaped guardrail. The image acquisition module is configured to acquire an image of the guardrail surface containing the reflected light strips; The region of interest determination module is configured to process the guardrail surface image and determine the region of interest of the reflected light stripe in the guardrail surface image; The feature point determination module is configured to extract the light stripe feature points of the reflected light stripe within the region of interest. Anchoring feature determination module, configured to determine anchoring features based on the geometric distribution of the light stripe feature points; The steering determination module is configured to determine the steering amount that causes the inspection robot to turn toward a preset orientation based on the deviation of the anchoring feature relative to a preset position. The differential speed determination module is configured to determine the differential speed control amount of the drive mechanism of the inspection robot in real time based on the steering amount. The drive module is configured to drive the drive mechanism according to the differential control amount, so that the inspection robot moves along the wave-shaped guardrail.

9. An electronic device, characterized in that, include: A processor and a memory storing a computer program, the processor being configured to implement the method as described in any one of claims 1-8 when the computer program is executed.

10. A program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images