A robot control method and device based on escalator detection

Abstract

The application discloses a robot control method and device based on escalator detection, which comprises the following steps: collecting a to-be-detected image in real time by using a camera configured by a robot, processing the to-be-detected image by using a Sobel operator, a preset binary algorithm and image morphological operation, determining a plurality of gradient connected regions in the to-be-detected image, generating a minimum circumscribed rectangle frame by calculating whether the area of each gradient connected region is greater than a set threshold, obtaining position information of the minimum circumscribed rectangle frame, and controlling the robot to perform corresponding operation according to the position information of the minimum circumscribed rectangle frame. By adopting the embodiment of the application, the position of the escalator can be directly positioned, data processing is reduced, computer resources are avoided from being wasted, the accuracy of detection is improved through various processing of the to-be-detected image.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a robot control method and device based on escalator detection. Background Technology

[0002] With technological advancements, robots are not only being used in manufacturing but are also increasingly appearing in service, entertainment, and military fields. In real life, escalators are widely used in various settings, especially in public places such as shopping malls, subway stations, and hospitals. During robot operation, to detect the presence of escalators ahead in real time, and to intelligently avoid obstacles and change movement patterns based on the detection results—operations crucial in practical applications—a robust target detection model needs to be trained. This requires collecting a large amount of data and spending considerable time manually labeling and preprocessing it. Furthermore, the multi-target detection model configured on the robot needs to perform real-time target detection on images acquired by cameras, but in many places (station plazas, parks, etc.), escalators are not present as targets.

[0003] Currently, target detection algorithms configured on intelligent robots fall into two categories: two-stage and one-stage methods. While two-stage methods offer high accuracy, their slow detection speed fails to meet the service requirements of robots in real-world scenarios. Therefore, the target detection algorithms currently deployed on service robots primarily employ one-stage methods. Escalators, as one of many target types, require extensive data collection and manual annotation, along with data preprocessing for learning robust target detection algorithms. This process is costly in terms of data acquisition and annotation, and time-consuming in training. Furthermore, multi-target detection models deployed on robots need to perform real-time target detection on images acquired by cameras, consuming significant computational resources during the detection process. Summary of the Invention

[0004] This invention provides a robot control method and device based on escalator detection, to solve the technical problem that robots need to collect and label a large amount of data in the early stage when detecting escalators.

[0005] To address the aforementioned technical problems, this invention provides a robot control method based on escalator detection, comprising:

[0006] The robot acquires images of the object to be detected in real time using a camera mounted on it.

[0007] An edge grayscale image is generated by extracting edge features from the image to be detected using the Sobel operator.

[0008] Based on a preset binarization algorithm and image morphology operations, multiple gradient connected regions in the edge grayscale image are determined, and the area of ​​each gradient connected region is calculated.

[0009] When any of the regions has an area greater than a set threshold, it is determined that an escalator exists in the image to be detected. Based on the gradient connected region with the largest region area, a minimum bounding rectangle is generated, and the position information of the minimum bounding rectangle is obtained.

[0010] Based on the minimum bounding rectangle and the position information, the robot is controlled to perform corresponding operations.

[0011] This invention acquires the image to be detected via a camera, extracts its edge features using the Sobel operator, and generates an edge grayscale image, improving detection accuracy. Furthermore, it determines the connected regions of the edge grayscale image based on a preset binarization algorithm and image morphological operations, and calculates the area of ​​each connected region. When the area of ​​a connected region exceeds a set threshold, the presence of an escalator in the image is confirmed, directly identifying the target and saving data processing time. A minimum bounding box is generated based on the connected region with the largest area, and its position is determined to obtain the escalator's location. Based on the minimum bounding box and the position information, the robot is controlled to perform corresponding operations, further reducing the waste of computer resources.

[0012] As a preferred example, before extracting the edge features of the image to be detected according to the Sobel operator, the method further includes:

[0013] The acquired image to be detected is converted to grayscale to obtain a first grayscale image. Then, the first grayscale image is subjected to Gaussian filtering and noise reduction to obtain a second grayscale image.

[0014] This invention acquires the image to be detected through a camera, performs image grayscale processing on the image to obtain a grayscale image, and then performs Gaussian filtering on the grayscale image for noise reduction, further amplifying the characteristics of the image to be detected and improving the accuracy of detection.

[0015] As a preferred example, the process of extracting edge features from the image to be detected based on the Sobel operator and generating an edge grayscale image further includes:

[0016] The second grayscale image is convolved with a preset template to obtain approximate values ​​of the horizontal and vertical brightness differences of the image corresponding to the second grayscale image.

[0017] Based on the preset weighting formula and the approximate values ​​of the brightness difference in the horizontal and vertical directions of the image, the edge features in the second grayscale image are calculated to generate the edge grayscale image.

[0018] This invention utilizes the Sobel operator to obtain the edge features of the grayscale image, which can better suppress noise. The edge features are then convolved with the image to be detected to obtain approximate values ​​of brightness difference in the horizontal and vertical directions of the image. Based on the image characteristics of the escalator target itself, the edge grayscale image of the image to be detected is obtained by weighting, which further amplifies the characteristics of the image to be detected and improves the accuracy of detection.

[0019] As a preferred example, the process of determining multiple gradient connected regions in the edge grayscale image according to a preset binarization algorithm and image morphological operations, and calculating the area of ​​each gradient connected region, further includes:

[0020] Based on a preset grayscale threshold, the edge grayscale image is binarized to obtain a first binary image;

[0021] Perform image morphology operations on the first binary image to remove noise points and obtain a second binary image;

[0022] Traverse each pixel in the second binary image, and generate multiple gradient connected regions using the seed filling method. Calculate the area of ​​each gradient connected region based on the number of pixels within each gradient connected region.

[0023] This invention obtains a new binary image by performing various morphological operations on the binary image, which can effectively eliminate narrow discontinuities, small holes, and discrete noise points, further improving detection accuracy. Then, based on the number of pixels in each gradient connected region, the number and area of ​​gradient connected regions in the image to be detected are calculated, without the need to process large amounts of data.

[0024] As a preferred example, the process of performing image morphology operations on the first binary image to remove noise points and obtain the second binary image further includes:

[0025] First, a dilation-erosion closing operation is performed on the first binary image based on the preset structuring element. Then, a new structuring element is set to perform an erosion-dilation opening operation. Finally, a dilation operation is performed to obtain the second binary image.

[0026] This invention uses more detailed morphological operations, and further extracts features from the detection image more accurately through closing, opening and dilation operations, thereby improving the accuracy of detection.

[0027] As a preferred example, after obtaining the area of ​​each of the gradient connected regions, the method further includes:

[0028] Determine whether the area of ​​each gradient-connected region is greater than the set threshold.

[0029] If the area of ​​each region is less than the set threshold, then it is determined that there is no escalator in the image to be detected;

[0030] If any of the regions has an area greater than a set threshold, then it is determined that an escalator exists in the image to be detected.

[0031] This invention determines the presence of escalators in an image by comparing the area of ​​the gradient connected region with a set threshold, effectively avoiding the waste of computer resources. If the area of ​​a connected region in the image is greater than the threshold, all regions with areas greater than the threshold are retained. This invention directly determines the presence of escalators in an image by comparing the area of ​​the connected region with the threshold, without relying on large amounts of data, thus solving the problem of needing to collect and label large amounts of data before detection.

[0032] As a preferred example, the process of generating the minimum bounding rectangle based on the gradient-connected region with the largest area and obtaining the position information of the minimum bounding rectangle specifically involves:

[0033] After obtaining the area of ​​each gradient connected region, retain the gradient connected regions whose area is greater than the set threshold, and select the gradient connected region block with the largest area among the retained gradient connected regions.

[0034] Based on the maximum and minimum x and y coordinates of the pixels in the gradient connected region with the largest area, the minimum bounding rectangle is calculated. Based on the coordinates of the upper left and lower right corners of the minimum bounding rectangle, the position information of the minimum bounding rectangle is obtained.

[0035] This invention selects the region with the largest area from the preserved gradient connected regions. Based on the maximum and minimum x-coordinates and y-coordinates of the pixels within the gradient connected regions, it calculates the minimum bounding rectangle. Then, based on the coordinates of the top-left and bottom-right corners of the minimum bounding rectangle, it determines the position of the minimum bounding rectangle, thereby identifying the location of the escalator and reducing data processing time. Subsequently, the robot is controlled to perform the corresponding operations. This invention can be deployed on low-power computing devices, and the program can be started at any time according to the scenario, solving the problem of high computational resource consumption during detection.

[0036] On the other hand, embodiments of the present invention provide a robot control device based on escalator detection, specifically including: an image acquisition module, a grayscale image generation module, a connected component calculation module, a position calculation module, and a control module;

[0037] The image acquisition module is used to acquire images to be detected in real time based on the camera terminal configured on the robot;

[0038] The grayscale image generation module is used to extract edge features of the image to be detected based on the Sobel operator and generate an edge grayscale image.

[0039] The connected region calculation module is used to determine multiple gradient connected regions in the edge grayscale image according to a preset binarization algorithm and image morphological operations, and to calculate the area of ​​each gradient connected region respectively.

[0040] The position calculation module is used to determine that there is an escalator in the image to be detected when any area of ​​the region is greater than a set threshold, and to generate a minimum bounding rectangle based on the gradient connected region with the largest area, and to obtain the position information of the minimum bounding rectangle.

[0041] The control module is used to control the robot to perform corresponding operations based on the minimum bounding rectangle and the position information.

[0042] This invention acquires the image to be detected through an image acquisition module, then extracts the edge features of the image using a grayscale image generation module to generate an edge grayscale image, improving detection accuracy. Furthermore, a connected region calculation module determines the connected regions of the edge grayscale image and calculates the area of ​​the gradient connected regions. When the area of ​​a gradient connected region exceeds a set threshold, the presence of an escalator in the image to be detected is confirmed, directly identifying the target and saving data processing time. A minimum bounding rectangle is generated from the gradient connected region with the largest area using a position calculation module, and the position of the minimum bounding rectangle is determined to obtain the escalator's location. Finally, a control module uses the minimum bounding rectangle and the position information to control a robot to perform corresponding operations, further reducing the waste of computer resources.

[0043] As a preferred example, the grayscale image generation module includes a grayscale image processing unit and an edge feature extraction unit, comprising:

[0044] The grayscale image processing unit is used to convert the acquired image to grayscale to generate a first grayscale image, and to perform Gaussian filtering and noise reduction processing on the first grayscale image to obtain a second grayscale image.

[0045] The edge feature extraction unit is used to extract the edge features of the second grayscale image according to the Sobel operator to generate an edge grayscale image.

[0046] This invention uses a grayscale image processing unit to convert the image to grayscale, obtaining a grayscale image. Then, Gaussian filtering is applied to the grayscale image for noise reduction. Finally, edge features are extracted from the grayscale image using the Sobel operator to generate an edge grayscale image. This further amplifies the characteristics of the image to be detected, improving detection accuracy.

[0047] As a preferred example, the connected component calculation module includes:

[0048] Based on a preset grayscale threshold, the edge grayscale image is binarized to obtain a first binary image;

[0049] Perform image morphology operations on the first binary image to remove noise points and obtain a second binary image;

[0050] Traverse each pixel in the second binary image, and generate multiple gradient connected regions using the seed filling method. Calculate the area of ​​each gradient connected region based on the number of pixels within each gradient connected region.

[0051] The connected region calculation module proposed in this invention utilizes binarization algorithms and image morphological operations to determine multiple gradient connected regions in the edge grayscale image, and calculates the area of ​​each gradient connected region, thereby further improving the accuracy of detection, while eliminating the need for extensive data processing.

[0052] In summary, this invention separates escalator detection from multi-object detection models. It acquires the image to be detected via a camera and directly locates the target position using traditional image processing techniques, eliminating the need for prior data acquisition and annotation. Furthermore, this invention utilizes the Sobel operator, along with grayscale and morphological processing steps, to improve detection accuracy. By determining the bounding box position, the escalator is directly located, reducing the need for extensive data processing and minimizing the waste of computer resources. The escalator detection program is only required to be activated at the appropriate time to directly obtain the final result. Based on the obtained escalator bounding box and position information, the robot is controlled to perform subsequent speed changes and alterations to its action state. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a robot control method based on escalator detection provided in an embodiment of the present invention.

[0054] Figure 2 This is another flowchart illustrating a robot control method based on escalator detection provided by an embodiment of the present invention.

[0055] Figure 3: A schematic diagram of an embodiment of a robot control device based on escalator detection provided by the present invention;

[0056] Figure 4 This is a schematic diagram of solving for the minimum bounding rectangle provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1

[0059] Please refer to Figure 1 This is a flowchart illustrating a robot control method based on escalator detection provided by an embodiment of the present invention. It can be applied to robots working in places with escalators, such as large shopping malls or subway stations. The method mainly includes steps 101 to 105, as detailed below:

[0060] Step 101: Acquire the image to be detected in real time using the camera mounted on the robot.

[0061] In this embodiment, the robot continuously takes pictures and identifies objects in front of it as it moves forward, so as to promptly identify whether there is an escalator ahead.

[0062] Step 102: Extract edge features from the image to be detected using the Sobel operator to generate an edge grayscale image.

[0063] In this embodiment, the image to be detected is first converted to grayscale to obtain a grayscale image. Then, the edge features of the grayscale image are extracted according to the Sobel operator to generate an edge grayscale image. This can more accurately extract the detection features of the image to be detected and improve the detection accuracy.

[0064] Step 103: Based on the preset binarization algorithm and image morphology operations, determine multiple gradient connected regions in the edge grayscale image, and calculate the area of ​​each gradient connected region.

[0065] In this embodiment, the edge grayscale image is converted into a binary image using a preset binarization algorithm. Then, image morphology operations are used to further extract features from the binary image, improving detection accuracy. Each pixel in the binary image is traversed, and multiple gradient connected regions are generated using a seed filling method. Based on these multiple gradient connected regions, the area of ​​each region is calculated, reducing the amount of data that needs to be processed and accelerating the detection speed.

[0066] Step 104: When the area of ​​any gradient connected region is greater than a set threshold, it is determined that there is an escalator in the image to be detected, and a minimum bounding rectangle is generated based on the gradient connected region with the largest area, and the position information of the minimum bounding rectangle is obtained.

[0067] In this embodiment, the presence of an escalator in the image to be detected is directly determined by comparing the area of ​​the gradient connected region with a set threshold, without relying on large amounts of data, thus speeding up the detection process. At the same time, a minimum bounding rectangle is generated based on the region block with the largest area in the gradient connected region, and the position of the minimum bounding rectangle is determined, saving time.

[0068] Step 105: Based on the minimum bounding rectangle and the position information, control the robot to perform the corresponding operation.

[0069] In this embodiment, the position of the escalator is determined based on the position of the minimum bounding rectangle, and the robot is controlled to perform corresponding operations.

[0070] In this embodiment, the present invention does not require preliminary data collection and annotation. It improves the accuracy of detection by using the Sobel operator and steps such as grayscale processing and morphological processing. The escalator is directly located by judging the position of the rectangular box, without the need for a large amount of data processing, thus reducing the waste of computer resources.

[0071] Please refer to Figure 2 This is another flowchart illustrating a robot control method based on escalator detection provided by an embodiment of the present invention, which mainly includes steps 201 to 205, as follows:

[0072] Step 201: Extract edge features from the image to be detected using the Sobel operator to generate an edge grayscale image; wherein the image to be detected is an image acquired by a camera; the edge grayscale image is obtained by performing the Sobel operator and grayscale processing on the image to be detected.

[0073] In this embodiment, the acquired image to be detected is converted to grayscale to generate a first grayscale image. Gaussian filtering and noise reduction are then applied to the first grayscale image to obtain a second grayscale image. The second grayscale image is then convolved with a preset template to obtain approximate horizontal and vertical brightness differences corresponding to the second grayscale image. The preset template is as follows:

[0074]

[0075] In the template above, A represents the second grayscale image, G x G represents the horizontal brightness difference of the image. y Representing the vertical brightness difference of the image, the edge features in the second grayscale image are calculated based on a preset weighting formula and approximate values ​​of the horizontal and vertical brightness differences of the image, thus generating the edge grayscale image. The preset weighting formula is as follows:

[0076] |G|=ω small |G x |+ω big |G y |

[0077] Step 202: Perform binarization and image morphology operations on the edge grayscale image.

[0078] In this embodiment, the edge grayscale image is converted into a first binary image by a set threshold. The first binary image is first closed to obtain a second binary image, and then opened to obtain a third binary image. Based on the third binary image, the image is traversed according to the seed filling method to obtain the area of ​​each gradient connected region, that is, the number of pixels in each gradient connected region.

[0079] In this embodiment, the closing operation formula used is:

[0080]

[0081] In the closing operation formula, A b′ Represents the second binary graph, A b This is the first binary graph, and B is the set structural element.

[0082] In this embodiment, the opening operation formula used is:

[0083]

[0084] In the formula for opening, A b″ Represents a third binary graph, A b′ This is the second binary graph, and C is the set structuring element.

[0085] Step 203: Determine whether the area of ​​the gradient connected region is greater than a set threshold; if the result is true, proceed to step 204; if the result is false, proceed to step 205.

[0086] In this embodiment, when the area of ​​the gradient connected region is greater than a set threshold, it is determined that an escalator exists, which does not require big data and reduces resource waste.

[0087] Step 204: Determine the position of the minimum bounding rectangle and control the robot to perform the operation.

[0088] In this embodiment, the minimum bounding rectangle is calculated using the maximum and minimum x-coordinates and y-coordinates of pixels in the gradient-connected region with the largest area. Based on the coordinates of the top-left and bottom-right corners of the minimum bounding rectangle, its position information is obtained. Figure 4 This is a schematic diagram for solving the minimum rectangular bounding box problem, and then the robot is controlled to perform operations such as speed changes and intelligent obstacle avoidance.

[0089] Step 205: Delete the gradient connected regions whose area is less than the threshold.

[0090] In this embodiment, when the area of ​​the gradient connected region is less than a set threshold, it is determined that there is no escalator at present.

[0091] In this embodiment, the present invention continuously extracts features from the image to be detected by utilizing the Sobel operator and steps such as grayscale processing and morphological processing, thereby improving the accuracy of detection. Then, the escalator is directly located by judging the position of the rectangular box, without the need for a large amount of data processing, thus reducing the waste of computer resources.

[0092] Please refer to Figure 3 This is a schematic diagram of an embodiment of a robot control device based on escalator detection provided by the present invention, which mainly includes: an image acquisition module 301, a grayscale image generation module 302, a connected region calculation module 303, a position calculation module 304, and a control module 305.

[0093] In this embodiment, the image acquisition module 301 is configured to acquire images to be detected in real time on the robot.

[0094] The grayscale image generation module 302 is used to perform grayscale processing on the image to be detected after the image acquisition module 301 has acquired the image to be detected, to obtain a first grayscale image, to perform Gaussian filtering and noise reduction processing on the first grayscale image to obtain a second grayscale image, and then to use the Sobel operator to extract edge features from the second grayscale image to generate an edge grayscale image.

[0095] The connected region calculation module 303 is used to perform binarization processing on the edge grayscale image generated by the grayscale image generation module 302 after the grayscale image generation module 302 generates the edge grayscale image to obtain a first binary image, perform image morphology operations on the first binary image to obtain a second binary image, and then traverse each pixel in the second binary image, using the seed filling method to generate multiple gradient connected regions, and calculate the area of ​​each gradient connected region based on the number of pixels in each gradient connected region.

[0096] The position calculation module 304 is used after the connected region calculation module 303 obtains the area of ​​each gradient connected region. It determines whether the area of ​​each gradient connected region is greater than the set threshold. If the area of ​​any gradient connected region is greater than the set threshold, the gradient connected region with the area greater than the set threshold is retained. Based on the maximum and minimum x and y coordinates of the pixels in the region block with the largest area in the retained gradient connected region, the minimum bounding rectangle is solved. Based on the coordinate information of the upper left and lower right corners of the minimum bounding rectangle, the position information of the minimum bounding rectangle is obtained, and the location of the escalator is determined.

[0097] The control module 305 is used to control the robot to perform corresponding operations such as speed change and direction change according to the position of the escalator after the position calculation module 304 determines the position of the escalator.

[0098] In this embodiment, the grayscale image generation module 302 includes a grayscale image processing unit and an edge feature extraction unit, comprising:

[0099] The grayscale image processing unit is used to convert the acquired image to grayscale to generate a first grayscale image, and to perform Gaussian filtering and noise reduction processing on the first grayscale image to obtain a second grayscale image.

[0100] The edge feature extraction unit is used to extract the edge features of the second grayscale image according to the Sobel operator to generate an edge grayscale image.

[0101] In this embodiment, the connected region calculation module 303 includes a binary map generation unit and an area calculation unit, comprising:

[0102] The binary image generation unit is used to binarize the edge grayscale image according to a preset grayscale threshold to obtain a first binary image; then, it performs image morphology operations on the first binary image to remove noise points in the binary image to obtain a second binary image.

[0103] The area calculation unit is used to traverse each pixel in the second binary image, combine the seed filling method to generate multiple gradient connected regions, and calculate the area of ​​each gradient connected region based on the number of pixels in each gradient connected region.

[0104] In this embodiment, the binary image generation unit performs image morphological processing on the first binary image to remove noise points and obtain a second binary image, specifically including:

[0105] First, a dilation-erosion closing operation is performed on the first binary image based on the preset structuring element. Then, a new structuring element is set to perform an erosion-dilation opening operation. Finally, a dilation operation is performed to obtain the second binary image.

[0106] In this embodiment, the location calculation module 304 includes an area judgment unit and a positioning unit, comprising:

[0107] The area determination unit is used to determine whether the area of ​​each gradient connected region is greater than the set threshold.

[0108] If the area of ​​each region is less than the set threshold, then it is determined that there is no escalator in the image to be detected;

[0109] If any of the regions has an area greater than a set threshold, then it is determined that an escalator exists in the image to be detected.

[0110] The positioning unit is used to solve for the minimum bounding rectangle based on the gradient connected region and determine the position of the minimum bounding rectangle.

[0111] In this embodiment, the positioning unit solves for the minimum bounding rectangle based on the gradient connected region and determines the position of the minimum bounding rectangle, specifically including:

[0112] After obtaining the area of ​​each gradient connected region, retain the gradient connected regions whose area is greater than the set threshold, and select the gradient connected region block with the largest area among the retained gradient connected regions.

[0113] Based on the maximum and minimum x and y coordinates of the pixels in the gradient connected region with the largest area, the minimum bounding rectangle is calculated. Based on the coordinates of the upper left and lower right corners of the minimum bounding rectangle, the position information of the minimum bounding rectangle is obtained.

[0114] In this embodiment, the present invention separates escalator detection from the multi-target detection model. The image acquisition module acquires the image to be detected, and traditional image processing techniques are used to directly locate the target position, eliminating the need for prior data acquisition and annotation. Furthermore, the grayscale image generation module utilizes the Sobel operator, along with grayscale and morphological processing steps, to improve detection accuracy. The connected component calculation module and the position calculation module directly locate the escalator by determining the bounding box position, reducing the need for extensive data processing and minimizing the waste of computer resources. The escalator detection program is only required to be activated at the appropriate time to directly obtain the final result. Based on the obtained minimum bounding box and position information, the robot is controlled to perform subsequent speed changes, alterations to its action state, and other corresponding operations.

[0115] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A robot control method based on escalator detection, characterized in that, include: The robot acquires images of the object to be detected in real time using a camera mounted on it. Edge features of the image to be detected are extracted using the Sobel operator to generate an edge grayscale image. Specifically, the image to be detected is converted to grayscale to generate a first grayscale image, which is then subjected to Gaussian filtering and noise reduction to obtain a second grayscale image. The second grayscale image is then convolved with a preset template to obtain approximate horizontal and vertical brightness differences. Edge features in the second grayscale image are calculated based on a preset weighting formula and the approximate horizontal and vertical brightness differences to generate the edge grayscale image. Based on a preset binarization algorithm and image morphology operations, multiple gradient connected regions in the edge grayscale image are determined, and the area of ​​each gradient connected region is calculated. Specifically, the edge grayscale image is binarized according to a preset grayscale threshold to obtain a first binary image; image morphology operations are performed on the first binary image to remove noise points, resulting in a second binary image; each pixel in the second binary image is traversed, and multiple gradient connected regions are generated using a seed filling method. The area of ​​each gradient connected region is calculated based on the number of pixels within each gradient connected region. When any of the regions has an area greater than a set threshold, it is determined that an escalator exists in the image to be detected. Based on the gradient connected region with the largest region area, a minimum bounding rectangle is generated, and the position information of the minimum bounding rectangle is obtained. Based on the minimum bounding rectangle and the position information, the robot is controlled to perform corresponding operations.

2. The robot control method based on escalator detection as described in claim 1, characterized in that, Perform image morphological operations on the first binary image to remove noise points and obtain a second binary image, specifically including: First, a dilation-erosion closing operation is performed on the first binary image based on the preset structuring element. Then, a new structuring element is set to perform an erosion-dilation opening operation. Finally, a dilation operation is performed to obtain the second binary image.

3. The robot control method based on escalator detection as described in claim 1, characterized in that, After obtaining the area of ​​each gradient-connected region, the method further includes: Determine whether the area of ​​each gradient-connected region is greater than the set threshold. If the area of ​​each region is less than the set threshold, then it is determined that there is no escalator in the image to be detected; If any of the regions has an area greater than a set threshold, then it is determined that an escalator exists in the image to be detected.

4. The robot control method based on escalator detection as described in claim 3, characterized in that, The step of generating a minimum bounding rectangle based on the gradient-connected region with the largest area, and obtaining the position information of the minimum bounding rectangle, specifically involves: After obtaining the area of ​​each gradient connected region, retain the gradient connected regions whose area is greater than the set threshold, and select the gradient connected region block with the largest area among the retained gradient connected regions. Based on the maximum and minimum x and y coordinates of the pixels in the gradient connected region with the largest area, the minimum bounding rectangle is calculated. Based on the coordinates of the upper left and lower right corners of the minimum bounding rectangle, the position information of the minimum bounding rectangle is obtained.

5. A robot control device based on escalator detection, characterized in that, include: The system includes an image acquisition module, a grayscale image generation module, a connected component calculation module, a location calculation module, and a control module. The image acquisition module is used to acquire images to be detected in real time based on the camera terminal configured on the robot; The grayscale image generation module is used to extract edge features of the image to be detected using the Sobel operator to generate an edge grayscale image. The grayscale image generation module includes a grayscale image processing unit and an edge feature extraction unit. The grayscale image processing unit converts the acquired image to grayscale to generate a first grayscale image, and performs Gaussian filtering and noise reduction on the first grayscale image to obtain a second grayscale image. The edge feature extraction unit extracts edge features of the second grayscale image using the Sobel operator to generate an edge grayscale image. The second grayscale image is convolved with a preset template to obtain approximate horizontal and vertical brightness differences corresponding to the second grayscale image. Edge features in the second grayscale image are calculated based on a preset weighting formula and the approximate horizontal and vertical brightness differences to generate the edge grayscale image. The connected region calculation module is used to determine multiple gradient connected regions in the edge grayscale image according to a preset binarization algorithm and image morphological operations, and calculate the area of ​​each gradient connected region. The connected region calculation module includes a binary image generation unit and an area calculation unit. The binary image generation unit is used to binarize the edge grayscale image according to a preset grayscale threshold to obtain a first binary image; then, it performs image morphological operations on the first binary image to remove noise points, obtaining a second binary image. The area calculation unit is used to traverse each pixel in the second binary image, combine a seed filling method to generate multiple gradient connected regions, and calculate the area of ​​each gradient connected region based on the number of pixels within each gradient connected region. The position calculation module is used to determine that there is an escalator in the image to be detected when any area of ​​the region is greater than a set threshold, and to generate a minimum bounding rectangle based on the gradient connected region with the largest area, and to obtain the position information of the minimum bounding rectangle. The control module is used to control the robot to perform corresponding operations based on the minimum bounding rectangle and the position information.

Images