Robot escape method and system based on image sensor

Abstract

The invention relates to a robot escape method and system based on an image sensor, and the method comprises the following steps: collecting front and rear environment images through the image sensor of a robot body, and carrying out the semantic segmentation and spatial distance calculation of the front and rear environment images, so as to obtain front and rear passable regions; and carrying out trafficability quantitative calculation based on the region, obtaining forward and backward trafficability scores, and determining a target escape direction through numerical comparison. According to the target direction, the escape motion instruction is generated, the robot motion controller executes the instruction, escape is achieved, and the technical problem that when the robot executes a task in a complex environment, the robot is prone to falling into difficulty in the moving process is solved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a robot escape method and system based on image sensors. Background Technology

[0002] With the rapid development of robotics technology, various mobile robots have been widely applied in industrial production, disaster relief, home services, outdoor exploration, and other fields, becoming important tools for improving production efficiency, ensuring operational safety, and optimizing life experiences. In practical applications, robots often need to perform tasks in unstructured environments. These environments are characterized by complex terrain, irregular obstacle distribution, and dynamically changing environmental information, making robots prone to getting stuck during movement, such as being surrounded by obstacles, trapped in low-lying areas, or encountering path blockages. If a robot fails to escape a predicament, it can not only lead to mission interruption and equipment damage, but in critical scenarios such as disaster relief, it can also cause more serious loss of life and property. Therefore, developing efficient and reliable robot escape technology has become one of the core needs that urgently needs to be addressed in the current mobile robotics field. Summary of the Invention

[0003] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a robot escape method based on an image sensor, comprising the following steps: The robot acquires environmental images of the area in front of and behind it using image sensors mounted on its body. Semantic segmentation and spatial distance calculation are performed on the front and rear environmental images respectively to obtain the passable front and rear areas. Based on the passable area in front and the passable area behind, passability quantification calculations are performed to obtain forward passability score and backward passability score; The forward mobility score and the backward mobility score are compared numerically to obtain the target escape direction. An escape motion command is generated based on the target escape direction and executed by the robot motion controller to enable the robot to get out of the predicament in the environment.

[0004] Furthermore, image sensors mounted on the robot body acquire environmental images of the area in front of and behind the robot, including: The robot takes multiple images of the area in front of it by using an image sensor mounted on the robot body. These images are then stitched together to obtain a complete image of the area in front of it. Multiple images of the rear area of ​​the robot's environment are captured from multiple angles to obtain multiple original images of the rear area. These multiple original images of the rear area are then stitched together to obtain a complete image of the rear area. The complete frontal region image and the complete rearal region image are subjected to sharpness enhancement processing to obtain the frontal region environmental image and the rearal region environmental image.

[0005] Furthermore, semantic segmentation and spatial distance calculation are performed on the front and rear environmental images respectively to obtain the passable front and rear areas, including: Pixel grayscale values ​​are analyzed for the front and rear environmental images respectively. By setting a grayscale threshold range, the areas in the image with pixel grayscale values ​​within the grayscale threshold range are initially marked as suspected ground areas, and the areas with pixel grayscale values ​​exceeding the grayscale threshold range are initially marked as suspected non-ground areas, thus obtaining a preliminary marked image. Based on the preliminary marked image, texture features are extracted from suspected ground areas. Suspected ground areas whose texture features match the preset ground texture features are identified as ground areas, and the position coordinates of the ground areas in the image are recorded to obtain the ground areas. Edge detection is performed on the portion of the ground region excluding the ground region. The edge contour of the object is determined by detecting abrupt changes in pixel grayscale values, and the region where the edge contour is closed is marked as the obstacle region. A continuous area of ​​a certain width in the ground region that is not covered by obstacle areas is defined as the passage area, and the boundary coordinates of the ground region are extracted. The depth measurement unit of the image sensor performs point-by-point distance measurement on the boundary coordinates to obtain boundary distance data, and performs nearest distance filtering on the obstacle area based on the boundary distance data to obtain obstacle distance values. The distance between the left and right boundaries of the passage area is calculated based on the boundary coordinates, and the distance values ​​of the obstacles and the distance between the left and right boundaries are integrated to obtain the passable area in front and the passable area behind.

[0006] Furthermore, based on the passable area ahead and the passable area behind, passability quantification calculations are performed to obtain forward passability scores and backward passability scores, including: Step S41: Perform numerical analysis on the distance values ​​of obstacles in the passable area ahead, compare the obstacle distance values ​​with the robot's preset safe passage distance threshold, if the obstacle distance value is greater than the safe passage distance threshold, mark it as a safe passage state, if it is less than or equal to the safe passage distance threshold, mark it as a risk state, and obtain the forward passage marking state; Step S42: Evaluate the width of the left and right boundary spacing in the forward passage marking state. Compare the left and right boundary spacing with the robot's own width. If the left and right boundary spacing is greater than the robot's own width by a certain proportion, it is marked as a sufficient width state. If it is less than or equal to the proportion, it is marked as a tight width state, thus obtaining the forward width marking state. Step S43: Based on the forward passage mark status and the forward width mark status, a comprehensive score is calculated. A higher score is assigned to the safe passage status, a lower score is assigned to the risky status, a higher score is assigned to the wide enough status, and a lower score is assigned to the narrow width status. The scores corresponding to each status are added together to obtain the forward passability score. Step S44: Repeat the above steps S1 to S3 for processing the forward passable area for the backward passable area to obtain the backward passability score.

[0007] Furthermore, the forward mobility score and the backward mobility score are numerically compared to obtain the target escape direction, including: Compare the numerical values ​​of the forward mobility score and the backward mobility score; If the forward mobility score is greater than the backward mobility score, then the direction corresponding to the forward mobility score is determined as the target escape direction; If the forward mobility score is less than the backward mobility score, then the direction corresponding to the backward mobility score is determined as the target escape direction; If the forward passability score and the backward passability score are equal, then the obstacle areas in the forward passability area and the backward passability area are divided into grids to obtain an obstacle distribution grid map, and the obstacle proportion of each grid in the obstacle distribution grid map is calculated to obtain the grid occupancy rate. Based on the grid occupancy rate, the passable areas in front and behind are statistically accumulated to obtain the obstacle distribution density. The obstacle distribution density is then compared numerically, and the direction with the lower obstacle distribution density is selected as the target escape direction.

[0008] Furthermore, based on the target escape direction, an escape movement command is generated, including: The centerline of the passable area corresponding to the escape direction of the target is extracted to obtain the center coordinate sequence of the channel, and the distance between adjacent coordinate points of the center coordinate sequence of the channel is calculated to obtain the segmented distance parameter. The robot's movement speed is set segment by segment based on the segmented distance parameter, and the movement speed is bound to the target escape direction by a movement direction identifier to obtain the escape movement command.

[0009] Furthermore, the robot's motion controller executes the escape movement commands to enable the robot to overcome its environmental difficulties, including: The direction identification in the escape movement command is processed to obtain a movement steering control signal, and the speed control sequence in the escape movement command is converted by pulse width modulation to obtain a movement speed control signal; The motion steering control signal and the motion speed control signal are applied to the robot's motion controller through the drive circuit in the robot, so as to enable the robot to get out of the predicament of the environment.

[0010] The present invention also provides a robot extrication system based on an image sensor, comprising: The acquisition module is used to acquire environmental images of the front and rear areas of the robot's environment through image sensors installed on the robot body. The first calculation module is used to perform semantic segmentation processing and spatial distance calculation on the front area environmental image and the rear area environmental image respectively to obtain the front passable area and the rear passable area. The second calculation module is used to perform passability quantification calculations based on the passable area in front and the passable area behind, respectively, to obtain the passability score for moving forward and the passability score for moving backward. The comparison module is used to compare the forward drivability score with the backward drivability score to obtain the target escape direction, generate an escape motion command based on the target escape direction, and execute the escape motion command through the robot motion controller to enable the robot to get out of the predicament in the environment.

[0011] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0012] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above methods.

[0013] This invention provides a robot escape method based on an image sensor, comprising the following steps: acquiring environmental images of the front and rear areas of the robot's environment using an image sensor mounted on the robot body; performing semantic segmentation and spatial distance calculation on the front and rear environmental images respectively to obtain a passable area in front and a passable area in rear; performing passability quantization calculation on the passable areas in front and rear respectively to obtain a forward passability score and a backward passability score; comparing the forward passability score and the backward passability score to obtain a target escape direction; and then, based on the target... The system generates escape direction and movement commands, which are then executed by the robot's motion controller to help the robot escape its predicament. This solves the technical problem that robots are prone to getting stuck in complex environments. It achieves direct determination of the target escape direction and generation of movement commands through numerical comparison, resulting in a simple and clear decision-making process and rapid response to challenging situations. Furthermore, the direct execution of commands by the robot's motion controller reduces intermediate data conversion and logical judgment steps, enabling the robot to quickly initiate escape actions. This effectively improves escape response speed and overall success rate, and reduces the risk of exacerbating the predicament due to decision-making delays. Attached Figure Description

[0014] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating the steps of the robot escape method based on an image sensor in an embodiment of the present invention. Figure 2 This is a structural block diagram of the robot escape system based on an image sensor in an embodiment of the present invention; Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0015] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0017] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0018] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0019] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0020] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0021] Reference Figure 1 This invention provides a robot escape method based on an image sensor, comprising the following steps: Step S1: Acquire environmental images of the area in front of the robot and the area behind the robot using image sensors installed on the robot body.

[0022] Specifically, the installation locations on the robot body are pre-selected, typically embedding matching image sensors at the center of the front and rear bumpers respectively. The sensor lenses must face directly in front of and behind the robot to ensure that the field of view is unobstructed by any mechanical structure. After installation, the communication protocol between the sensors and the robot's main control unit is adapted, generally using a USB or Ethernet interface to establish a data transmission link. At the same time, the two sensors are time-synchronized to avoid timing deviations. During actual data acquisition, the main control unit sends a trigger command, and the two sensors simultaneously start image acquisition. The front sensor captures environmental images within a 120° field of view in front of the robot's direction of travel, while the rear sensor captures environmental images within a 120° field of view in the opposite direction of travel. For example, in a narrow indoor passage, the front sensor can capture images of the distribution of obstacles in front of the passage, while the rear sensor simultaneously captures images of whether there is any space to avoid obstacles behind the passage. The acquired image data is compressed and transmitted to the main control unit in real time for storage and subsequent processing.

[0023] Step S2: Perform semantic segmentation and spatial distance calculation on the front and rear environmental images respectively to obtain the passable front and rear areas.

[0024] Specifically, in one embodiment, the semantic segmentation process uses the U-Net network model. The input environmental images of the front and rear regions are first normalized to 512×512 pixels, and then features are extracted through three convolutional layers with a kernel size of 3×3. The convolution stride is set to 1, and the edge padding uses SamePadding. Finally, the segmentation result is output through the Softmax activation function—the image pixels are divided into two categories: passable (label 0) and impassable (label 1). For example, in an indoor scene, ground pixels are labeled as label 0, and wall and obstacle pixels are labeled as label 1; empty... The spatial distance calculation employs a monocular vision ranging algorithm. Based on the semantically segmented image, feature points (such as intersections of ground textures) within the passable area are selected. After correction using the camera intrinsic parameter matrix (fx=1200, fy=1200, cx=256, cy=256, in pixels) and distortion coefficients (k1=-0.3, k2=0.1), the distance from the feature points to the image sensor is calculated by combining the mapping relationship between pixel coordinates and actual physical dimensions. Finally, the distance data of all feature points within the passable area are summarized to obtain the spatial range information of the passable areas in front and behind. In another embodiment, the semantic segmentation can also use the SegNet model, normalizing the input image to 256×256 pixels and adjusting the number of convolutional layers to 4. The spatial distance calculation can use a binocular vision ranging scheme, acquiring disparity maps through two synchronously acquired image sensors, and then calculating the actual distance by combining it with a baseline distance of 60mm, meeting the requirements of different hardware configurations.

[0025] Step S3: Based on the passable area in front and the passable area behind, perform passability quantification calculation to obtain the passability score for moving forward and the passability score for moving backward.

[0026] Specifically, the passability quantification calculation is based on the spatial range information of the passable areas ahead and behind obtained earlier. It focuses on three key indicators: the percentage of passable area, the maximum passable width within the area, and the obstacle distribution density within the area. The weights of each indicator are set to 0.4, 0.3, and 0.3 respectively (the weights can be fine-tuned according to the actual scenario). For example, in a narrow indoor passage scenario, if the passable area ahead accounts for 35% of the total image area, the maximum passable width is 0.8m, and the obstacle distribution density is 0.2, the original data for each indicator is first normalized to the [0,1] interval (e.g., a normalized value of 0.35 corresponds to an area percentage of 35%). Then, the passability score is calculated according to the formula: "Quantification score = Area percentage × 0.4 + Maximum passable width × 0.3 + (1 - Obstacle distribution density) × 0.3". The passable area behind is processed using the same indicators, weights, and calculation logic to obtain the passability score for retreating. It should be noted here that the normalization adopts the min-max standardization. Taking the area ratio as an example, the formula is "normalized value = (original area ratio - minimum ratio threshold 0%) / (maximum ratio threshold 100% - minimum ratio threshold 0%)", which ensures that the numerical range of different indicators is consistent.

[0027] Step S4: Compare the forward mobility score with the backward mobility score to obtain the target escape direction, generate an escape motion command based on the target escape direction, and execute the escape motion command through the robot motion controller to enable the robot to get out of the predicament in the environment.

[0028] Specifically, the forward and backward mobility scores obtained earlier are retrieved, and a threshold of 0.5 is set. If the forward mobility score (e.g., 0.6) is higher than the backward mobility score (e.g., 0.3) and also exceeds the threshold, the target escape direction is forward. If the backward score is higher and exceeds the threshold, backward is selected as the target direction. If both are below the threshold, a fine-tuning of the posture in place and a reassessment can be triggered. After determining the direction, the main control unit generates escape motion commands according to a preset protocol. For example, the forward direction command includes a speed parameter of 0.2 m / s and a duration of 2 seconds, while the backward command includes a reverse speed of 0.15 m / s and a duration of 1.8 seconds. The commands are encapsulated using the CAN bus protocol and transmitted to the robot motion controller. After parsing the commands, the controller drives the motors to execute the corresponding actions, completing the escape.

[0029] In a specific embodiment, image sensors mounted on the robot body acquire environmental images of the area in front of and behind the robot, including: The robot takes multiple images of the area in front of it by using an image sensor mounted on the robot body. These images are then stitched together to obtain a complete image of the area in front of it. Multiple images of the rear area of ​​the robot's environment are captured from multiple angles to obtain multiple original images of the rear area. These multiple original images of the rear area are then stitched together to obtain a complete image of the rear area. The complete frontal region image and the complete rearal region image are subjected to sharpness enhancement processing to obtain the frontal region environmental image and the rearal region environmental image.

[0030] Specifically, the installation layout of the image sensors is first determined. One CMOS image sensor is installed on each of the left and right sides and the middle of the front of the robot body, and three sensors of the same model are installed in the corresponding positions at the rear. The focal length of each lens is set to 8mm, the field of view is 60°, and the overlap rate of the field of view of adjacent sensors is controlled at 15%-20% to ensure no blind spots in multi-angle shooting. During the shooting phase, the main control unit sends a synchronous trigger signal, and the three front sensors start shooting simultaneously, acquiring original images of the front area in the directions of directly in front of the robot, 30° to the left front, and 30° to the right front, respectively. The resolution of each image is set to 1920×1080 pixels, and the frame rate is 30fps. The three rear sensors simultaneously acquire original images of the rear area in the directions of directly behind, 30° to the left rear, and 30° to the right rear, with the same parameters as the front.

[0031] During image stitching, feature points are extracted from each original image of the foreground region. A feature point matching threshold of 0.7 is set, and mismatched points are removed (iteration count set to 1000 times, inlier threshold 2 pixels). Image registration and fusion are then performed based on the homography matrix to obtain the complete foreground region image. The original images of the background region are processed using the same stitching logic to obtain the complete background region image. For sharpness enhancement, the complete foreground and background region images are first converted to the HSV color space. Gaussian filtering (Gaussian kernel size 5×5, standard deviation 1.2) is applied to the luminance channel to separate the illumination and reflection components.

[0032] The clarity is improved by stretching the dynamic range of the reflection components, and finally converted back to RGB space to obtain environmental images of the front and rear areas that meet the requirements. For example, if the robot is in an aisle between warehouse shelves, the front sensor can collect partial images of the left and right shelves. After stitching, the entire aisle in front can be presented. After enhancement processing, the edges of the shelves and the texture of the ground can be clearly distinguished, avoiding the loss of details due to dim lighting.

[0033] In a specific embodiment, semantic segmentation and spatial distance calculation are performed on the front and rear environmental images respectively to obtain a passable area in front and a passable area in rear, including: Pixel grayscale values ​​are analyzed for the front and rear environmental images respectively. By setting a grayscale threshold range, the areas in the image with pixel grayscale values ​​within the grayscale threshold range are initially marked as suspected ground areas, and the areas with pixel grayscale values ​​exceeding the grayscale threshold range are initially marked as suspected non-ground areas, thus obtaining a preliminary marked image. Based on the preliminary marked image, texture features are extracted from suspected ground areas. Suspected ground areas whose texture features match the preset ground texture features are identified as ground areas, and the position coordinates of the ground areas in the image are recorded to obtain the ground areas. Edge detection is performed on the portion of the ground region excluding the ground region. The edge contour of the object is determined by detecting abrupt changes in pixel grayscale values, and the region where the edge contour is closed is marked as the obstacle region. A continuous area of ​​a certain width in the ground region that is not covered by obstacle areas is defined as the passage area, and the boundary coordinates of the ground region are extracted. The depth measurement unit of the image sensor performs point-by-point distance measurement on the boundary coordinates to obtain boundary distance data, and performs nearest distance filtering on the obstacle area based on the boundary distance data to obtain obstacle distance values. The distance between the left and right boundaries of the passage area is calculated based on the boundary coordinates, and the distance values ​​of the obstacles and the distance between the left and right boundaries are integrated to obtain the passable area in front and the passable area behind.

[0034] Specifically, the semantic segmentation process uses an improved U-Net model. First, the environmental images of the foreground and background regions obtained earlier are uniformly scaled to 512×512 pixels and normalized before being input into the model (pixel values ​​divided by 255 to map to the [0,1] interval). The model extracts image features through a 4-layer encoding module with a 3×3 convolutional kernel and a stride of 1. Each encoding layer is followed by a BN layer and a ReLU activation function, and then a 4-layer decoding module completes feature fusion. Finally, a Softmax classifier outputs a 3-channel semantic map, corresponding to the ground region (pixel label 0), obstacle region (pixel label 1), and channel region (pixel label 2), respectively. Next, the Canny edge detection algorithm is used to extract the boundary coordinates of the ground region. Double thresholds of 50 and 150 are set to obtain the set of pixel coordinates of the boundary points. For example, the coordinates of a point on the ground region boundary are (200, 350), and a coordinate system is established with the top left corner of the image as the origin.

[0035] In one embodiment, a Time-of-Flight (TOF) depth measurement unit integrated with an image sensor measures the distance to each boundary point at a frequency of 30Hz. During each measurement, the pixel coordinates of the boundary points are converted into three-dimensional coordinates in the camera coordinate system using a camera intrinsic parameter matrix (fx=1100, fy=1100, cx=256, cy=256), thus obtaining the boundary distance data (unit: m). The distance data corresponding to all pixels within the obstacle area are then sorted, and the minimum value is selected as the obstacle distance value. For example, if the selected obstacle distance value is 1.2m, it represents the distance between the robot and the nearest obstacle.

[0036] In one embodiment, the left and right boundary spacing of the channel area is calculated based on the boundary coordinates of the ground area. Specifically, the leftmost and rightmost boundary points on the same horizontal row of pixels in the channel area are taken, the pixel distance between the two points is calculated, and then the actual left and right boundary spacing is obtained by combining the conversion ratio between the pixel spacing and the actual physical size (1 pixel corresponds to 0.002m). For example, when the pixel distance is 400, the actual spacing is 0.8m. Finally, the obtained obstacle distance values ​​and left and right boundary spacing are integrated to form data on the passable area in front and the passable area behind, which include distance and width information. For example, the passable area data in front includes an obstacle distance value of 1.2m, a left and right boundary spacing of 0.8m, and the coordinate range of the corresponding area. First, pixel grayscale value analysis is performed on the environmental images of the front and rear areas, and the grayscale threshold range is set to [180, 250] (which can be finely adjusted according to indoor and outdoor lighting, such as [200, 250] under strong outdoor light). Areas in the image whose grayscale values ​​fall within this range are initially marked as suspected ground areas, and those outside are marked as suspected non-ground areas. This gives us the preliminary marked image. Next, based on this preliminary labeled image, texture features are extracted from suspected ground areas. The LBP algorithm is used to calculate the texture histogram. The preset ground texture features are those where the histogram peaks are concentrated in the range of [50, 150]. Suspected ground areas that meet this condition are identified as ground areas, and their image coordinates (with the top left corner of the image as the origin, in pixels) are recorded. For example, the coordinate range of a certain ground area is (100, 200) to (800, 600). Then, edge detection is performed on the parts of the preliminary labeled image other than the ground areas using the Canny operator. The dual thresholds are set to 50 and 150. By detecting abrupt changes in pixel grayscale values, the edge contours of objects are delineated. Areas with closed contours are marked as obstacle areas. Closed contours such as shelves and walls will be identified as obstacles. Finally, in the ground area, areas that are not covered by obstacles and have a continuous width of not less than 0.5m are selected as channel areas. Here, the width is calculated based on 1 pixel corresponding to 0.002m. For example, if the width of a continuous area is 250 pixels, it is actually 0.5m. If it meets the requirements, it is determined as a channel area.

[0037] In a specific embodiment, mobility quantification calculations are performed based on the forward and backward accessible areas respectively to obtain forward mobility scores and backward mobility scores, including: Step S41: Perform numerical analysis on the distance values ​​of obstacles in the passable area ahead, compare the obstacle distance values ​​with the robot's preset safe passage distance threshold, if the obstacle distance value is greater than the safe passage distance threshold, mark it as a safe passage state, if it is less than or equal to the safe passage distance threshold, mark it as a risk state, and obtain the forward passage marking state; Step S42: Evaluate the width of the left and right boundary spacing in the forward passage marking state. Compare the left and right boundary spacing with the robot's own width. If the left and right boundary spacing is greater than the robot's own width by a certain proportion, it is marked as a sufficient width state. If it is less than or equal to the proportion, it is marked as a tight width state, thus obtaining the forward width marking state. Step S43: Based on the forward passage mark status and the forward width mark status, a comprehensive score is calculated. A higher score is assigned to the safe passage status, a lower score is assigned to the risky status, a higher score is assigned to the wide enough status, and a lower score is assigned to the narrow width status. The scores corresponding to each status are added together to obtain the forward passability score. Step S44: Repeat the processing steps S41 to S43 for the forward passable area for the backward passable area to obtain the backward passability score.

[0038] Specifically, in practice, the implementation of the passability quantification calculation needs to connect with the passable area data for the front and rear obtained earlier. First, step S41 is executed to retrieve the obstacle distance values ​​for the passable area in front. At the same time, the robot's preset safe passage distance threshold is set to 0.8m. The obstacle distance value is directly compared with this threshold. For example, if the measured obstacle distance is 1.2m, it is marked as a safe passage state if it is greater than the threshold. If it is only 0.6m, it is marked as a risky state, thus obtaining the forward passage marking state. Next, step S42 is advanced for width evaluation. First, the robot's own width is determined to be 0.6m. The "certain proportion" in the step is set to 1.2 times, and the minimum safe width of 0.72m is calculated. Then, the left and right boundary spacing corresponding to the forward passage marking state is compared with 0.72m. A spacing of 0.8m indicates sufficient width, while 0.7m indicates tight width, thus obtaining the forward width marking state. Next, step S43 involves a comprehensive scoring process, with specific scores assigned directly: 8 points for a safe passage, 3 points for a risky passage, 7 points for a sufficient width, and 2 points for a narrow passage. The scores for both states are added together to calculate the forward passability score; for example, safe and sufficient width equals 15 points, while risky and narrow passage equals 5 points. Finally, step S44 is executed. When processing the passable area behind, parameters do not need to be reset; the comparison and scoring logic from the previous steps is directly applied to calculate the backward passability score. For example, if the distance to the obstacle behind is 1.0m and the distance between the left and right boundaries is 0.9m, the final score is also 15 points.

[0039] In a specific embodiment, the forward drivability score and the backward drivability score are numerically compared to obtain the target escape direction, including: Compare the numerical values ​​of the forward mobility score and the backward mobility score; If the forward mobility score is greater than the backward mobility score, then the direction corresponding to the forward mobility score is determined as the target escape direction; If the forward mobility score is less than the backward mobility score, then the direction corresponding to the backward mobility score is determined as the target escape direction; If the forward passability score and the backward passability score are equal, then the obstacle areas in the forward passability area and the backward passability area are divided into grids to obtain an obstacle distribution grid map, and the obstacle proportion of each grid in the obstacle distribution grid map is calculated to obtain the grid occupancy rate. Based on the grid occupancy rate, the passable areas in front and behind are statistically accumulated to obtain the obstacle distribution density. The obstacle distribution density is then compared numerically, and the direction with the lower obstacle distribution density is selected as the target escape direction.

[0040] Specifically, determining the target escape direction requires first retrieving the previously calculated forward and backward mobility scores, and then directly comparing the two scores without using any additional complex algorithms. If the forward mobility score is higher than the backward mobility score, for example, 15 points for forward mobility and 5 points for backward mobility, then the forward direction is directly determined as the target escape direction. If the score for moving forward is lower than the score for moving backward, for example, 5 points forward and 15 points backward, then the direction of moving backward is set as the target direction for escaping the predicament. When encountering two identical scores, such as 10 points each, it's not enough to just look at the score. We need to statistically analyze the obstacle distribution density in both the forward and backward passable areas. First, define a fixed calculation area, taking a 1m x 1m unit area within the passable area. Count the number of obstacle pixels within this area, then divide the number of pixels by the total number of pixels per unit area to obtain the specific obstacle distribution density value. Finally, compare the density values ​​in the forward and backward areas, selecting the direction with the lower density as the target escape direction. For example, if the forward density is 0.2 and the backward density is 0.3, then forward is selected as the target direction. First, retrieve the obstacle area data corresponding to the forward and backward passable areas. Divide these two areas into grids, setting the division criteria according to the image pixel size. For example, divide the image into a 20x20 equal grid, with each grid size being 50x50 pixels (this can be fine-tuned according to the image resolution; this division is more suitable for 1920x1080 resolution). After division, you will obtain an obstacle distribution grid map. Next, the obstacle percentage for each grid is calculated. First, the number of pixels belonging to the obstacle area within a single grid is counted. Then, this number is divided by the total number of pixels in a single grid (50 × 50 = 2500 pixels) to obtain the grid occupancy rate of each grid. For example, if a grid has 1250 obstacle pixels, the percentage is 0.5. Then, based on the grid occupancy rate, the passable areas in front and behind are summed and averaged. This average is the obstacle distribution density. For example, if the sum of the occupancy rates of all grids in the front area is 20 (20 grids in total), the density is 1.0; if the sum in the rear area is 12, the density is 0.6. Finally, the obstacle distribution densities in front and behind are compared, and the direction with the lower density is selected as the target escape direction. In the example above, with a density of 1.0 in front and 0.6 in the rear, the rear is selected as the target escape direction.

[0041] In a specific embodiment, generating an escape movement command based on the target escape direction includes: The centerline of the passable area corresponding to the escape direction of the target is extracted to obtain the center coordinate sequence of the channel, and the distance between adjacent coordinate points of the center coordinate sequence of the channel is calculated to obtain the segmented distance parameter. The robot's movement speed is set segment by segment based on the segmented distance parameter, and the movement speed is bound to the target escape direction by a movement direction identifier to obtain the escape movement command.

[0042] Specifically, in one embodiment, when extracting the channel centerline, the information of the passable area corresponding to the target escape direction is first retrieved (for example, if the target direction is determined to be forward, the left and right boundary coordinates of the passable area ahead are retrieved). The binary image of the passable area is iteratively refined, and the iteration termination condition is set to no pixels that can be deleted, resulting in a channel centerline with a width of one pixel. Then, the centerline is sampled from near to far according to the robot's movement direction to obtain the channel center coordinate sequence, with the coordinate format being (x, y, z), where z is the depth value of the distance image sensor. For example, the sampled sequence is (256, 280, 0.5), (256, 282, 1.0), (256, 281, 1.5)... Then, the Euclidean distance between adjacent coordinate points is calculated as the segmented distance parameter, with the formula √[(x2-x1)²+(y2-y1)²+(z2-z1)²]. For example, the segmented distance parameter for the first two adjacent points is calculated to be 0.51m, and the subsequent adjacent points are calculated to be 0.5m.

[0043] In one embodiment, when setting the movement speed based on segmented distance parameters, speed levels are first divided: when the segmented distance parameter is ≥ 0.8m, the movement speed is set to 0.3m / s; when 0.4m ≤ segmented distance parameter < 0.8m, the speed is set to 0.15m / s; and when the segmented distance parameter < 0.4m, the speed is set to 0.08m / s. This adapts to the safety passage requirements of different passage segments. Then, movement direction identifiers are bound. If the target escape direction is forward, a field "Direction Identifier: Forward" is added to the speed parameter; if it is backward, a field "Direction Identifier: Backward" is added. For example, if a segmented distance parameter is 0.6m, corresponding to a speed of 0.15m / s, and the target direction is forward, the final generated escape movement command is "Speed: 0.15m / s, Direction Identifier: Forward". Commands corresponding to multiple parameters are integrated in coordinate sequence to form a complete escape movement command set. For example, if the channel center coordinate sequence corresponds to three segment distance parameters, namely 0.9m, 0.5m, and 0.3m, then it will generate continuous commands such as "0.3m / s forward", "0.15m / s forward", and "0.08m / s forward".

[0044] In a specific embodiment, the robot's motion controller executes the escape movement command to enable the robot to overcome the predicament in its environment, including: The direction identification in the escape movement command is processed to obtain a movement steering control signal, and the speed control sequence in the escape movement command is converted by pulse width modulation to obtain a movement speed control signal; The motion steering control signal and the motion speed control signal are applied to the robot's motion controller through the drive circuit in the robot, so as to enable the robot to get out of the predicament of the environment.

[0045] Specifically, in one embodiment, when executing the escape movement command, the direction indicator in the command is first processed for direction determination. The main control unit has built-in direction determination logic. When it reads "direction indicator: forward", it outputs a high-level motion steering control signal (voltage value 3.3V); when it reads "direction indicator: backward", it outputs a low-level motion steering control signal (voltage value 0V), thus distinguishing between forward and backward steering. Next, the speed control sequence is processed. The speed parameters in the command are converted into motion speed control signals through pulse width modulation (PWM). The PWM carrier frequency is set to 1kHz, and the duty cycle has a linear relationship with the speed—for example, 0.3m / s corresponds to 80% duty cycle, 0.15m / s corresponds to 40% duty cycle, and 0.08m / s corresponds to 20% duty cycle. The duty cycle is calculated using the formula "duty cycle = (target speed / maximum speed 0.3m / s) × 80%".

[0046] In one embodiment, the robot's drive circuit uses an H-bridge drive module. It first receives the aforementioned motion steering control signal and motion speed control signal, and then uses an optocoupler isolation circuit to perform anti-interference processing on the signals to prevent noise from affecting control accuracy. Subsequently, the drive circuit applies the processed signals to the robot's motion controller. The MCU inside the motion controller analyzes the signals and outputs drive current to the robot's walking motor—the forward steering signal triggers the motor's forward rotation drive circuit, and the backward signal triggers the reverse rotation circuit. The speed control signal corresponding to the PWM duty cycle adjusts the motor's output power, thereby controlling the robot to move according to the commanded speed and direction. For example, when receiving the "0.15m / s forward" command, the drive circuit outputs a PWM signal with a 40% duty cycle and a high-level steering signal, causing the motor to rotate forward and drive the robot forward at the corresponding speed, gradually overcoming the obstacle.

[0047] The above describes the robot obstacle avoidance method based on an image sensor in the embodiments of the present invention. The following describes the robot obstacle avoidance system based on an image sensor in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the robot escape system based on an image sensor in this invention includes: The acquisition module 21 is used to acquire environmental images of the front and rear areas of the robot's environment through an image sensor installed on the robot body. The first calculation module 22 is used to perform semantic segmentation processing and spatial distance calculation on the front area environmental image and the rear area environmental image respectively to obtain the front passable area and the rear passable area. The second calculation module 23 is used to perform passability quantification calculations based on the passable area in front and the passable area behind, respectively, to obtain the passability score for moving forward and the passability score for moving backward. The comparison module 24 is used to compare the forward drivability score with the backward drivability score to obtain the target escape direction, generate an escape motion command based on the target escape direction, and execute the escape motion command through the robot motion controller to enable the robot to get rid of the predicament in the environment.

[0048] In this embodiment, the specific implementation of each unit in the above system embodiment is the same as that in the above method embodiment, and will not be repeated here.

[0049] like Figure 3 As shown in the diagram, this embodiment of the invention provides a structural schematic block diagram of a computer device, including: At least one processor; At least one memory for storing at least one program; When at least one program is executed by at least one processor, the at least one processor implements the above-described image sensor-based robot escape method.

[0050] It is evident that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented in this device embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0051] Furthermore, this application also discloses a computer program product or computer program stored in a computer-readable storage medium. A processor of a computer device can read the computer program from the computer-readable storage medium and execute the computer program, causing the computer device to perform the aforementioned image sensor-based robot escape method. Similarly, the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0052] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A robot escape method based on an image sensor, characterized in that, Includes the following steps: The robot acquires environmental images of the area in front of and behind it using image sensors mounted on its body. Semantic segmentation and spatial distance calculation are performed on the front and rear environmental images respectively to obtain the passable front and rear areas. Based on the passable area in front and the passable area behind, passability quantification calculations are performed to obtain forward passability score and backward passability score; The forward mobility score and the backward mobility score are compared numerically to obtain the target escape direction. An escape motion command is generated based on the target escape direction and executed by the robot motion controller to enable the robot to get out of the predicament in the environment.

2. The robot escape method based on an image sensor according to claim 1, characterized in that, Image sensors mounted on the robot body acquire environmental images of the area in front of and behind the robot, including: The robot takes multiple images of the area in front of it by using an image sensor mounted on the robot body. These images are then stitched together to obtain a complete image of the area in front of it. Multiple images of the rear area of ​​the robot's environment are captured from multiple angles to obtain multiple original images of the rear area. These multiple original images of the rear area are then stitched together to obtain a complete image of the rear area. The complete frontal region image and the complete rearal region image are subjected to sharpness enhancement processing to obtain the frontal region environmental image and the rearal region environmental image.

3. The robot escape method based on an image sensor according to claim 1, characterized in that, Semantic segmentation and spatial distance calculation are performed on the front and rear environmental images respectively to obtain the passable front and rear areas, including: Pixel grayscale values ​​are analyzed for the front and rear environmental images respectively. By setting a grayscale threshold range, the areas in the image with pixel grayscale values ​​within the grayscale threshold range are initially marked as suspected ground areas, and the areas with pixel grayscale values ​​exceeding the grayscale threshold range are initially marked as suspected non-ground areas, thus obtaining a preliminary marked image. Based on the preliminary marked image, texture features are extracted from suspected ground areas. Suspected ground areas whose texture features match the preset ground texture features are identified as ground areas, and the position coordinates of the ground areas in the image are recorded to obtain the ground areas. Edge detection is performed on the portion of the ground region excluding the ground region. The edge contour of the object is determined by detecting abrupt changes in pixel grayscale values, and the region where the edge contour is closed is marked as the obstacle region. A continuous area of ​​a certain width in the ground region that is not covered by obstacle areas is defined as the passage area, and the boundary coordinates of the ground region are extracted. The depth measurement unit of the image sensor performs point-by-point distance measurement on the boundary coordinates to obtain boundary distance data, and performs nearest distance filtering on the obstacle area based on the boundary distance data to obtain obstacle distance values. The distance between the left and right boundaries of the passage area is calculated based on the boundary coordinates, and the distance values ​​of the obstacles and the distance between the left and right boundaries are integrated to obtain the passable area in front and the passable area behind.

4. The robot escape method based on an image sensor according to claim 3, characterized in that, Based on the passable areas ahead and behind, passability quantification calculations are performed to obtain forward passability scores and backward passability scores, including: Step S41: Perform numerical analysis on the distance values ​​of obstacles in the passable area ahead, compare the obstacle distance values ​​with the robot's preset safe passage distance threshold, if the obstacle distance value is greater than the safe passage distance threshold, mark it as a safe passage state, if it is less than or equal to the safe passage distance threshold, mark it as a risk state, and obtain the forward passage marking state; Step S42: Evaluate the width of the left and right boundary spacing in the forward passage marking state. Compare the left and right boundary spacing with the robot's own width. If the left and right boundary spacing is greater than the robot's own width by a certain proportion, it is marked as a sufficient width state. If it is less than or equal to the proportion, it is marked as a tight width state, thus obtaining the forward width marking state. Step S43: Based on the forward passage mark status and the forward width mark status, a comprehensive score is calculated. A higher score is assigned to the safe passage status, a lower score is assigned to the risky status, a higher score is assigned to the wide enough status, and a lower score is assigned to the narrow width status. The scores corresponding to each status are added together to obtain the forward passability score. Step S44: Repeat the processing steps S41 to S43 for the forward passable area for the backward passable area to obtain the backward passability score.

5. The robot escape method based on an image sensor according to claim 4, characterized in that, The forward mobility score and the backward mobility score are numerically compared to obtain the target escape direction, including: Compare the numerical values ​​of the forward mobility score and the backward mobility score; If the forward mobility score is greater than the backward mobility score, then the direction corresponding to the forward mobility score is determined as the target escape direction; If the forward mobility score is less than the backward mobility score, then the direction corresponding to the backward mobility score is determined as the target escape direction; If the forward passability score and the backward passability score are equal, then the obstacle areas in the forward passability area and the backward passability area are divided into grids to obtain an obstacle distribution grid map, and the obstacle proportion of each grid in the obstacle distribution grid map is calculated to obtain the grid occupancy rate. Based on the grid occupancy rate, the passable areas in front and behind are statistically accumulated to obtain the obstacle distribution density. The obstacle distribution density is then compared numerically, and the direction with the lower obstacle distribution density is selected as the target escape direction.

6. The robot escape method based on an image sensor according to claim 1, characterized in that, Generate escape movement commands based on the target escape direction, including: The centerline of the passable area corresponding to the escape direction of the target is extracted to obtain the center coordinate sequence of the channel, and the distance between adjacent coordinate points of the center coordinate sequence of the channel is calculated to obtain the segmented distance parameter. The robot's movement speed is set segment by segment based on the segmented distance parameter, and the movement speed is bound to the target escape direction by a movement direction identifier to obtain the escape movement command.

7. The robot escape method based on an image sensor according to claim 1, characterized in that, The robot's motion controller executes the escape movement commands to enable the robot to overcome its environmental difficulties, including: The direction identification in the escape movement command is processed to obtain a movement steering control signal, and the speed control sequence in the escape movement command is converted by pulse width modulation to obtain a movement speed control signal; The motion steering control signal and the motion speed control signal are applied to the robot's motion controller through the drive circuit in the robot, so as to enable the robot to get out of the predicament of the environment.

8. A robot escape system based on an image sensor, characterized in that, include: The acquisition module is used to acquire environmental images of the front and rear areas of the robot's environment through image sensors installed on the robot body. The first calculation module is used to perform semantic segmentation processing and spatial distance calculation on the front area environmental image and the rear area environmental image respectively to obtain the front passable area and the rear passable area. The second calculation module is used to perform passability quantification calculations based on the passable area in front and the passable area behind, respectively, to obtain the passability score for moving forward and the passability score for moving backward. The comparison module is used to compare the forward drivability score with the backward drivability score to obtain the target escape direction, generate an escape motion command based on the target escape direction, and execute the escape motion command through the robot motion controller to enable the robot to get out of the predicament in the environment.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, The processor executes the steps of any one of claims 1 to 7 when executing a computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, it implements the steps of the method of any one of claims 1 to 7.

Images