Robot slip detection method, mobile robot and storage medium

Abstract

The invention provides a robot slip detection method, a mobile robot and a computer readable storage medium. The robot slip detection method comprises the following steps: acquiring a current image shot by a camera of the robot; performing infrared device identification on an easy-to-slide area of the current image to obtain an infrared device identification result of the current image; if the infrared device identification result is that the infrared device exists, obtaining the accumulated slip mileage of the current image; and detecting whether the robot slips or not according to the accumulated slipping mileage. According to the invention, the complexity of robot slip detection can be reduced, the resource consumption is reduced, and the real-time performance of robot slip detection is improved.

Description

Technical Field

[0001] The present application relates to the field of robot technology, and in particular to a robot slip detection method, a mobile robot and a computer-readable storage medium. Background Art

[0002] As robot technology matures and consumers’ demand for automated services increases, robots are being used in more and more scenarios, such as hotels, retail, warehousing and logistics. Under normal circumstances, mobile robots can identify obstacles in the environment in real time and avoid obstacles through intelligent path planning to complete tasks safely. However, during the driving process, mobile robots will inevitably encounter slipping problems due to slippery ground, wide gaps, and high slopes. However, in the related technologies, the robot slip detection solution has high design complexity and high resource consumption. Summary of the invention

[0003] The present application provides a robot slip detection method, a mobile robot and a computer-readable storage medium, which can reduce the complexity of robot slip detection, reduce resource consumption, and improve the real-time performance of robot slip detection.

[0004] In a first aspect, the present application provides a robot slip detection method, the method comprising:

[0005] Acquire a current image captured by a camera of the robot;

[0006] Performing infrared device recognition of the easy-slip area on the current image to obtain an infrared device recognition result of the current image;

[0007] If the infrared device recognition result is that the infrared device exists, obtaining the accumulated skidding mileage of the current image;

[0008] Whether the robot has slipped is detected according to the accumulated slip mileage.

[0009] In a second aspect, the present application further provides a mobile robot, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes any one of the robot slip detection methods provided in the present application.

[0010] In a third aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the robot slippage detection method.

[0011] In the present application, the infrared device of the slippery area is identified by the current image obtained by the camera of the robot. When the infrared device exists in the current image, the cumulative slip mileage of the current image can be directly obtained to determine whether the robot is slipping, without comparing the actual moving mileage of the robot with the odometer data to determine whether the robot is slipping. Therefore, there is no need to detect the actual moving mileage of the robot, so the complex detection process of the actual moving mileage can be reduced (for example, there is no need to compare the position change of the robot in the environmental observation information obtained at different times to obtain the actual moving mileage of the robot), thereby reducing the complexity of the robot's slip detection, reducing resource consumption, and improving the real-time performance of the robot's slip detection. As a result, the robot can be prevented from slipping for a long time, thereby reducing the loss of positioning due to the robot's slipping, and improving the execution efficiency of the robot's autonomous navigation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 is a schematic structural block diagram of a mobile robot provided in an embodiment of the present application;

[0014] Figure 2 It is a flow chart of a robot slippage detection method provided in an embodiment of the present application;

[0015] Figure 3 is a schematic diagram for explaining the relative installation position between the camera and the driving wheel in the embodiment of the present application;

[0016] Figure 4 is a schematic diagram illustrating adjacent pixels in an embodiment of the present application;

[0017] Figure 5 It is a schematic diagram of an embodiment of the robot slip detection process provided in the embodiments of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0019] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0020] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0021] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0022] The embodiments of the present application provide a robot slip detection method, a mobile robot, and a computer-readable storage medium. The mobile robot may be a cleaning robot, a food delivery robot, etc.

[0023] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0024] Figure 1 It is a structural schematic block diagram of a mobile robot provided in an embodiment of the present application.

[0025] like Figure 1 As shown, the mobile robot 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected via a bus 103, such as a PCIe (Peripheral Component Interconnect Express) bus.

[0026] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire mobile robot 100. The processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0027] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0028] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the embodiment scheme of the present application, and does not constitute a limitation on the mobile robot to which the embodiment scheme of the present application is applied. The specific mobile robot may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0029] The processor 101 is used to run the computer program stored in the memory 102, and implement any one of the robot slip detection methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is used to run the computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0030] Acquire a current image captured by a camera of the robot; perform infrared device recognition on the slippery area of ​​the current image to obtain an infrared device recognition result of the current image; if the infrared device recognition result indicates that an infrared device exists, acquire the cumulative slip mileage of the current image; and detect whether the robot is slipping based on the cumulative slip mileage.

[0031] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0032] According to the movement direction of the robot, the current image is segmented to obtain a first sub-image and a second sub-image; infrared area recognition is performed on the first sub-image to obtain the number of effective infrared areas of the first sub-image; infrared area recognition is performed on the second sub-image to obtain the number of effective infrared areas of the second sub-image; based on the number of effective infrared areas of the first sub-image and the number of effective infrared areas of the second sub-image, the infrared device recognition result of the current image is determined.

[0033] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0034] From the pixel points that have not been traversed in the first sub-image, the pixel points that meet the preset color conditions are obtained as the first infrared feature seed points; the number of first infrared pixel points in the first infrared area of ​​the first infrared feature seed points is obtained; if the number of the first infrared pixel points meets the preset number conditions, the first infrared area is used as the effective infrared area of ​​the first sub-image; until the traversal of the pixel points of the first sub-image is completed, the number of effective infrared areas of the first sub-image is obtained.

[0035] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0036] Establish a first infrared point queue of the first infrared feature seed point, and add the first infrared feature seed point to the first infrared point queue; if the first infrared point queue is not empty, obtain the first adjacent pixel point of the head pixel point of the first infrared point queue; if the first adjacent pixel point meets the preset color condition, add the first adjacent pixel point to the first infrared point queue; until the first infrared point queue is empty, obtain the cumulative number of pixels in the first infrared point queue as the first infrared pixel point number.

[0037] In some embodiments, the preset color condition is that the first color channel value of the pixel point is greater than a first preset color threshold, the second color channel value is less than a second preset color threshold, and the third color channel value is less than a third preset color threshold.

[0038] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0039] If the number of valid infrared areas of the first sub-image is greater than a preset number threshold, or the number of valid infrared areas of the second sub-image is greater than a preset number threshold, an infrared device will exist as the infrared device recognition result of the current image; or, if the number of valid infrared areas of the first sub-image and the number of valid infrared areas of the second sub-image are both less than or equal to the preset number threshold, there will be no infrared device as the infrared device recognition result of the current image.

[0040] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0041] From the pixel points that have not been traversed in the current image, the pixel points that meet the preset color conditions are obtained as the third infrared feature seed points; the number of third infrared pixel points in the infrared area corresponding to the third infrared feature seed points is obtained; and determination processing is performed based on the number of third infrared pixel points to obtain the infrared device recognition result of the current image.

[0042] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0043] If the number of the third infrared pixels meets a preset number condition, the existence of an infrared device is taken as the infrared device recognition result of the current image.

[0044] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0045] If the number of the third infrared pixel points meets the preset number condition, the infrared area corresponding to the third infrared feature seed point is used as the valid infrared area of ​​the current image; until the traversal of each pixel point of the current image is completed, the number of valid infrared areas of the current image is obtained; if the number of valid infrared areas of the current image is greater than the preset number threshold, the existence of an infrared device is used as the infrared device recognition result of the current image.

[0046] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0047] A third infrared point queue of the third infrared feature seed point is established, and the third infrared feature seed point is added to the third infrared point queue; if the third infrared point queue is not empty, the third adjacent pixel point of the head pixel point of the third infrared point queue is obtained; if the third adjacent pixel point meets the preset color condition, the third adjacent pixel point is added to the third infrared point queue; until the third infrared point queue is empty, the cumulative number of pixels in the third infrared point queue is obtained as the third infrared pixel point number.

[0048] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0049] If the infrared device recognition result is that an infrared device exists and the current image is not the first frame infrared image of the robot, the accumulated slip mileage of the current image is obtained according to the odometer data of the first frame infrared image and the odometer data of the current image.

[0050] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0051] If the accumulated slip mileage is greater than the preset change threshold, it is determined that the robot is slipping;

[0052] Alternatively, if the accumulated slip mileage is less than or equal to the preset change threshold, it is determined that the robot is not slipping.

[0053] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0054] If the current image is the first frame of the infrared image of the robot, it is determined that the robot does not slip.

[0055] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0056] If it is detected that the robot is slipping, a slip warning process is performed.

[0057] In some embodiments, the camera of the robot is disposed on the top of the robot, and the camera of the robot and the driving wheels of the robot are located on the same vertical plane.

[0058] It should be noted that, those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the mobile robot described above can refer to the corresponding process in the following robot slip detection method embodiment, and will not be repeated here.

[0059] In the following, Figure 1 The mobile robot shown in is used as an example to perform the robot slip detection method, and the robot slip detection method provided by the embodiment of the present application is described in detail. For the sake of simplicity and convenience of description, the execution subject will be omitted in the subsequent method embodiments. Figure 1 The scenario is only used to explain the robot slip detection method provided in the embodiment of the present application, but does not constitute a limitation on the application scenario of the robot slip detection method provided in the embodiment of the present application.

[0060] See also Figure 2 , Figure 2 201 to 204, wherein:

[0061] 201. Obtain a current image captured by a camera of the robot.

[0062] Among them, the robot can be a cleaning robot, a food delivery robot, etc.

[0063] like Figure 3 As shown, in order to improve the perception field of the robot, in this embodiment, a camera is installed on the top of the robot, and the camera of the robot and the driving wheel of the robot are located on the same vertical section. Therefore, on the one hand, by installing the camera of the robot on the top of the robot, it can be ensured that the robot has enough field of view to perceive the infrared device in the slippery area (such as the elevator infrared device installed inside on both sides of the elevator entrance). On the other hand, by setting the camera and the driving wheel of the robot on the same vertical section, rather than setting the camera directly above the center of the robot, it is possible to more accurately capture the slipping situation when the driving wheel is stuck in the joint of the slippery area (such as the joint of the elevator), ensuring that the movement of the infrared device on the image can more truly and accurately reflect the situation of the robot passing through the slippery area (such as the movement of the elevator infrared device on the image can more truly and accurately reflect the situation of the robot passing through the elevator).

[0064] For example, taking the elevator area as an example, when the robot enters the elevator area, the robot's camera is controlled to take pictures, and the latest image captured by the robot's camera in real time is used as the current image.

[0065] 202. Perform infrared device recognition of the easy-slip area of ​​the current image to obtain an infrared device recognition result of the current image.

[0066] Among them, the slippery area is equipped with an infrared device, and the slippery area can be set according to the actual business scenario requirements. For example, the elevator area can be set as a slippery area. By shooting the image of the elevator area, it is observed whether the robot stays at the joint of the elevator to detect whether the robot slips when entering the elevator.

[0067] The infrared device recognition result is used to indicate whether there is an infrared device in the current image.

[0068] There are many ways to determine the infrared device recognition result in step 202, illustratively including:

[0069] (1) A deep learning model can be used to perform infrared device recognition on an image. At this time, the preset deep learning model can be trained in advance using image samples to obtain an infrared device recognition model. For example, step 202 may specifically include: preprocessing the current image, which may include but is not limited to: adjusting the size, brightness and contrast of the image, image denoising, etc., to obtain a preprocessed current image; inputting the preprocessed current image into the infrared device recognition model, and extracting features of the preprocessed current image through the infrared device recognition model to obtain image features of the current image; and predicting (such as segmentation, classification, etc.) according to the image features of the current image through the infrared device recognition model to obtain an infrared device recognition result of the current image.

[0070] (2) Detect the effective infrared area of ​​the current image using the RGB color distribution of the pixel points of the current image. When it is detected that the current image has an effective infrared area, it is determined that there is an infrared device in the current image; when it is detected that the current image does not have an effective infrared area, it is determined that there is no infrared device in the current image, thereby obtaining the infrared device recognition result of the current image. Exemplarily, at this time, step 202 may specifically include the following steps 2021A to 2023A:

[0071] 2021A. From the pixel points that have not been traversed in the current image, obtain the pixel points that meet the preset color conditions as the third infrared feature seed points.

[0072] The third infrared characteristic seed point is a pixel point that meets a preset color condition among the pixel points that have not been traversed in the current image.

[0073] The preset color condition is that the first color channel value R of the pixel is greater than the first preset color threshold C0, the second color channel value G is less than the second preset color threshold C1, and the third color channel value B is less than the third preset color threshold C2.

[0074] The pixel points that potentially belong to the infrared device are searched by preset color conditions. Since the preset color conditions are mainly judged on the RGB color channel, there is no need for complex image feature extraction and processing, which can improve the search speed of the pixel points that potentially belong to the infrared device.

[0075] For example, assuming that the current image I includes M rows×N columns of pixels, the pixel points (x3, y3) that are not traversed by the current image I are traversed from left to right and from top to bottom. If the RGB color distribution of the pixel point (x3, y3) satisfies: the first color channel value R is greater than the first preset color threshold C0, the second color channel value G is less than the second preset color threshold C1, and the third color channel value B is less than the third preset color threshold C2, then the pixel point (x3, y3) that is not traversed by the current image I is used as the third infrared feature seed point. When the third infrared feature seed point is detected, the process of steps 2022A to 2023A is entered to detect whether the third infrared feature seed point and its vicinity are valid infrared areas, so as to detect the presence of infrared devices in the current image.

[0076] Among them, the pixel point (x3, y3) represents the pixel point corresponding to the y3th row and x3th column position in the current image I.

[0077] 2022A. Obtain the number of third infrared pixel points in the infrared area corresponding to the third infrared feature seed point.

[0078] The third infrared pixel point quantity is the number of infrared pixel points in the infrared area corresponding to the third infrared feature seed point, which is detected by taking the third infrared feature seed point as the seed point.

[0079] Specifically, in step 2022A, a third infrared point queue of the third infrared feature seed point can be established, and the third infrared feature seed point can be added to the third infrared point queue; if the third infrared point queue is not empty, the third adjacent pixel point of the head pixel point of the third infrared point queue is obtained; if the third adjacent pixel point meets the preset color condition, the third adjacent pixel point is added to the third infrared point queue; until the third infrared point queue is empty, the cumulative number of pixels in the third infrared point queue is obtained as the third infrared pixel number.

[0080] Exemplarily, step 2022A may include the following steps a1 to a8:

[0081] a1. Establish a third infrared point queue Q3 for the infrared area corresponding to the third infrared feature seed point, add the third infrared feature seed point to the third infrared point queue, mark the third infrared feature seed point as traversed, set the cumulative number of pixel points p3 of the third infrared point queue to 0, and proceed to step a2.

[0082] For example, when a pixel point that meets the preset color condition is traversed in step 2021A, the pixel point of the preset color condition is used as the third infrared feature seed point (x3, y3) and jumps to a1-a8 to detect the number of third infrared pixel points in the infrared area of ​​the third infrared feature seed point. First, in step a1, a third infrared point queue Q3 of the infrared area corresponding to the third infrared feature seed point (x3, y3) is established, and the third infrared feature seed point (x3, y3) is added to the third infrared point queue Q3, and the third infrared feature seed point (x3, y3) is marked as traversed, and then step a2 is entered.

[0083] a2. Detect whether the third infrared spot queue is empty. If the third infrared spot queue is not empty, proceed to step a3; if the third infrared spot queue is empty, proceed to step a8.

[0084] a3. If the third infrared spot queue is not empty, obtain the first pixel of the third infrared spot queue, and set the accumulated number of pixels of the third infrared spot queue p3=p3+1.

[0085] a4. Obtain the third adjacent pixel point of the first pixel point of the third infrared dot queue, and proceed to step a5.

[0086] The third adjacent pixel point is the pixel point to the upper left, upper, upper right, left, right, lower left, lower, and lower right of the first pixel point of the third infrared spot queue. Figure 4 As shown, assuming that the first pixel in the team is O(x, y), the third adjacent pixels include the upper left pixel A(x-1, y-1), the upper pixel B(x, y-1), the upper right pixel C(x+1, y-1), the left pixel D(x-1, y), the right pixel E(x+1, y), the lower left pixel F(x-1, y+1), the lower pixel G(x, y+1), and the lower right pixel H(x+1, y+1).

[0087] a5. Detect whether the third adjacent pixel point meets the preset color condition. If the third adjacent pixel point meets the preset color condition, proceed to step a6; if the third adjacent pixel point meets the preset color condition, proceed to step a7.

[0088] a6. If the third adjacent pixel point meets the preset color condition, the third adjacent pixel point is added to the third infrared point queue, and the third adjacent pixel point is marked as traversed, and the process goes to step a2.

[0089] For ease of understanding, the example of step a4 is continued. For example, if the RGB color distribution of the upper left pixel A (x-1, y-1) satisfies: the first color channel value R is greater than the first preset color threshold C0, the second color channel value G is less than the second preset color threshold C1, and the third color channel value B is less than the third preset color threshold C2, then the upper left pixel A (x-1, y-1) is added to the third infrared point queue; and so on, determine whether to add the upper pixel B (x, y-1), the upper right pixel C (x+1, y-1), the left pixel D (x-1, y), the right pixel E (x+1, y), the lower left pixel F (x-1, y+1), the lower pixel G (x, y+1), and the lower right pixel H (x+1, y+1) to the third infrared point queue.

[0090] a7. If the third adjacent pixel point meets the preset color condition, the third adjacent pixel point is marked as traversed and the process goes to step a2.

[0091] a8. When the third infrared point queue is empty, the accumulated number of pixels in the third infrared point queue is obtained as the third infrared pixel number.

[0092] Through steps a1 to a8, the third infrared feature seed point is used as the seed point, and the pixel points belonging to the infrared area around the third infrared feature seed point are searched. The number of pixel points in a certain infrared area can be determined without traversing the pixel points of the entire image, thereby reducing the pixel data processing volume, improving the detection speed of the third infrared pixel point number, and thereby improving the real-time performance of robot slip detection.

[0093] 2023A. Perform determination processing based on the number of the third infrared pixels to obtain an infrared device recognition result of the current image.

[0094] There are many ways to implement step 2023A, illustratively including:

[0095] ① When a valid infrared area is detected in the current image, the existence of an infrared device is directly used as the infrared device recognition result of the current image, and the traversal of the pixels that have not been traversed in the current image is stopped to improve the recognition efficiency of the infrared device recognition result, thereby improving the real-time recognition of the robot slipping. Specifically, the number of the third infrared pixel points is used to detect whether the infrared area corresponding to the third infrared feature seed point is a valid infrared area. When a valid infrared area is detected in the current image, the existence of an infrared device is directly used as the infrared device recognition result of the current image, and the traversal of the pixels that have not been traversed in the current image is stopped; otherwise, the pixels that have not been traversed in the current image are continued to search for the next third infrared feature seed point and detect the valid infrared area until the existence of an infrared device is determined when a valid infrared area is detected in the current image, or the existence of an infrared device is determined when the traversal of the pixels in the current image is completed and no valid infrared area is detected. Exemplarily, at this time, step 2023A may specifically include the following steps (1.1) to (1.8):

[0096] (1.1) Detect whether the number of the third infrared pixel points meets the preset number condition; if the number of the third infrared pixel points meets the preset number condition, enter step (1.2); if the number of the third infrared pixel points does not meet the preset number condition, enter step (1.3).

[0097] Exemplarily, the preset quantity condition may be: the third infrared pixel quantity p3>the first set value l0, and the third infrared pixel quantity p3<the second set value l1.

[0098] (1.2) If the number of the third infrared pixel points meets the preset number condition, the infrared area corresponding to the third infrared feature seed point is considered to be the valid infrared area of ​​the current image, and the existence of the infrared device is used as the infrared device recognition result of the current image, and the traversal of the pixel points that have not been traversed in the current image is stopped, and the process goes to step (1.8).

[0099] (1.3) If the number of the third infrared pixel points does not meet the preset number condition, proceed to step (1.4).

[0100] (1.4) Detect whether all pixel points of the current image have been traversed. If all pixel points of the current image have been traversed, proceed to step (1.7); if all pixel points of the current image have not been traversed, proceed to step (1.5).

[0101] (1.5) Continue to traverse the pixel points that have not been traversed in the current image, and detect whether the next traversed pixel point meets the preset color condition; if the next traversed pixel point meets the preset color condition, use the next traversed pixel point as the new third infrared feature seed and enter step (1.6); if the next traversed pixel point does not meet the preset color condition, enter step (1.4).

[0102] (1.6) If the next traversed pixel point meets the preset color condition, enter steps a1 to a8 to obtain the number of third infrared pixel points in the infrared area corresponding to the new third infrared characteristic seed. After obtaining the number of third infrared pixel points in the infrared area corresponding to the new third infrared characteristic seed, enter step (1.1).

[0103] (1.7) If the traversal of all pixel points of the current image is completed and there is no valid infrared area in the current image, the absence of an infrared device is taken as the infrared device identification result of the current image, and the process goes to step (1.8).

[0104] (1.8) The determination process is terminated and the infrared device recognition result of the current image is obtained.

[0105] Through steps (1.1) to (1.8), when a valid infrared area of ​​the current image is detected (that is, when the number of the third infrared pixel points meets the preset number condition), it can be determined that there is an infrared device in the current image, and the traversal of the pixels that have not been traversed in the current image will be stopped. The infrared device recognition result can be determined without waiting for the traversal of all the pixels of the current image to be completed, thereby improving the detection efficiency of the infrared device recognition result.

[0106] ② After traversing all the pixels in the current image and detecting all the valid infrared areas in the current image, determine whether there is an infrared device in the current image based on the number of valid infrared areas in the current image (for example, if the number of valid infrared areas in the current image is greater than a preset number threshold, such as when the number of valid infrared areas is greater than 0, the existence of an infrared device is used as the infrared device identification result of the current image). Exemplarily, at this time, step 2023A may specifically include the following steps (2.1) to (2.11):

[0107] (2.1) Detect whether the number of the third infrared pixel points meets the preset number condition; if the number of the third infrared pixel points meets the preset number condition, proceed to step (2.2); if the number of the third infrared pixel points does not meet the preset number condition, proceed to step (2.3).

[0108] (2.2) If the number of the third infrared pixel points meets the preset number condition, the infrared area corresponding to the third infrared feature seed point is considered to be the valid infrared area of ​​the current image, and the process goes to step (2.4).

[0109] Exemplarily, the preset quantity condition may be: the third infrared pixel quantity p3>the first set value l0, and the third infrared pixel quantity p3<the second set value l1.

[0110] (2.3) If the number of the third infrared pixel points does not meet the preset number condition, proceed to step (2.4).

[0111] (2.4) Check whether all pixel points of the current image have been traversed. If all pixel points of the current image have been traversed, proceed to step (2.7); if all pixel points of the current image have not been traversed, proceed to step (2.5).

[0112] (2.5) Continue to traverse the pixel points that have not been traversed in the current image, and detect whether the next traversed pixel point meets the preset color condition; if the next traversed pixel point meets the preset color condition, use the next traversed pixel point as the new third infrared feature seed and enter step (2.6); if the next traversed pixel point does not meet the preset color condition, enter step (2.4).

[0113] (2.6) If the next traversed pixel point meets the preset color condition, enter steps a1 to a8 to obtain the number of third infrared pixel points in the infrared area corresponding to the new third infrared characteristic seed. After obtaining the number of third infrared pixel points in the infrared area corresponding to the new third infrared characteristic seed, enter step (2.1).

[0114] (2.7) When all pixel points of the current image are traversed, the number of valid infrared areas of the current image is obtained, and the process goes to step (2.8).

[0115] (2.8) Detect whether the number of valid infrared areas of the current image is greater than a preset number threshold; if it is greater than the preset number threshold, proceed to step (2.9); if it is less than or equal to the preset number threshold, proceed to step (2.10).

[0116] (2.9) The existence of the infrared device is taken as the infrared device recognition result of the current image, and the process goes to step (2.11).

[0117] (2.10) The absence of an infrared device is regarded as the infrared device recognition result of the current image, and the process goes to step (2.11).

[0118] (2.11) The determination process is terminated, and the infrared device recognition result of the current image is obtained.

[0119] Through steps 2021A to 2023A, pixel points that meet the preset color conditions are first obtained from the pixel points that have not been traversed in the current image as the third infrared feature seed points, so that the infrared area with the third infrared feature seed points as the seed points can be found, and then the infrared devices that potentially belong to the slippery area can be quickly screened out, and then based on the number of third infrared pixels in the infrared area corresponding to the third infrared feature seed points, it can be quickly determined whether the infrared area with the third infrared feature seed points as the seed points is a truly valid infrared device, thereby quickly determining whether there is an infrared device in the current image.

[0120] (3) In some cases (for example, when the robot's camera is a fisheye camera), in order to improve the detection accuracy of the infrared device recognition result, the current image can be switched into two sub-images and then the infrared device recognition of the slippery area is performed for each sub-image. Exemplarily, at this time, step 202 can specifically include the following steps 2021B to 2024B:

[0121] 2021B. According to the movement direction of the robot, the current image is segmented to obtain a first sub-image and a second sub-image.

[0122] Specifically, please refer to Figure 3 , the current image can be evenly divided into a left sub-image and a right sub-image according to the movement direction of the robot, and the left sub-image and the right sub-image are respectively de-distorted and converted into plane images to obtain a first sub-image and a second sub-image. Therefore, by evenly dividing the current image into a left sub-image and a right sub-image according to the movement direction of the robot, on the one hand, it is beneficial to de-distort the image to avoid the problem that the image is deformed and the infrared area in the image cannot be accurately identified; on the other hand, the elevator infrared device can be identified on both sides of the elevator door respectively to avoid the problem that the elevator infrared device is only installed on one side of the elevator door (for example, when the elevator infrared device is only installed on the left / right side of the elevator door, the elevator infrared device cannot be accurately detected for the right / left side detection of the elevator door) and the elevator infrared device cannot be accurately and comprehensively detected, thereby improving the recognition rate of the elevator infrared device.

[0123] 2022B. Perform infrared region recognition on the first sub-image to obtain the number of valid infrared regions of the first sub-image.

[0124] Exemplarily, step 2022B may specifically include the following steps b1 to b9:

[0125] b1. From the pixels that are not traversed in the first sub-image, obtain the pixel points that meet the preset color conditions as the first infrared feature seed points.

[0126] The first infrared characteristic seed point is a pixel point that meets a preset color condition among the pixel points that are not traversed by the first sub-image.

[0127] b2. Obtain the number of first infrared pixel points in the first infrared region of the first infrared feature seed point.

[0128] The first infrared pixel point quantity is the number of infrared pixel points in the infrared region corresponding to the first infrared feature seed point, which is detected by taking the first infrared feature seed point as the seed point.

[0129] Specifically, in step b2, a first infrared point queue Q1 of the first infrared feature seed point can be established, and the first infrared feature seed point can be added to the first infrared point queue; if the first infrared point queue is not empty, the first adjacent pixel point of the head pixel point of the first infrared point queue is obtained; if the first adjacent pixel point meets the preset color condition, the first adjacent pixel point is added to the first infrared point queue, and the first adjacent pixel point is marked as traversed; until the first infrared point queue is empty, the cumulative number of pixels in the first infrared point queue is obtained as the first infrared pixel point number.

[0130] Exemplarily, step b2 may specifically include the following steps (3.1) to (3.8):

[0131] (3.1) Establish a first infrared point queue Q1 of the first infrared feature seed point, add the first infrared feature seed point to the first infrared point queue, mark the first infrared feature seed point as traversed, set the cumulative number of pixels in the first infrared point queue p1=0, and proceed to step (3.2);

[0132] (3.2) Detect whether the first infrared spot queue is empty. If the first infrared spot queue is not empty, proceed to step (3.3); if the first infrared spot queue is empty, proceed to step (3.8).

[0133] (3.3) If the first infrared point queue is not empty, obtain the first pixel of the first infrared point queue, and set the accumulated number of pixels in the first infrared point queue p1=p1+1.

[0134] (3.4) Obtain the first adjacent pixel point of the first pixel point of the first infrared point queue and proceed to step (3.5).

[0135] (3.5) Detect whether the first adjacent pixel point meets the preset color condition. If the first adjacent pixel point meets the preset color condition, proceed to step (3.6); if the first adjacent pixel point meets the preset color condition, proceed to step (3.7).

[0136] (3.6) If the first neighboring pixel point meets the preset color condition, the first neighboring pixel point is added to the first infrared point queue, and the first neighboring pixel point is marked as traversed, and the process proceeds to step (3.2).

[0137] (3.7) If the first neighboring pixel point meets the preset color condition, the first neighboring pixel point is marked as traversed and the process proceeds to step (3.2).

[0138] (3.8) When the first infrared point queue is empty, the accumulated number of pixels in the first infrared point queue is obtained as the first infrared pixel number.

[0139] Through steps (3.1) to (3.8), the first infrared feature seed point is used as the seed point, and the pixel points belonging to the infrared area around the first infrared feature seed point are searched. The number of pixel points in a certain infrared area can be determined without traversing the pixel points of the entire image, thereby reducing the pixel data processing volume, improving the detection speed of the first infrared pixel point number, and thereby improving the real-time performance of robot slip detection.

[0140] b3. Detect whether the number of the first infrared pixels meets the preset number condition; if the number of the first infrared pixels meets the preset number condition, proceed to step b4; if the number of the first infrared pixels does not meet the preset number condition, proceed to step b5.

[0141] b4. If the number of the first infrared pixel points meets the preset number condition, the infrared region corresponding to the first infrared feature seed point is considered to be the valid infrared region of the first sub-image, and the process goes to step b6.

[0142] Exemplarily, the preset quantity condition may be: the first infrared pixel quantity p1>the first set value l0, and the first infrared pixel quantity p1<the second set value l1.

[0143] b5. If the number of the first infrared pixels does not meet the preset number condition, proceed to step b6.

[0144] b6. Check whether all the pixel points of the first sub-image have been traversed. If all the pixel points of the first sub-image have been traversed, go to step b9; if all the pixel points of the first sub-image have not been traversed, go to step b7.

[0145] b7. Continue to traverse the pixel points that have not been traversed in the first sub-image, and detect whether the next traversed pixel point meets the preset color condition; if the next traversed pixel point meets the preset color condition, use the next traversed pixel point as the new first infrared feature seed and enter step b8; if the next traversed pixel point does not meet the preset color condition, enter step b6.

[0146] b8. If the next traversed pixel point meets the preset color condition, enter steps (3.1) to (3.8) to obtain the number of first infrared pixel points in the infrared area corresponding to the new first infrared characteristic seed. After obtaining the number of first infrared pixel points in the infrared area corresponding to the new first infrared characteristic seed, enter step b3.

[0147] b9. When all pixel points of the first sub-image are traversed, the number of valid infrared areas of the first sub-image is obtained.

[0148] Through steps b1 to b9, pixel points that meet the preset color conditions are obtained from the pixel points that are not traversed in the second sub-image as second infrared feature seed points, so that the infrared area with the second infrared feature seed points as seed points can be found, and then the infrared devices that are potentially slippery areas can be quickly screened out. Based on the number of second infrared pixels in the infrared area corresponding to the second infrared feature seed points, it can be quickly determined whether the infrared area with the second infrared feature seed points as seed points is a truly valid infrared device, thereby quickly determining the number of valid infrared areas of the second sub-image.

[0149] 2023B. Perform infrared region recognition on the second sub-image to obtain the number of valid infrared regions of the second sub-image.

[0150] Exemplarily, step 2023B may specifically include the following steps c1 to c9:

[0151] c1. From the pixels that are not traversed in the second sub-image, obtain the pixels that meet the preset color condition as the second infrared feature seed points.

[0152] The second infrared characteristic seed point is a pixel point that meets a preset color condition among the pixel points that are not traversed by the second sub-image.

[0153] c2. Obtain the number of second infrared pixel points in the second infrared region of the second infrared feature seed point.

[0154] The second infrared pixel point quantity is the number of infrared pixel points in the infrared region corresponding to the second infrared feature seed point, which is detected by taking the second infrared feature seed point as the seed point.

[0155] Specifically, in step c2, a second infrared point queue Q2 of the second infrared feature seed point can be established, and the second infrared feature seed point can be added to the second infrared point queue; if the second infrared point queue is not empty, the second adjacent pixel point of the head pixel point of the second infrared point queue is obtained; if the second adjacent pixel point meets the preset color condition, the second adjacent pixel point is added to the second infrared point queue, and the second adjacent pixel point is marked as traversed; until the second infrared point queue is empty, the cumulative number of pixels in the second infrared point queue is obtained as the second infrared pixel point number.

[0156] Exemplarily, step c2 may specifically include the following steps (4.1) to (4.8):

[0157] (4.1) Establish a second infrared point queue Q2 of the second infrared feature seed point, add the second infrared feature seed point to the second infrared point queue, mark the second infrared feature seed point as traversed, set the cumulative number of pixel points p2 of the second infrared point queue to 0, and proceed to step (4.2);

[0158] (4.2) Check whether the second infrared spot queue is empty. If the second infrared spot queue is not empty, proceed to step (4.3); if the second infrared spot queue is empty, proceed to step (4.8).

[0159] (4.3) If the second infrared spot queue is not empty, obtain the first pixel of the second infrared spot queue, and set the accumulated number of pixels in the second infrared spot queue p2=p2+1.

[0160] (4.4) Obtain the second adjacent pixel point of the first pixel point of the second infrared point queue and proceed to step (4.5).

[0161] (4.5) Detect whether the second adjacent pixel point meets the preset color condition. If the second adjacent pixel point meets the preset color condition, proceed to step (4.6); if the second adjacent pixel point meets the preset color condition, proceed to step (4.7).

[0162] (4.6) If the second neighboring pixel point meets the preset color condition, the second neighboring pixel point is added to the second infrared point queue, and the second neighboring pixel point is marked as traversed, and the process proceeds to step (4.2).

[0163] (4.7) If the second neighboring pixel point meets the preset color condition, the second neighboring pixel point is marked as traversed and the process proceeds to step (4.2).

[0164] (4.8) until the second infrared point queue is empty, the accumulated number of pixels in the second infrared point queue is obtained as the second infrared pixel number.

[0165] Through steps (4.1) to (4.8), the second infrared feature seed point is used as the seed point, and the pixel points belonging to the infrared area around the second infrared feature seed point are searched. The number of pixel points in a certain infrared area can be determined without traversing the pixel points of the entire image, thereby reducing the amount of pixel data processing, improving the detection speed of the second infrared pixel point number, and thereby improving the real-time performance of robot slip detection.

[0166] c3. Detect whether the number of the second infrared pixels meets the preset number condition; if the number of the second infrared pixels meets the preset number condition, proceed to step c4; if the number of the second infrared pixels does not meet the preset number condition, proceed to step c5.

[0167] c4. If the number of the second infrared pixel points meets the preset number condition, the infrared area corresponding to the second infrared feature seed point is considered to be the valid infrared area of ​​the second sub-image, and the process goes to step c6.

[0168] Exemplarily, the preset quantity condition may be: the second infrared pixel quantity p2> the second set value l0, and the second infrared pixel quantity p2< the second set value l1.

[0169] c5. If the number of the second infrared pixel points does not meet the preset number condition, proceed to step c6.

[0170] c6. Check whether all the pixel points of the second sub-image have been traversed. If all the pixel points of the second sub-image have been traversed, go to step c9; if all the pixel points of the second sub-image have not been traversed, go to step c7.

[0171] c7. Continue to traverse the pixels that have not been traversed in the second sub-image, and detect whether the next traversed pixel meets the preset color condition; if the next traversed pixel meets the preset color condition, use the next traversed pixel as the new second infrared feature seed and enter step c8; if the next traversed pixel does not meet the preset color condition, enter step c6.

[0172] c8. If the next traversed pixel point meets the preset color condition, enter steps (4.1) to (4.8) to obtain the number of second infrared pixels in the infrared area corresponding to the new second infrared characteristic seed. After obtaining the number of second infrared pixels in the infrared area corresponding to the new second infrared characteristic seed, enter step c3.

[0173] c9. When the traversal of each pixel point of the second sub-image is completed, the number of valid infrared areas of the second sub-image is obtained.

[0174] Through steps c1 to c9, pixel points that meet the preset color conditions are obtained from the pixel points that are not traversed in the first sub-image as the first infrared feature seed points, so that the infrared area with the first infrared feature seed point as the seed point can be found, and then the infrared devices that are potentially easy to slip can be quickly screened out. Based on the number of first infrared pixel points in the infrared area corresponding to the first infrared feature seed point, it can be quickly determined whether the infrared area with the first infrared feature seed point as the seed point is a truly valid infrared device, thereby quickly determining the number of valid infrared areas of the first sub-image.

[0175] 2024B. Determine the infrared device recognition result of the current image based on the number of valid infrared areas of the first sub-image and the number of valid infrared areas of the second sub-image.

[0176] Specifically, if the number of effective infrared areas of the first sub-image is greater than a preset number threshold, or the number of effective infrared areas of the second sub-image is greater than a preset number threshold, the existence of an infrared device is used as the infrared device recognition result of the current image; or, if the number of effective infrared areas of the first sub-image and the number of effective infrared areas of the second sub-image are both less than or equal to the preset number threshold, the absence of an infrared device is used as the infrared device recognition result of the current image. Thus, by determining that the current image has an infrared device when the number of effective infrared areas of the first sub-image is greater than the preset number threshold, or the number of effective infrared areas of the second sub-image is greater than the preset number threshold, the determination can be made in combination with the infrared area detection conditions of the two images, thereby avoiding the problem that the infrared device is located on the side of the slippery area and cannot be accurately detected.

[0177] Through steps 2021B to 2024B, the current image is first segmented according to the movement direction of the robot to obtain a first sub-image and a second sub-image, and then infrared area recognition is performed on the first sub-image and the second sub-image respectively. Based on the number of effective infrared areas of the first sub-image and the number of effective infrared areas of the second sub-image, it is determined whether there is an infrared area in the current image. On the one hand, the problem that the infrared device cannot be accurately identified due to the distortion of the current image can be avoided. On the other hand, the infrared device recognition can be performed on both sides of the slippery area respectively to avoid the problem that the infrared device is only installed on one side and the infrared device cannot be accurately and comprehensively detected (for example, when the elevator infrared device is installed only on the left / right side of the elevator door, the elevator infrared device cannot be accurately detected for the right / left side detection of the elevator door), thereby improving the recognition rate of the infrared device.

[0178] 203. If the infrared device recognition result is that an infrared device exists, the accumulated skidding mileage of the current image is obtained.

[0179] The accumulated skidding mileage is the difference between the odometer data of the first frame infrared image and the odometer data of the current image.

[0180] The first infrared image is the first image in which the presence of an infrared device is detected among the images collected after the robot enters the elevator area.

[0181] Exemplarily, when the infrared device recognition result is that the infrared device exists, it is detected whether the current image is the first frame image in which the infrared device is detected; if the current image is the first frame image in which the infrared device is detected, the current image is used as the first frame infrared image of the robot, and it can be determined that the robot has not slipped. If the current image is not the first frame image in which the infrared device is detected, it is considered that the current image is not the first frame infrared image of the robot, and the accumulated slip mileage of the current image is obtained based on the odometer data of the first frame infrared image and the odometer data of the current image. For example, the odometer data of the current image can be subtracted from the odometer data of the first frame infrared image, and the result delt_t can be used as the accumulated slip mileage of the current image.

[0182] 204. Detect whether the robot slips according to the accumulated slip mileage.

[0183] Specifically, if the accumulated slip mileage is greater than the preset change threshold (e.g., accumulated slip mileage delt_t>preset change threshold delt_t0), it is determined that the robot is slipping; or, if the accumulated slip mileage is less than or equal to the preset change threshold (e.g., accumulated slip mileage delt_t≤preset change threshold delt_t0), it is determined that the robot is not slipping. Therefore, this embodiment can identify the infrared device in the slippery area through digital image processing technology, track the movement of the infrared device on the image and associate it with the real movement of the robot, and determine whether the robot has a slip problem such as an elevator by evaluating whether the moving distance of the odometer data during the robot slip detection (i.e., the accumulated slip mileage) is greater than a preset distance threshold (i.e., the preset change threshold), without the need for a complex detection process of the actual moving mileage, thereby improving the real-time performance of the robot slip detection.

[0184] Furthermore, when the robot is detected to be slipping, a slip warning can be performed to prevent the robot from slipping for a long time, thereby reducing the loss of positioning due to robot slipping and improving the execution efficiency of the robot's autonomous navigation tasks; reducing the problem of important public resources such as elevators and narrow passages being occupied for a long time, and avoiding affecting the travel efficiency of relevant users in hotel or building scenarios.

[0185] To better understand this embodiment, please refer to Figure 5 ,The following takes the elevator area as an example to explain the robot's slip detection process as follows:

[0186] 1. The robot enters the elevator area and starts the elevator slip detection function.

[0187] 2. Get the latest image captured by the current image robot's camera in real time as the current image, and get the odometer data Odom of the current image i . (refer to Figure 5 "Synchronously obtain a frame of image data and odometer data"

[0188] 3. Perform elevator infrared device recognition on the current image to obtain the elevator infrared device recognition result; if the elevator infrared device recognition result is that the infrared device exists, proceed to step 5; otherwise, if the elevator infrared device recognition result is that the infrared device does not exist, proceed to step 4. (Refer to Figure 5 "Identify the elevator infrared device in the current image"

[0189] 4. Set the first frame of infrared image to null value and go to step 2. (Refer to Figure 5 "Evaluation reference frame Odom0 is set to NULL")

[0190] 5. Check whether the current image is the first frame image in which the infrared device is detected; if so, proceed to step 6; if not, proceed to step 7. (Refer to Figure 5 "Exists" in

[0191] 6. Use the current image as the first infrared image frame, and use the odometer data of the current image as the odometer data Odom0 of the first infrared image frame. (Refer to Figure 5 "Record the current odometer data as a reference point for evaluating the displacement of the skid detection, denoted as Odom0")

[0192] 7. Calculate the difference between the odometer data of the current image and the odometer data of the first frame of infrared image as the accumulated slip mileage delt_t of the current image (reference Figure 5 "Calculate the displacement change between the current time and the time when the infrared device is first detected, denoted as delt_t, delt_t = |Odom i -Odom0|t”)

[0193] 8. Check whether the accumulated slip mileage delt_t of the current image is greater than the preset change threshold delt_0; if it is greater, go to step 9; if it is less than or equal to it, go back to step 2. (Refer to Figure 5 (“delt_t>delt_0?”)

[0194] 9. When the accumulated slip mileage delt_t is greater than the preset change threshold delt_0, the robot is judged to be slipping and a slip warning is performed. (Refer to Figure 5 "Elevator slippage detected, triggering system emergency response"

[0195] From the above content, it can be seen that by performing infrared device recognition of slippery areas on the current image obtained by the robot's camera, when there is an infrared device in the current image, the cumulative slip mileage of the current image can be directly obtained to determine whether the robot has slipped, without comparing the robot's actual moving mileage with the odometer data to determine whether the robot has slipped. Therefore, there is no need to detect the robot's actual moving mileage, thereby reducing the complex detection process of the actual moving mileage (for example, there is no need to compare the robot's position change in the environmental observation information obtained at different times to obtain the robot's actual moving mileage), thereby reducing the complexity of the robot's slip detection, reducing resource consumption, and improving the real-time performance of the robot's slip detection. As a result, the robot can be prevented from slipping for a long time, thereby reducing the loss of positioning due to the robot's slipping and improving the execution efficiency of the robot's autonomous navigation tasks.

[0196] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned robot slip detection method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0197] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored, and the computer program can be loaded by a processor to execute any robot slip detection method provided in the embodiment of the present application. For example, the computer program can be loaded by a processor to execute the following steps:

[0198] Acquire a current image captured by a camera of the robot; perform infrared device recognition on the slippery area of ​​the current image to obtain an infrared device recognition result of the current image; if the infrared device recognition result indicates that an infrared device exists, acquire the cumulative slip mileage of the current image; and detect whether the robot is slipping based on the cumulative slip mileage.

[0199] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0200] In the above robot slip detection method, computer readable storage medium, and mobile robot embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, refer to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the computer readable storage medium, mobile robot and its corresponding units described above can refer to the description of the robot slip detection method in the above embodiment, and will not be repeated here.

[0201] The above is a detailed introduction to a robot slip detection method, a mobile robot and a computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A robot slip detection method, characterized in that: The method comprises: Acquire a current image captured by a camera of the robot; Performing infrared device recognition of the easy-slip area on the current image to obtain an infrared device recognition result of the current image; If the infrared device recognition result is that the infrared device exists, obtaining the accumulated skidding mileage of the current image; Whether the robot has slipped is detected according to the accumulated slip mileage.

2. The robot slip detection method according to claim 1, characterized in that: The performing infrared device recognition of the easy-slip area on the current image to obtain the infrared device recognition result of the current image includes: According to the movement direction of the robot, the current image is segmented to obtain a first sub-image and a second sub-image; Performing infrared region recognition on the first sub-image to obtain the number of valid infrared regions of the first sub-image; performing infrared region recognition on the second sub-image to obtain the number of valid infrared regions of the second sub-image; Based on the number of valid infrared areas of the first sub-image and the number of valid infrared areas of the second sub-image, an infrared device recognition result of the current image is determined.

3. The robot slip detection method according to claim 2, characterized in that: The performing infrared region recognition on the first sub-image to obtain the number of valid infrared regions of the first sub-image includes: From the pixels that are not traversed in the first sub-image, a pixel point that meets a preset color condition is obtained as a first infrared feature seed point; Acquire the number of first infrared pixel points in the first infrared region of the first infrared feature seed point; If the number of the first infrared pixels meets a preset number condition, taking the first infrared area as a valid infrared area of ​​the first sub-image; When the traversal of each pixel point of the first sub-image is completed, the number of valid infrared areas of the first sub-image is obtained.

4. The robot slip detection method according to claim 3, characterized in that: The step of obtaining the number of first infrared pixel points in the first infrared region of the first infrared feature seed point includes: Establishing a first infrared point queue of the first infrared feature seed point, and adding the first infrared feature seed point to the first infrared point queue; If the first infrared point queue is not empty, obtaining the first adjacent pixel point of the head pixel point of the first infrared point queue; If the first adjacent pixel point meets the preset color condition, adding the first adjacent pixel point to the first infrared point queue; When the first infrared point queue is empty, the accumulated number of pixels in the first infrared point queue is obtained as the first infrared pixel number.

5. The robot slip detection method according to claim 3, characterized in that: The preset color condition is that the first color channel value of the pixel point is greater than the first preset color threshold, the second color channel value is less than the second preset color threshold, and the third color channel value is less than the third preset color threshold.

6. The robot slip detection method according to claim 2, characterized in that: The determining the infrared device recognition result of the current image based on the number of valid infrared areas of the first sub-image and the number of valid infrared areas of the second sub-image includes: If the number of valid infrared regions of the first sub-image is greater than a preset number threshold, or the number of valid infrared regions of the second sub-image is greater than a preset number threshold, the existence of an infrared device is used as the infrared device recognition result of the current image; Alternatively, if the number of valid infrared areas of the first sub-image and the number of valid infrared areas of the second sub-image are both less than or equal to a preset number threshold, the absence of an infrared device is used as the infrared device recognition result of the current image.

7. The robot slip detection method according to claim 1, characterized in that: The performing infrared device recognition of the easy-slip area on the current image to obtain the infrared device recognition result of the current image includes: From the pixels that are not traversed in the current image, a pixel point that meets a preset color condition is obtained as a third infrared feature seed point; Acquire the number of third infrared pixel points in the infrared area corresponding to the third infrared feature seed point; A determination process is performed based on the third number of infrared pixels to obtain an infrared device recognition result of the current image.

8. The robot slip detection method according to claim 7, characterized in that: The determining based on the number of the third infrared pixels to obtain the infrared device recognition result of the current image includes: If the number of the third infrared pixels meets a preset number condition, the existence of an infrared device is taken as the infrared device recognition result of the current image.

9. The robot slip detection method according to claim 7, characterized in that: The determining based on the number of the third infrared pixels to obtain the infrared device recognition result of the current image includes: If the number of the third infrared pixel points meets a preset number condition, the infrared region corresponding to the third infrared feature seed point is used as the effective infrared region of the current image; When all pixel points of the current image are traversed, the number of valid infrared areas of the current image is obtained; If the number of valid infrared regions of the current image is greater than a preset number threshold, the existence of an infrared device is regarded as the infrared device recognition result of the current image.

10. The robot slip detection method according to claim 1, characterized in that: If the infrared device recognition result is that the infrared device exists, the accumulated skidding mileage of the current image is obtained, including: If the infrared device recognition result is that an infrared device exists and the current image is not the first frame infrared image of the robot, the accumulated slip mileage of the current image is obtained according to the odometer data of the first frame infrared image and the odometer data of the current image.

11. The robot slip detection method according to claim 1, characterized in that: The detecting whether the robot is slipping according to the accumulated slip mileage includes: If the accumulated slip mileage is greater than the preset change threshold, it is determined that the robot is slipping; Alternatively, if the accumulated slip mileage is less than or equal to the preset change threshold, it is determined that the robot is not slipping.

12. The robot slip detection method according to claim 1, characterized in that: The method further comprises: If the current image is the first frame of the infrared image of the robot, it is determined that the robot does not slip.

13. The robot slip detection method according to claim 1, characterized in that: The method further comprises: If it is detected that the robot is slipping, a slip warning process is performed.

14. The robot slip detection method according to claim 1, characterized in that: The camera of the robot is arranged on the top of the robot, and the camera of the robot and the driving wheel of the robot are located on the same vertical section plane.

15. A mobile robot, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the robot slippage detection method according to any one of claims 1 to 14 is executed.

16. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the robot slip detection method according to any one of claims 1 to 14.

Images