Slipping detection method and device, electronic equipment and storage medium

Abstract

The invention relates to a slip detection method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the feature point-based optical flow matching pursuit of a first image and a second image in collected image data, calculating the average optical flow value of the successfully matched feature points, calculating the linear speed of a robot based on the chassis wheel speed meter data, and carrying out the detection of the slip. When the average optical flow value is smaller than a preset threshold value and the linear speed is larger than or equal to a preset speed, it is determined that the robot is in a slipping state, the method has good robustness on low-cost hardware, and the robot has the efficient slipping detection capacity in an easy-to-slip scene.

Description

Technical Field

[0001] The present application relates to the field of robotics, and in particular to a slip detection method, device, electronic device, and storage medium. Background Art

[0002] With the continuous advancement of automation technology, robots are being used in an increasingly wide range of fields. For example, outdoor lawn mower robots are becoming increasingly popular. They can automatically move across lawns and mow them without human intervention. However, in complex outdoor environments, lawn mower robots may encounter various challenges, one of which is slippage.

[0003] Slipping refers to the situation where the friction between the wheels or tracks and the ground is insufficient due to slippery, uneven ground or other reasons during the robot's movement, making it impossible for the robot to move or turn normally. In this case, the robot may fall into a circular motion and be unable to effectively complete the mowing task, and may even damage the lawn or itself. Therefore, how to effectively detect robot slipping has become an urgent problem to be solved. Summary of the Invention

[0004] The present application provides a slip detection method, device, electronic device and storage medium to solve the technical problem of how to effectively detect slip on a robot.

[0005] In a first aspect, the present application provides a method for detecting slippage, the method comprising:

[0006] Obtain image data collected by the robot and chassis wheel speed meter data;

[0007] Performing optical flow matching tracking based on feature points on a first image and a second image in the image data, and calculating an average optical flow value of the successfully matched feature points; wherein the first image is a first frame image at a target moment, and the second image is another frame image after the target moment;

[0008] Calculating the linear speed of the robot according to the chassis wheel speed meter data;

[0009] When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, it is determined that the robot is in a slipping state.

[0010] Optionally, performing optical flow matching tracking based on feature points on the first image and the second image in the image data, and calculating an average optical flow value of the successfully matched feature points, includes:

[0011] Obtaining the acquisition frequency of the robot's camera;

[0012] determining an interval between the first image and the second image according to the acquisition frequency; wherein the acquisition frequency is positively correlated with the interval;

[0013] determining the second image from the image data according to the interval;

[0014] Extracting feature points from the first image to obtain first feature points;

[0015] Perform optical flow matching tracking of the first feature points in the second image, and calculate the average optical flow value of the successfully matched target feature points.

[0016] Optionally, before extracting feature points from the first image to obtain first feature points, the method further includes:

[0017] performing grayscale processing on the first image and the second image respectively to obtain grayscale images;

[0018] performing cropping processing on the grayscale image according to the installation height and tilt angle of the camera of the robot to obtain a cropped image;

[0019] The cropped image is subjected to resolution scaling to obtain the processed first image and the second image.

[0020] Optionally, performing optical flow matching tracking of the first feature point in the second image and calculating an average optical flow value of successfully matched target feature points includes:

[0021] performing optical flow matching tracking of the first feature points in the second image, and determining a successfully matched target feature point from the first feature points;

[0022] determining a moving distance of each of the target feature points according to the first image and the second image;

[0023] The average optical flow value is calculated according to the movement distances corresponding to all the target feature points.

[0024] Optionally, when the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, determining that the robot is in a slipping state includes:

[0025] Get the count value of the counter;

[0026] When the average optical flow value is less than a preset threshold, the count value is increased by one as the current count value;

[0027] If the current count value does not reach the preset value, re-execute the step of acquiring the image data and chassis wheel speedometer data collected by the robot until the step of adding one to the count value as the current count value if the average optical flow value is less than the preset threshold;

[0028] When the current count value reaches a preset value and the linear speed is greater than or equal to a preset speed, it is determined that the robot is in a slipping state.

[0029] Optionally, the method further includes:

[0030] When the average optical flow value is greater than or equal to the preset threshold, the count value is set to zero.

[0031] Optionally, before calculating the linear velocity of the robot according to the chassis wheel speedometer data, the method further includes:

[0032] Synchronize the acquisition time of the current image with the acquisition time of the current chassis wheel speed meter data.

[0033] In a second aspect, the present application provides a slip detection device, comprising:

[0034] An acquisition module is used to obtain image data collected by the robot and chassis wheel speed meter data;

[0035] a matching tracking module, configured to perform optical flow matching tracking based on feature points on a first image and a second image in the image data, and calculate an average optical flow value of the successfully matched feature points; wherein the first image is a first frame image at a target moment, and the second image is another frame image after the target moment;

[0036] A linear speed calculation module, configured to calculate the linear speed of the robot based on the chassis wheel speedometer data;

[0037] The determination module is configured to determine that the robot is in a slipping state when the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed.

[0038] In a third aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0039] Memory for storing computer programs;

[0040] The processor is configured to implement the slip detection method described in any one of the embodiments of the first aspect when executing a program stored in the memory.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the slip detection method as described in any one of the embodiments of the first aspect.

[0042] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application obtains image data and chassis wheel speedometer data collected by the robot; performs optical flow matching tracking based on feature points on the first image and the second image in the image data, and calculates the average optical flow value of the successfully matched feature points; wherein the first image is the first frame image at the target time, and the second image is another frame image after the target time; calculates the linear velocity of the robot based on the chassis wheel speedometer data; and determines that the robot is in a slipping state when the average optical flow value is less than a preset threshold and the linear velocity is greater than or equal to a preset speed. The method can perform optical flow matching tracking based on feature points on the first image and the second image in the collected image data, calculate the average optical flow value of the successfully matched feature points, and determine that the robot is in a slipping state when the average optical flow value is less than a preset threshold and the linear velocity is greater than or equal to a preset velocity based on the linear velocity of the robot calculated from the chassis wheel speedometer data. The method can have good robustness on low-cost hardware, so that the robot has the ability to efficiently detect slipping in slip-prone scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0046] Figure 1 A system architecture diagram of a skid detection method provided in one embodiment of the present application;

[0047] Figure 2 A schematic flow chart of a skid detection method provided in one embodiment of the present application;

[0048] Figure 3A A schematic diagram of an original image provided in one embodiment of the present application;

[0049] Figure 3B A schematic diagram of a cropped grayscale image provided in one embodiment of the present application;

[0050] Figure 3C A schematic diagram of a grayscale image after resolution scaling provided by an embodiment of the present application;

[0051] Figure 3D A schematic diagram of a tracking image with N frames of feature point extraction provided by an embodiment of the present application;

[0052] Figure 3E A schematic diagram of an optical flow tracking and matching image with a distance of N frames provided in one embodiment of the present application;

[0053] Figure 4 A schematic flow chart of a skid detection method provided in another embodiment of the present application;

[0054] Figure 5 A schematic structural diagram of a skid detection device provided in one embodiment of the present application;

[0055] Figure 6 A schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, these are merely examples and are not intended to limit the present application. In addition, the present application may repeat reference numbers and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0058] Some related solutions for slip detection generally rely on visual state estimation. For example, the robot's state, which mainly includes position and posture information, is first calculated using monocular or binocular image information. Secondly, the robot's displacement increments are calculated by aligning the chassis' wheel speedometer's speed information or position increments with the visual information and calculating the robot's displacement increments before and after visual positioning. If the cumulative displacement increments over multiple consecutive visual frames are less than a certain threshold of the wheel speedometer's cumulative increments, the robot is considered to be slipping. However, these solutions have the following drawbacks:

[0059] Defect 1: High computing power consumption

[0060] For low-computing-power platforms, it is difficult for monocular / binocular visual positioning to effectively achieve real-time output, and its positioning accuracy and stability are difficult to guarantee, thus affecting the stability and accuracy of slip detection based on visual position information.

[0061] Defect 2: Occlusion problem

[0062] The visual sensor has a limited field of view. If the robot cannot see enough feature points at certain positions or angles, it may not be able to accurately locate itself. Occlusion by vegetation, obstacles, or other objects may affect the visual sensor's field of view, making it impossible to obtain the necessary feature points.

[0063] Defect 3: Strong environmental dependence

[0064] Lighting changes: Changes in lighting conditions (such as shadows, strong light, and nighttime) can affect the performance of visual sensors, resulting in a decrease in the accuracy of feature point detection and matching.

[0065] Environmental dynamics: If there are dynamic objects in the environment (such as pedestrians, animals, etc.), these dynamic objects may interfere with the accuracy of visual positioning.

[0066] Lack of texture: In some environments, such as green lawns or single-color ground, the lack of obvious feature points makes visual positioning difficult.

[0067] Defect 4: Robustness Issues

[0068] Dust, raindrops, fog, etc. in the environment may increase image noise, affecting the detection and matching of feature points, and dynamic changes in the environment (such as lawn swaying caused by wind) may make visual positioning unstable.

[0069] Flaw 5: Hardware Limitations

[0070] The performance of visual sensors (such as resolution and frame rate) and the processing power of the computing platform limit the practical application of the system. The stability of visual positioning has strong requirements on the frame rate and image resolution during the sensing period.

[0071] Due to the existence of the above-mentioned defects, it is impossible to achieve effective slip detection of the robot. In order to solve the above-mentioned technical problems in the prior art, the present application provides a slip detection method, device, electronic device and storage medium, which can accurately and efficiently identify the slip state of the robot through the average optical flow value obtained by optical flow matching tracking based on feature points, and the linear speed calculated based on the chassis wheel speed meter data. It has good robustness on low-cost hardware, so that the robot has the ability to efficiently detect slippage in scenes prone to slippage.

[0072] The first embodiment of the present application provides a method for detecting slippage, which can be applied to Figure 1 The system architecture shown in FIG. 1 includes at least a data acquisition module 101 and a data processing module 102, which establish a communication connection with the data acquisition module 101 and the data processing module 102. Specifically, the system architecture may be a robot, such as a lawn mowing robot.

[0073] Next, based on the system architecture, the slip detection method is described in detail. Figure 2 The skid detection method comprises:

[0074] Step 201: Acquire image data and chassis wheel speed meter data collected by the robot.

[0075] The robot can be a lawn mowing robot equipped with a monocular camera. The monocular camera collects image data at a pre-set collection frequency and inputs it into the robot's data processing module. The collection frequency is the image input frame rate. Similarly, the chassis wheel speed meter data is also input into the robot's data processing module in real time.

[0076] Step 202 , performing optical flow matching tracking based on feature points on the first image and the second image in the image data, and calculating the average optical flow value of the successfully matched feature points; wherein the first image is the first frame image at the target moment, and the second image is another frame image after the target moment.

[0077] The target moment can be the start moment of the current optical flow matching tracking. The first image is the first frame image at the start of the current optical flow matching tracking. The second image is the Nth frame image after the start of the current optical flow matching tracking. N can be preset as needed. For example, N can be 3, 4, 5, 6, 7, 8, etc. It can also be automatically adjusted according to the acquisition frequency of the monocular camera without limitation.

[0078] In one embodiment, feature point-based optical flow matching and tracking is performed on a first image and a second image in image data, and an average optical flow value of successfully matched feature points is calculated, including: obtaining an acquisition frequency of a robot's camera; determining an interval between the first image and the second image based on the acquisition frequency; wherein the acquisition frequency is positively correlated with the interval; determining the second image from the image data based on the interval; extracting feature points from the first image to obtain first feature points; performing optical flow matching and tracking of the first feature points in the second image, and calculating an average optical flow value of successfully matched target feature points.

[0079] In this embodiment, the selection of an image after the first frame as the second image can be determined based on the acquisition frequency of the monocular camera. For example, if the acquisition frequency of the monocular camera is low, N can be set to 3 or 4, i.e., the first image is the first frame, and the second image is the third or fourth frame. If the acquisition frequency is high, N can be set to 6-8, without limitation. After determining the interval between the first and second images based on the acquisition frequency, the second image can be determined from the image data, thereby ensuring that there are partial differences between the first and second images, facilitating optical flow matching tracking. During optical flow matching tracking, feature points are first extracted from the first image to obtain first feature points. The first feature points may include multiple feature points in the first image. Optical flow matching tracking is then performed on the first feature points in the second image, and the average optical flow value of successfully matched target feature points is calculated. The first feature points can be unique and stable points identified from the image. These points can be areas in the image that can be reliably identified under different viewing angles, lighting conditions, or scales. The purpose of feature point extraction is to accurately describe key information in the image to facilitate subsequent tasks such as image matching, target tracking, and 3D reconstruction. Specifically, the feature point may be corner point information in an image.

[0080] In one embodiment, optical flow matching tracking of the first feature point is performed in the second image, and the average optical flow value of the successfully matched target feature points is calculated, including: performing optical flow matching tracking of the first feature point in the second image, determining the successfully matched target feature points from the first feature points; determining the movement distance of each target feature point based on the first image and the second image; and calculating the average optical flow value based on the movement distances corresponding to all target feature points.

[0081] In this embodiment, optical flow matching tracking of the first feature point is first performed in the second image. The successfully matched ones are called target feature points. Then, the moving distance of each target feature point is determined by combining the first image and the second image (that is, the average pixel distance moved by all feature points is counted), and the average optical flow value averageFlow is calculated based on the moving distances corresponding to all target feature points.

[0082] In one embodiment, before extracting feature points from the first image and obtaining the first feature points, the method further includes: grayscale processing the first image and the second image to obtain a grayscale image; cropping the grayscale image according to the installation height and tilt angle of the robot's camera to obtain a cropped image; and scaling the resolution of the cropped image to obtain the processed first image and second image.

[0083] In this embodiment, to increase data processing speed, the images to be used for optical flow matching tracking can be preprocessed before optical flow matching tracking. The preprocessing method for the first and second images is the same. Here, the first and second images before preprocessing are collectively referred to as images. First, the images are grayscaled to obtain grayscale images. Then, the grayscale images are cropped based on the installation height and tilt angle of the robot camera to obtain cropped images. Finally, the cropped images are resolution-scaled to obtain the processed first and second images.

[0084] For ease of understanding, combined Figure 3A-Figure 3E A detailed description is given, in which Figure 3A The original image is shown in Figure 1. After grayscale processing, the image is cropped according to the installation height and tilt angle of the monocular camera. The cropped grayscale image is shown in Figure 1. Figure 3B The purpose is to avoid the interference introduced by extracting distant feature points. Finally, the resolution of the cropped image is scaled. The schematic diagram of the grayscale image after resolution scaling is as follows: Figure 3C Next, feature point extraction is performed. Figure 3D Schematic diagram of tracking image extraction of feature points N frames before and after, where: Figure 3D The left side is the starting frame (first image), and the right side is the Nth frame (i.e., the second image, which can be adjusted according to the driving frequency and the actual speed of the robot). Finally, the optical flow tracking and matching is performed. The schematic diagram of the optical flow tracking and matching image of the N-frame distance before and after is as follows: Figure 3E , Figure 3E The left side of the image shows the starting frame, and the right side shows the Nth frame. Optical flow tracking is performed on the starting and Nth frames, and the average movement distance of all feature points is calculated to obtain the average optical flow value. Optical flow tracking is a technique used in computer vision to estimate pixel motion in image sequences. It assumes that pixels in an image change continuously over time within a small spatial neighborhood. The goal of optical flow tracking is to infer the direction and speed of motion of each pixel by analyzing the changes in pixel brightness between adjacent frames.

[0085] In this embodiment, image cropping has no strong requirements or dependence on the installation angle and height of the camera. Usually, it is sufficient to crop or block the upper half of the image. Similarly, there is no strong requirement to scale the resolution of the cropped image. It only needs to be scaled proportionally in length and width. The purpose is to reduce resource consumption while meeting the robustness of the algorithm. The specific scaling standard can be adjusted according to the original resolution size of the actual input image.

[0086] Step 203: Calculate the linear velocity of the robot based on the chassis wheel speed meter data.

[0087] The theoretical linear velocity of the robot can be calculated based on the chassis wheel speed meter data. It should be understood that when the robot is in a slipping state, the actual moving speed of the robot is much smaller than the calculated theoretical linear velocity.

[0088] In one embodiment, before calculating the linear velocity of the robot based on the chassis wheel speed meter data, the method further includes: synchronizing the acquisition time of the current image with the acquisition time of the current chassis wheel speed meter data.

[0089] In this embodiment, to ensure data alignment, the acquisition time of the current image is synchronized with the acquisition time of the current chassis wheel speedometer data, so as to synchronize the currently calculated optical flow tracking result and the speed information of the wheel speedometer.

[0090] Step 204 : When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, it is determined that the robot is in a slipping state.

[0091] This method can perform optical flow matching tracking based on feature points on the first image and the second image in the collected image data, calculate the average optical flow value of the successfully matched feature points, and the linear speed of the robot calculated based on the chassis wheel speed meter data. When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to the preset speed, it is determined that the robot is in a slipping state. This method can have good robustness on low-cost hardware, so that the robot has the ability to efficiently detect slipping in scenes prone to slipping.

[0092] In one embodiment, when the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, determining that the robot is in a slipping state includes:

[0093] Get the count value of the counter;

[0094] When the average optical flow value is less than the preset threshold, the count value is increased by one as the current count value;

[0095] If the current count value does not reach the preset value, re-execute the step of acquiring the image data and chassis wheel speed meter data collected by the robot until the average optical flow value is less than the preset threshold, and then increase the count value by one as the current count value;

[0096] When the current count value reaches a preset value and the linear speed is greater than or equal to the preset speed, it is determined that the robot is in a slipping state.

[0097] In this embodiment, in order to ensure the stability of detection, the average optical flow value can be calculated several times in succession. For example, a counter is configured. When the robot starts working, the count value of the counter is cleared. When the average optical flow value is less than the preset threshold, the count value is increased by one. For example, the preset value can be set to 3. If the current count value of the counter does not reach 3, the optical flow matching tracking step is re-executed. When the count value reaches 3 and the calculated linear velocity is greater than or equal to the preset speed (for example, set to 0.1m / s), it indicates that the robot is in a slipping state. During the continuous optical flow matching tracking process, if the average optical flow value calculated at a certain time is greater than or equal to the preset threshold (that is, there is no slipping), the count value is reset to zero and the slip detection is performed again.

[0098] In a specific embodiment, the slip detection method is as follows: Figure 4 ,include:

[0099] Obtain input information, including monocular RGB image input and chassis wheel speed meter input;

[0100] After receiving the image data, set the counter frame_cnt++. If frame_cnt=N, crop the image, scale the resolution of the cropped image, extract feature points, perform optical flow tracking and matching, and calculate the average optical flow value averageFlow of all successfully tracked feature points.

[0101] Calculate the wheel speed meter data vel_odom according to the chassis wheel speed meter input. vel_odom represents the linear speed data of the robot driven by the chassis wheel speed meter input.

[0102] If the averageFlow is less than a set threshold m, the counter flow_cnt is increased by 1, otherwise the counter is cleared to zero (the purpose of this is to ensure the stability of the detection, and the averageFlow value calculated several times in a row must be less than m, that is, it is counted by the counter flow_cnt). When flow_cnt is accumulated for more than 3 times in a row and the absolute value of the vel_odom linear velocity is greater than or equal to 0.1m / s, it means that the robot is in a slipping state.

[0103] In this embodiment, frame_cnt represents an image input counter. When the first input image is received, frame_cnt is incremented by 1, and the Nth image is continuously input. N is set according to the input frequency of the actual camera. In practice, it is a specific value. If the driving input frequency is low, N is set to be small, generally between 3 and 4. If the driving input frequency is high, N is set to be large, generally between 6 and 8, until frame_cnt = N. At this time, the frame_cnt counter is reset, and the images with frame_cnt = 1 and frame_cnt = N are cropped and the resolution is reduced. Then, feature points are extracted from the image with frame_cnt = 1, and the feature points are counted in the frame. Perform optical flow tracking and matching on the image with me_cnt=N, and calculate the average optical flow value averageFlow of all successfully matched feature points (statisticing the average pixel distance moved by all feature points). If averageFlow is less than a set threshold m, the counter flow_cnt is increased by 1, otherwise the counter is cleared (the purpose of this is to ensure the stability of detection, and the averageFlow value calculated several times in a row must be less than m, that is, it is counted by the counter flow_cnt). When flow_cnt is accumulated for 3 times in a row and the absolute value of the vel_odom linear velocity is greater than or equal to 0.1m / s, it means that the robot is slipping in place.

[0104] In the above embodiments of the present application, the average optical flow of the tracked feature points can be calculated based on low-resolution images by relying on feature point extraction and optical flow tracking. The average optical flow threshold can be limited according to actual experimental tests. It is still effective in low-light environments and when the camera is mostly blocked. According to the input frequency of the camera image, it can be effectively detected within 1 to 1.5 seconds with an accuracy rate of more than 95%. This method is independent of hardware performance, input frame rate and robot operation speed. The method is simple and effective, and can have good robustness on low-cost hardware with a monocular camera, so that the robot has the ability to efficiently detect slipping in prone-to-slip scenes and can quickly take measures to escape, thereby improving the robot's work efficiency.

[0105] Based on the same technical concept, the second embodiment of the present application provides a slip detection device, such as Figure 5 , the device comprises:

[0106] An acquisition module 501 is used to acquire image data and chassis wheel speed meter data collected by the robot;

[0107] a matching tracking module 502 configured to perform optical flow matching tracking based on feature points on a first image and a second image in the image data, and calculate an average optical flow value of the successfully matched feature points; wherein the first image is a first frame image at a target moment, and the second image is another frame image after the target moment;

[0108] A linear speed calculation module 503 is used to calculate the linear speed of the robot based on the chassis wheel speed meter data;

[0109] The determination module 504 is configured to determine that the robot is in a slipping state when the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed.

[0110] The device can perform optical flow matching tracking based on feature points on the first image and the second image in the collected image data, calculate the average optical flow value of the successfully matched feature points, and calculate the linear speed of the robot based on the chassis wheel speed meter data. When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to the preset speed, it is determined that the robot is in a slipping state. This method can have good robustness on low-cost hardware, so that the robot has the ability to efficiently detect slipping in scenes prone to slipping.

[0111] like Figure 6 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0112] Memory 113, for storing computer programs;

[0113] In one embodiment of the present application, the processor 111 is configured to execute a program stored in the memory 113 to implement the slip detection method provided by any of the aforementioned method embodiments, including:

[0114] Obtain image data collected by the robot and chassis wheel speed meter data;

[0115] Performing optical flow matching tracking based on feature points on a first image and a second image in the image data, and calculating an average optical flow value of the successfully matched feature points; wherein the first image is a first frame image at a target moment, and the second image is another frame image after the target moment;

[0116] Calculating the linear speed of the robot according to the chassis wheel speed meter data;

[0117] When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, it is determined that the robot is in a slipping state.

[0118] The communication bus mentioned in the terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0119] The communication interface is used for communication between the above terminal and other devices.

[0120] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0121] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0122] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the slip detection method provided in any one of the aforementioned method embodiments is implemented.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0124] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0125] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0126] It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. In the description, suffixes such as "module," "component," or "unit" used to represent elements are used solely to facilitate the description of the present application and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0127] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A skid detection method, characterized in that: The method comprises: Obtain image data collected by the robot and chassis wheel speed meter data; Performing optical flow matching tracking based on feature points on a first image and a second image in the image data, and calculating an average optical flow value of the successfully matched feature points; wherein the first image is a first frame image at a target moment, and the second image is another frame image after the target moment; Calculating the linear speed of the robot according to the chassis wheel speed meter data; When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, it is determined that the robot is in a slipping state.

2. The method according to claim 1, characterized in that Performing optical flow matching tracking based on feature points on the first image and the second image in the image data, and calculating the average optical flow value of the successfully matched feature points, including: Obtaining the acquisition frequency of the robot's camera; determining an interval between the first image and the second image according to the acquisition frequency; wherein the acquisition frequency is positively correlated with the interval; determining the second image from the image data according to the interval; Extracting feature points from the first image to obtain first feature points; Perform optical flow matching tracking of the first feature points in the second image, and calculate the average optical flow value of the successfully matched target feature points.

3. The method according to claim 2, characterized in that Before extracting feature points from the first image to obtain first feature points, the method further includes: performing grayscale processing on the first image and the second image respectively to obtain grayscale images; performing cropping processing on the grayscale image according to the installation height and tilt angle of the camera of the robot to obtain a cropped image; The cropped image is subjected to resolution scaling to obtain the processed first image and the second image.

4. The method according to claim 2, characterized in that Performing optical flow matching tracking of the first feature point in the second image and calculating an average optical flow value of successfully matched target feature points includes: performing optical flow matching tracking of the first feature points in the second image, and determining a successfully matched target feature point from the first feature points; determining a moving distance of each of the target feature points according to the first image and the second image; The average optical flow value is calculated according to the movement distances corresponding to all the target feature points.

5. The method according to claim 1, wherein When the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed, determining that the robot is in a slipping state includes: Get the count value of the counter; When the average optical flow value is less than a preset threshold, the count value is increased by one as the current count value; If the current count value does not reach the preset value, re-execute the step of acquiring the image data and chassis wheel speedometer data collected by the robot until the step of adding one to the count value as the current count value if the average optical flow value is less than the preset threshold; When the current count value reaches a preset value and the linear speed is greater than or equal to a preset speed, it is determined that the robot is in a slipping state.

6. The method according to claim 5, characterized in that The method further comprises: When the average optical flow value is greater than or equal to the preset threshold, the count value is set to zero.

7. The method according to claim 1, characterized in that Before calculating the linear velocity of the robot according to the chassis wheel speed meter data, the method further includes: Synchronize the acquisition time of the current image with the acquisition time of the current chassis wheel speed meter data.

8. A skid detection device, characterized in that: The device comprises: An acquisition module is used to obtain image data collected by the robot and chassis wheel speed meter data; a matching tracking module, configured to perform optical flow matching tracking based on feature points on a first image and a second image in the image data, and calculate an average optical flow value of the successfully matched feature points; wherein the first image is a first frame image at a target moment, and the second image is another frame image after the target moment; A linear speed calculation module, configured to calculate the linear speed of the robot based on the chassis wheel speedometer data; The determination module is configured to determine that the robot is in a slipping state when the average optical flow value is less than a preset threshold and the linear speed is greater than or equal to a preset speed.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is configured to implement the slip detection method according to any one of claims 1 to 7 when executing a program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the slip detection method according to any one of claims 1 to 7 is implemented.

Images