Obstacle avoidance method and device of mobile equipment, equipment, storage medium and product

Abstract

The invention discloses an obstacle avoidance method and device for mobile equipment, equipment, a storage medium and a product. The mobile equipment comprises an equipment body, a light emitter and an image acquisition device, the light emitter is used for emitting initial light, the image acquisition device is used for acquiring a light spot image of a light spot formed by the initial light, and the obstacle avoidance method of the mobile equipment comprises the steps of obtaining a first light spot image of the mobile equipment in an initial state, determining a first pixel point position of the first light spot image; obtaining a second light spot image in the operation process of the mobile equipment, and determining the position of a second pixel point of the second light spot image; based on the first pixel point position and the second pixel point position, determining an offset result of whether the light spot offsets along the height direction; and determining whether there is an obstacle in front of the mobile device based on the offset result. Automatic detection of obstacles in front of the mobile equipment can be realized based on the combination of the light emitter and the image acquisition device, the obstacle avoidance cost is low, and large-scale deployment is facilitated.

Description

Technical Field

[0001] The present application relates to the field of mobile control, and in particular to an obstacle avoidance method, apparatus, device, storage medium and product for a mobile device. Background Art

[0002] With the development of intelligent technology, there are more mobile devices in home scenes, such as mobile sweeping robots, mobile companion robots, etc. Mobile devices have the need to avoid obstacles in practical applications. In related technologies, laser radar or visual evaluation and SLAM (Simultaneous Localization and Mapping) technology is often introduced to achieve obstacle avoidance. Among them, laser radar technology relies on precision machinery and high-end optical components, such as laser emission system, laser receiving system, information processing system and scanning system; in addition, there are high-precision clocks, synchronization controllers and special cables required for time synchronization systems. These components are not only expensive, but also require a large economic investment when maintained and replaced. Visual SLAM technology is very sensitive to the lighting conditions of the environment. Too strong or too dark light will affect the image quality captured by the camera; in addition, dynamic obstacles will also cause inaccurate mapping. In the SLAM system using laser radar, factors such as smoke or rain and fog can scatter or absorb laser signals, affecting the measurement of distance and the detection of obstacles. Summary of the invention

[0003] In view of this, embodiments of the present application provide an obstacle avoidance method, apparatus, device, storage medium, and product for a mobile device, aiming to reduce the obstacle avoidance cost of the mobile device and improve the reliability of obstacle avoidance.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides an obstacle avoidance method for a mobile device, wherein the mobile device includes a device body, a light emitter, and an image acquisition device, wherein the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit an initial light, and the image acquisition device is used to acquire a light spot image of a light spot formed by the initial light, and the method includes:

[0006] Acquire a first light spot image of the mobile device in an initial state, and determine a first pixel point position of the first light spot image;

[0007] Acquire a second light spot image during the operation of the mobile device, and determine a second pixel point position of the second light spot image;

[0008] Determine, based on the first pixel point position and the second pixel point position, whether the light spot is shifted in a height direction;

[0009] Based on the offset result, it is determined whether there is an obstacle in front of the mobile device.

[0010] In the above solution, determining whether there is an obstacle in front of the mobile device based on the offset result includes:

[0011] If it is determined that the offset result is the first result indicating no offset, it is determined that there is no obstacle in front of the mobile device;

[0012] If it is determined that the displacement result is the second result indicating an upward displacement, it is determined that there is a first obstacle of a protruding type in front of the mobile device;

[0013] If it is determined that the displacement result is the third result indicating a downward displacement, it is determined that a second obstacle of a concave type exists in front of the mobile device.

[0014] In the above solution, after determining that there is a first obstacle of the protruding type in front of the mobile device, the method further includes:

[0015] Acquire a first offset of the light spot upwardly along a height direction;

[0016] Based on the first offset and a triangle similarity rule, a first distance between the mobile device and the first obstacle is determined.

[0017] In the above solution, the image acquisition device is a binocular camera, and the step of obtaining a first offset of the light spot upwardly shifted in the height direction includes:

[0018] Determine a first offset of the light spot upward in the height direction at the first obstacle position based on a ranging algorithm of the binocular camera;

[0019] The determining, based on the first offset and a triangle similarity rule, a first distance between the mobile device and the first obstacle comprises:

[0020] Determine a first distance between the mobile device and the first obstacle based on the first offset, initial calibration parameters and the triangle similarity law;

[0021] Among them, the initial calibration parameters include at least: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device and a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state.

[0022] In the above solution, after determining that there is a second obstacle of a concave type in front of the mobile device, the method further includes:

[0023] Acquire a second offset of the light spot downwardly along a height direction;

[0024] Based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a recess depth corresponding to the second obstacle is determined.

[0025] In the above solution, the image acquisition device is a binocular camera, and the step of obtaining the second offset of the light spot shifted downward in the height direction includes:

[0026] In response to the binocular camera capturing a second light spot image of the light spot corresponding to the second obstacle, determining, based on a ranging algorithm of the binocular camera, a second offset amount by which the light spot is offset downward in a height direction at a position where the initial light rays intersect the horizontal ground;

[0027] The determining, based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle includes:

[0028] Determine a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle based on the second offset, the initial calibration parameters and the triangle similarity law;

[0029] Among them, the initial calibration parameters at least include: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device, a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state, and a third calibration parameter characterizing the height of the image acquisition device.

[0030] In the above scheme, the method further comprises:

[0031] The operating state of the mobile device is controlled based on the first distance or based on at least one of the second distance and the recess depth.

[0032] In the above solution, the light emitter is located above the image acquisition device, and the line between the light emitter and the image acquisition device is perpendicular to the horizontal ground. The method further includes:

[0033] Obtaining the inclination angle of the mobile device during operation;

[0034] The posture of the mobile device is adjusted based on the tilt angle so that the connecting line is perpendicular to the horizontal ground.

[0035] In a second aspect, an embodiment of the present application further provides an obstacle avoidance device for a mobile device, the mobile device comprising a device body, a light emitter and an image acquisition device, the light emitter and the image acquisition device being arranged on the device body, the light emitter being used to emit an initial light, the image acquisition device being used to acquire a light spot image of a light spot formed by the initial light, the obstacle avoidance device comprising:

[0036] A first acquisition module, used to acquire a first light spot image of the mobile device in an initial state, and determine a first pixel point position of the first light spot image;

[0037] A second acquisition module, used to acquire a second light spot image during the operation of the mobile device, and determine a second pixel point position of the second light spot image;

[0038] A processing module, configured to determine a shift result of whether the light spot shifts along a height direction based on the first pixel point position and the second pixel point position;

[0039] The determination module is used to determine whether there is an obstacle in front of the mobile device based on the offset result.

[0040] In a third aspect, an embodiment of the present application provides a mobile device, comprising a device body, a light emitter and an image acquisition device, wherein the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit initial light, and the image acquisition device is used to acquire a light spot image of a light spot formed by the initial light. The mobile device also includes: a processor and a memory for storing a computer program that can be run on the processor, wherein when the processor is used to run the computer program, the steps of the method described in the first aspect of the embodiment of the present application are executed.

[0041] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect of the embodiment of the present application are implemented.

[0042] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect of the embodiment of the present application.

[0043] The technical solution provided by the embodiment of the present application is that the mobile device includes a device body, a light emitter and an image acquisition device, the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit the initial light, and the image acquisition device is used to acquire the light spot image of the light spot formed by the initial light, and the obstacle avoidance method of the mobile device includes: obtaining the first light spot image of the mobile device in the initial state, and determining the first pixel point position of the first light spot image; obtaining the second light spot image during the operation of the mobile device, and determining the second pixel point position of the second light spot image; based on the first pixel point position and the second pixel point position, determining the offset result of whether the light spot is offset along the height direction; based on the offset result, determining whether there is an obstacle in front of the mobile device. In this way, the automatic detection of obstacles in front of the mobile device can be realized based on the combination of the light emitter and the image acquisition device, and the obstacle avoidance cost is low, which is conducive to large-scale deployment; in addition, the installation and debugging are simple, the failure points are reduced by simplifying the components, there is no need to consider the time synchronization problem, and the illumination changes in the compatible environment are also improved. The reliability of obstacle avoidance of the mobile device is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the principle of single-point laser ranging in the related technology;

[0045] Figure 2 This is a schematic diagram of the structure of a mobile device according to an embodiment of the present application;

[0046] Figure 3 A flowchart of an obstacle avoidance method for a mobile device according to an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a process flow of an application example method of this application;

[0048] Figure 5 This is a schematic diagram of the basic principle of obstacle avoidance and ranging in the application embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the distance measurement principle of a wall obstacle in the application embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the distance measurement principle of a recessed obstacle in an application embodiment of the present application;

[0051] Figure 8 A schematic diagram of the structure of an obstacle avoidance device for a mobile device according to an embodiment of the present application;

[0052] Fig. 9 Another structural diagram of a mobile device according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0055] Among the related technologies, the core principle of laser ranging technology based on LiDAR is to accurately calculate the position and distance of an object by measuring the time or distance from the laser emission to the reflection back to the receiver, so as to avoid obstacles. It has the advantages of high precision, fast response, long-distance measurement and non-contact measurement. Common laser ranging methods include time of flight (TOF), phase ranging and triangulation ranging. Among them, triangulation ranging calculates the distance of an object through the principle of geometric optics and has high measurement accuracy and reliability. In addition to laser ranging, modern robot obstacle avoidance technology also includes methods such as visual estimation and simultaneous localization and mapping (SLAM). These technologies complement each other and improve the overall performance and adaptability of the system.

[0056] Among them, the triangulation distance measurement method refers to the laser emitting laser, and after irradiating the object, the reflected light is received by the linear CCD (Charge-coupled Device) detector. Since the laser and the detector are separated by a distance, objects at different distances will be imaged at different positions on the CCD according to the optical path. By calculating according to the triangulation formula, the distance of the measured object can be deduced.

[0057] Figure 1 The schematic diagram of the principle of single-point laser ranging is shown. The laser head Laser and the camera camera are on the baseline, the distance between them is s, the focal length of the camera is f, and the angle between the laser head and the baseline is β. Assume that the target object Object is irradiated by the point laser and the position reflected back to the camera imaging plane is point P. Based on geometric knowledge, similar triangles can be drawn. The triangle formed by the laser head, camera and target object is similar to the camera, imaging point P and auxiliary point P′.

[0058] Assume PP′=x, q, d as Figure 1 As shown, q represents the distance between the target object Object and the baseline, and d represents the distance between the laser head Laser and the target object Object. From similar triangles, we can get: f / x=q / s==>q=f*s / x, which can be calculated in two parts: (1) x=x1+x2=f / tanβ+pixelSize*position, where pixelSize is the pixel unit size, and position is the position of the imaged pixel coordinate relative to the imaging center; (2) Finally, the distance d can be obtained: d=q / sinβ.

[0059] The core principle of the TOF ranging method is to shoot a very short laser beam at the detection object, and directly measure the flight time of the laser emission, hitting the detection object and returning to the detector to infer the distance between the detector and the object being measured. This ranging method is similar to traditional microwave radar ranging, and the return time delay corresponds to twice the target distance.

[0060] Among them, phase laser ranging is also a TOF method, but it reverses the TOF by solving the phase difference between the transmitted wave and the received wave. And this phase is not the original phase of the light, but the phase of the modulated light intensity. Specifically, it is to modulate the light intensity of the transmitted light wave, and indirectly measure the time by measuring the phase difference, which is much easier to process than directly measuring the round-trip time.

[0061] Among them, the core principle of SLAM is that the robot simultaneously locates and builds a map in an unknown environment. Using sensors such as lidar, cameras, and inertial measurement units (IMUs), SLAM technology can create a three-dimensional map of the environment in real time while determining the position of the robot in the map. This process involves complex data fusion and algorithm processing, such as particle filtering, extended Kalman filtering, etc., to optimize the accuracy of positioning and the reliability of the map. As a branch of SLAM technology, visual SLAM mainly uses cameras as sensors to capture environmental images. By analyzing feature point matching and motion estimation between consecutive image frames, visual SLAM can infer the trajectory of the camera and reconstruct the structure of the surrounding environment. The advantage of this method is that it is low cost and has low hardware requirements, but it is generally effective in low-texture or fast-moving environments.

[0062] It should be pointed out that the above-mentioned ranging technologies have the defects of high cost and poor reliability. For example, traditional LiDAR technology relies on precision machinery and high-end optical components, such as laser emission systems, laser receiving systems, information processing systems and scanning systems; in addition, there are high-precision clocks, synchronization controllers and special cables required for time synchronization systems. These components are not only expensive, but also require a large economic investment when they are maintained and replaced. There is a time synchronization problem between the photosensitive unit and the laser transmitter in the TOF ranging method. In the TOF system, the laser transmitter emits a laser pulse, and the photosensitive unit must accurately record the time point when the pulse is reflected back by the target. The time of emission and reception needs to be synchronized very accurately to ensure the accuracy of the measurement. Visual SLAM technology is very sensitive to the lighting conditions of the environment. Too strong or too dark light will affect the quality of the image captured by the camera; in addition, dynamic obstacles will also cause inaccurate mapping. In the SLAM system using LiDAR, factors such as smoke or rain and fog can scatter or absorb laser signals, affecting the measurement of distance and the detection of obstacles.

[0063] Based on this, in various embodiments of the present application, a low-cost and highly reliable obstacle avoidance method for a mobile device is provided.

[0064] Before describing the obstacle avoidance method of the application embodiment, the mobile device involved in the application embodiment is described below. Figure 2 As shown, the mobile device includes a device body 201, a light emitter 202 and an image acquisition device 203, wherein the light emitter 202 and the image acquisition device 203 are both arranged on the device body 201, wherein the light emitter 202 is used to emit initial light, and the image acquisition device 203 is used to acquire a light spot image of a light spot formed by the initial light.

[0065] Here, the device body 201 may be a robot with walking ability, the light emitter 202 may be a laser emitter or a visible light emitter (e.g., an infrared emitter), and the image acquisition device 203 may be a binocular camera. It should be noted that the light spot formed by the initial light is different depending on whether there is an obstacle in front of the mobile device, and the image acquisition device 203 can convert the actual displacement of the light spot based on the pixel displacement of the light spot imaging.

[0066] like Figure 3 As shown, an embodiment of the present application provides an obstacle avoidance method for a mobile device, the method comprising:

[0067] Step 301: Acquire a first light spot image of the mobile device in an initial state, and determine a first pixel point position of the first light spot image.

[0068] Here, the initial state refers to a state in which there are no obstacles in front of the mobile device. The image acquisition device can capture the first light spot image corresponding to the light spot formed by the initial light emitted by the light emitter on the horizontal ground, and then obtain the first pixel point position corresponding to the light spot on the imaging plane.

[0069] Step 302: Acquire a second light spot image during the operation of the mobile device, and determine a second pixel point position of the second light spot image.

[0070] Here, the image acquisition device can periodically capture the second light spot image during the operation of the mobile device, that is, capture the second light spot image corresponding to the light spot formed by the initial light emitted by the light emitter on the horizontal ground or on an obstacle during the operation of the mobile device, and then obtain the second pixel point position corresponding to the light spot on the imaging plane.

[0071] Step 303: Based on the first pixel point position and the second pixel point position, determine whether the light spot is offset along the height direction.

[0072] Here, the mobile device may compare the acquired first pixel point position with the second pixel point position to determine whether the light spot is offset along the height direction, and obtain an offset result.

[0073] Step 304: Determine whether there is an obstacle in front of the mobile device based on the offset result.

[0074] Here, the mobile device may determine whether there is an obstacle in front of the mobile device based on the aforementioned offset result.

[0075] It can be understood that the embodiments of the present application can realize automatic detection of obstacles in front of the mobile device based on the combination of the light emitter and the image acquisition device, with low obstacle avoidance cost, which is conducive to large-scale deployment; in addition, the installation and debugging are simple, the failure points are reduced by simplifying the components, there is no need to consider time synchronization issues, and it is compatible with the changes in lighting in the environment, which also improves the reliability of obstacle avoidance of the mobile device.

[0076] Exemplarily, determining whether there is an obstacle in front of the mobile device based on the offset result includes:

[0077] If it is determined that the offset result is the first result indicating no offset, it is determined that there is no obstacle in front of the mobile device;

[0078] If it is determined that the displacement result is the second result indicating an upward displacement, it is determined that there is a first obstacle of a protruding type in front of the mobile device;

[0079] If it is determined that the displacement result is the third result indicating a downward displacement, it is determined that a second obstacle of a concave type exists in front of the mobile device.

[0080] Here, if the mobile device determines that the offset result is the first result that characterizes that the light spot is not offset in the height direction, it indicates that the initial light emitted by the light emitter forms a light spot on the horizontal ground, and it is determined that there is no obstacle in front of the mobile device; if the offset result is the second result that characterizes that the light spot is offset upward in the height direction, it indicates that the initial light emitted by the light emitter forms a light spot above the horizontal ground, and it is determined that there is a first obstacle of the convex type in front of the mobile device; if the offset result is the third result that characterizes that the light spot is offset downward in the height direction, it indicates that the initial light emitted by the light emitter forms a light spot below the horizontal ground, and it is determined that there is a second obstacle of the concave type in front of the mobile device. In this way, based on the offset result of whether the light spot is offset in the height direction, it is possible to automatically detect whether there is an obstacle in front of the mobile device during the displacement process, thereby providing a basis for the mobile device to automatically avoid obstacles.

[0081] Exemplarily, after determining that there is a first obstacle of the protruding type in front of the mobile device, the method further includes:

[0082] Acquire a first offset of the light spot upwardly along a height direction;

[0083] A first distance between the mobile device and the first obstacle is determined based on the first offset and a triangle similarity rule.

[0084] Here, after the mobile device determines that there is a first protruding obstacle in front of it during its movement, it can also obtain a first offset of the light spot upward in the height direction, and use the triangle similarity law to determine the first distance between the mobile device and the first obstacle.

[0085] Exemplarily, the image acquisition device is a binocular camera, and obtaining a first offset of the light spot shifted upward in a height direction includes:

[0086] Determine a first offset of the light spot upward in the height direction at the first obstacle position based on a ranging algorithm of the binocular camera;

[0087] The determining, based on the first offset and a triangle similarity rule, a first distance between the mobile device and the first obstacle comprises:

[0088] Determine a first distance between the mobile device and the first obstacle based on the first offset, initial calibration parameters and the triangle similarity law;

[0089] Among them, the initial calibration parameters include at least: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device and a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state.

[0090] It can be understood that the ranging algorithm based on the binocular camera can, after detecting the position of the light spot on the first obstacle, obtain the first offset of the light spot upward in the height direction at the position of the first obstacle based on the position of the light spot in the initial state, and then determine the first distance based on the triangle similarity law, the first calibration parameter and the second calibration parameter.

[0091] Exemplarily, after determining that there is a second obstacle of a concave type in front of the mobile device, the method further includes:

[0092] Acquire a second offset of the light spot downwardly along a height direction;

[0093] Based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a recess depth corresponding to the second obstacle is determined.

[0094] Here, after the mobile device determines that there is a second concave-like obstacle in front of it during its movement, it can also obtain a second offset of the light spot downward in the height direction, and use the triangle similarity law to determine the second distance between the mobile device and the second obstacle and / or the depth of the concave corresponding to the second obstacle.

[0095] Exemplarily, the image acquisition device is a binocular camera, and obtaining a second offset of the light spot shifted downward in a height direction includes:

[0096] In response to the binocular camera capturing a second light spot image of the light spot corresponding to the second obstacle, determining, based on a ranging algorithm of the binocular camera, a second offset amount by which the light spot is offset downward in a height direction at a position where the initial light rays intersect the horizontal ground;

[0097] The determining, based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle includes:

[0098] Determine a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle based on the second offset, the initial calibration parameters and the triangle similarity law;

[0099] Among them, the initial calibration parameters at least include: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device, a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state, and a third calibration parameter characterizing the height of the image acquisition device.

[0100] It can be understood that after detecting the position of the light spot on the second obstacle, the ranging algorithm based on the binocular camera can obtain the second offset of the light spot downward in the height direction at the intersection of the initial light and the horizontal ground based on the position of the light spot in the initial state, and then determine the second spacing and / or the depth of the depression corresponding to the second obstacle based on the triangle similarity law, the first calibration parameter, the second calibration parameter and the third calibration parameter.

[0101] Exemplarily, the method further includes:

[0102] The operating state of the mobile device is controlled based on the first distance or based on at least one of the second distance and the recess depth.

[0103] Here, the mobile device can control the operating state of the mobile device based on the first distance to achieve obstacle avoidance control of the first obstacle. For example, if it is determined that the first distance is less than the first set distance threshold, the mobile device is controlled to brake until the displacement stops; the mobile device can also control the operating state of the mobile device based on the second distance and / or the depression depth to achieve obstacle avoidance control of the second obstacle. For example, if it is determined that the second distance is less than the second set distance threshold and / or the depression depth is greater than the set depth threshold, the mobile device is controlled to brake until the displacement stops.

[0104] Exemplarily, the light emitter is located above the image acquisition device, and a line between the light emitter and the image acquisition device is perpendicular to the horizontal ground, and the method further includes:

[0105] Obtaining the inclination angle of the mobile device during operation;

[0106] The posture of the mobile device is adjusted based on the tilt angle so that the connecting line is perpendicular to the horizontal ground.

[0107] Here, a tilt sensor can be set on the mobile device, and the tilt angle of the mobile device during operation can be obtained based on the detection value of the tilt sensor, and the posture of the mobile device can be adjusted based on the tilt angle, so that the connection line between the light emitter and the image acquisition device on the mobile device is always continuous and perpendicular to the horizontal ground. Then, based on the right triangle constructed between the connection line and the horizontal ground, the aforementioned first spacing, second spacing and / or depression depth can be obtained using the law of similar triangles.

[0108] The present application is further described in detail below in conjunction with application examples.

[0109] In this application embodiment, the device body of the mobile device is a robot, and a laser emitter, a binocular camera, and an inclination sensor are arranged on the robot. The laser emitter is used to emit laser light, and the laser light emitted by the laser emitter is at a set angle with the center line of the robot; the binocular camera is used to capture the pixel displacement of the light spot, and map the pixel displacement to the displacement of the actual light spot according to the binocular camera algorithm; the inclination sensor detects the inclination to ensure that the line connecting the laser emitter and the binocular camera is perpendicular to the horizontal ground.

[0110] Figure 4 The method of this application embodiment is shown, and the method includes the following steps:

[0111] Step 401, system initialization.

[0112] The mobile device is powered on and initialized. The binocular camera captures the initial position of the laser point, reads the inclination sensor, and determines and adjusts the line between the laser transmitter and the binocular camera on the robot to be perpendicular to the horizontal ground.

[0113] Step 402, determine whether the laser point is offset in the height direction during the operation of the mobile device. If it is the first result indicating no offset, execute step 403; if it is the second result indicating an upward offset, execute step 404; if it is the third result indicating a downward offset, execute step 405.

[0114] Step 403: determine that there is no obstacle in front of the mobile device and move normally.

[0115] Step 404: determine whether there is a first obstacle of the protruding type in front of the mobile device, calculate a first distance, and perform obstacle avoidance control based on the first distance.

[0116] Step 405 , determining whether there is a second obstacle of the concave type in front of the mobile device, calculating a second distance and a depth of the concave, and performing obstacle avoidance control based on the second distance and the depth of the concave.

[0117] like Figure 5 As shown, in this application embodiment, when the mobile device calculates the distance between the robot and an obstacle (such as a wall or a depression) and the depth of the depression based on a laser transmitter and a binocular camera, a similar triangle can be constructed using a triangle formed by three points A, B, and C and a new triangle formed by A, B and a new offset point after encountering an obstacle, thereby determining the distance and depth to the obstacle.

[0118] Reference Figure 5 The robot is shown as a rectangular structure. Point A corresponds to the laser emitter, point B corresponds to the binocular camera, point C is the contact point between the emitted laser and the horizontal ground, d is the length between the binocular camera and the laser emitter, d' is the height between the binocular camera and the ground, d+d'=h, and l is the farthest ranging distance, that is, the horizontal emission distance of the emitted laser. When the emitted laser encounters a wall or a pit, the pixel point of the laser point on the imaging plane will move up or down. Through the ranging algorithm of the binocular camera, the displacement of the pixel point can be converted into the actual displacement distance corresponding to the laser point.

[0119] Figure 6 The figure shows the situation corresponding to the first obstacle of the raised type (for example, a wall), where the laser is emitted from point A and reflected at point G on the wall, and the reflected light is captured by point B. The distance from the robot to the wall is calculated using the principle of similar triangles through the actual displacement of the light spot on the wall, combined with the known binocular camera height and the distance between the binocular camera and the laser emitter.

[0120] This algorithm measures the distance between the robot and the wall (obstacle) to prevent collision and damage to the robot or object. It includes the following steps:

[0121] Step 1) Set variables

[0122] Figure 6Point A is the laser emission point (i.e., the location of the laser emitter), point B is the reflected light capture point (i.e., the location of the binocular camera), point C is the laser reflection point corresponding to the initial state (i.e., the obstacle-free state), point G is the reflection point of the laser on the wall, point S is the imaging position of the laser on the wall in the initial state, x is the distance from the robot to the wall (i.e., JE), y is the actual displacement of the light spot on the wall converted by the binocular camera (corresponding to the aforementioned first offset), and d is the known distance between the laser emitter and the binocular camera.

[0123] Step 2), through the relationship of similar triangles, determine △AJC∽△GEC, and the similarity ratio is: d / y.

[0124] Based on the above similarity ratio, we can get: CE / CJ=y / d.

[0125] Step 3) Calculate JE by similarity ratio.

[0126] JE=CJ-CE, CE=CJ*y / d, JE=CJ*(1-y / d), that is, the first distance between the robot and the wall is obtained.

[0127] Step 4) The robot performs obstacle avoidance action according to the first distance.

[0128] Here, the robot determines that the first distance is less than the first set distance threshold, then determines that the robot is at risk of collision, and controls the robot to brake until the displacement stops.

[0129] Figure 7 The figure shows the situation corresponding to the second obstacle of the concave type. When there is a concave in front, the robot cannot see point G at first. Only when the robot can see point G can it calculate: calculate the offset s from point C to point K. Apply the principle of similar triangles again to calculate the distance from the binocular camera to the edge of the concave, as well as the depth of the concave.

[0130] This algorithm measures the distance between the robot and the depression (pit) and the depth of the depression (pit) to prevent the robot from falling. The specific steps include:

[0131] Step 1) Set variables

[0132] Figure 7Point A is the laser emission point (i.e., the location of the laser emitter), point B is the reflected light capture point (i.e., the location of the binocular camera), point C is the laser reflection point corresponding to the initial state (i.e., the obstacle-free state), point G is the laser reflection point at the depression, BG is the line connecting the intersection of the binocular camera and the depression, point E corresponds to the location of the virtual curtain, point F is the intersection of the binocular camera and the horizontal ground, point K is the imaging position of point G in the virtual curtain, JC is the extension line of the horizontal ground, QG is the horizontal extension line of the depression, and s is the actual displacement distance between points C and K on the virtual curtain (corresponding to the second offset mentioned above), which is calculated by the binocular camera according to the pixel points that the light spot moves down on the screen.

[0133] Step 2) Determine △ABG∽△CKG through the relationship between similar triangles, and the similarity ratio is: d / s.

[0134] Step 3) Calculate the depression depth and the second spacing by similarity ratio.

[0135] Based on the above similarity ratio, we can get: CG = (s / d) * AG. In similar triangles △AJC and △CEG, CE / AJ = CG / AC. From the relationship, we can know that: AG / AC = d / (ds). Substituting it into the calculation, we can get: CE = (s*h) / (ds), so we can conclude that the depth of the depression is: s*h / (ds).

[0136] In similar triangles △BJF and △BQG, JF / QG=d' / (d'+CE), in similar triangles △BJF and △FHG, HG / JF=CE / d', in △AJC and △AQG, JC / QG=h / (h+CE), according to the above relationship, we can get: All the above variables can be calculated, so the distance to the depression can be calculated: That is, the second spacing is obtained.

[0137] Step 4) The robot performs obstacle avoidance actions.

[0138] Here, the robot determines that the second spacing is less than the second set distance threshold and the depression depth is greater than the set depth threshold, then it is determined that the robot is at risk of tipping over, and the robot is controlled to brake until it stops moving.

[0139] It should be pointed out that the method of this application embodiment, combined with a simple and low-cost camera and laser transmitter to replace the complex lidar system, significantly reduces the cost and simplifies the system structure, thereby improving the reliability of the system and the convenience of maintenance. In addition, this solution avoids the need for precise time synchronization, reducing the complexity and cost of technical implementation. At the same time, it has strong adaptability to environmental changes, can effectively deal with dynamic obstacles, and is suitable for a variety of lighting conditions, greatly expanding the feasibility of application scenarios and deployment.

[0140] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an obstacle avoidance device for a mobile device, which corresponds to the above-mentioned obstacle avoidance method, and each step in the above-mentioned obstacle avoidance method embodiment is also fully applicable to the present obstacle avoidance device embodiment.

[0141] like Figure 8 As shown, the obstacle avoidance device includes: a first acquisition module 801, a second acquisition module 802, a processing module 803 and a determination module 804. The first acquisition module 801 is used to acquire a first light spot image of the mobile device in an initial state and determine a first pixel point position of the first light spot image; the second acquisition module 802 is used to acquire a second light spot image during the operation of the mobile device and determine a second pixel point position of the second light spot image; the processing module 803 is used to determine a displacement result of whether the light spot is displaced in the height direction based on the first pixel point position and the second pixel point position; the determination module 804 is used to determine whether there is an obstacle in front of the mobile device based on the displacement result.

[0142] Exemplarily, the determination module 804 is specifically configured to:

[0143] If it is determined that the offset result is the first result indicating no offset, it is determined that there is no obstacle in front of the mobile device;

[0144] If it is determined that the displacement result is the second result indicating an upward displacement, it is determined that there is a first obstacle of a protruding type in front of the mobile device;

[0145] If it is determined that the displacement result is the third result indicating a downward displacement, it is determined that a second obstacle of a concave type exists in front of the mobile device.

[0146] Exemplarily, after the determining module 804 determines that there is a first obstacle of the protruding type in front of the mobile device, the processing module 803 is further configured to:

[0147] Acquire a first offset of the light spot upwardly along a height direction;

[0148] Based on the first offset and a triangle similarity rule, a first distance between the mobile device and the first obstacle is determined.

[0149] Exemplarily, the image acquisition device is a binocular camera, and obtaining a first offset of the light spot shifted upward in a height direction includes:

[0150] Determine a first offset of the light spot upward in the height direction at the first obstacle position based on a ranging algorithm of the binocular camera;

[0151] The determining, based on the first offset and a triangle similarity rule, a first distance between the mobile device and the first obstacle comprises:

[0152] Determine a first distance between the mobile device and the first obstacle based on the first offset, initial calibration parameters and the triangle similarity law;

[0153] Among them, the initial calibration parameters include at least: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device and a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state.

[0154] Exemplarily, after the determining module 804 determines that there is a second obstacle of a concave type in front of the mobile device, the processing module 803 is further configured to:

[0155] Acquire a second offset of the light spot downwardly along a height direction;

[0156] Based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a recess depth corresponding to the second obstacle is determined.

[0157] Exemplarily, the image acquisition device is a binocular camera, and obtaining a second offset of the light spot shifted downward in a height direction includes:

[0158] In response to the binocular camera capturing a second light spot image of the light spot corresponding to the second obstacle, determining, based on a ranging algorithm of the binocular camera, a second offset amount by which the light spot is offset downward in a height direction at a position where the initial light rays intersect the horizontal ground;

[0159] The determining, based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle includes:

[0160] Determine a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle based on the second offset, the initial calibration parameters and the triangle similarity law;

[0161] Among them, the initial calibration parameters at least include: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device, a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state, and a third calibration parameter characterizing the height of the image acquisition device.

[0162] Exemplarily, the obstacle avoidance device further includes: a control module 805, configured to control the operating state of the mobile device based on the first distance or based on at least one of the second distance and the recess depth.

[0163] Exemplarily, the light emitter is located above the image acquisition device, and the line between the light emitter and the image acquisition device is perpendicular to the horizontal ground. The control module 805 is also used to: obtain the inclination angle of the mobile device during operation; adjust the posture of the mobile device based on the inclination angle so that the line is perpendicular to the horizontal ground.

[0164] In actual application, the first acquisition module 801, the second acquisition module 802, the processing module 803, the determination module 804 and the control module 805 can be implemented by a processor in the mobile device. Of course, the processor needs to run the computer program in the memory to implement its functions.

[0165] It should be noted that the obstacle avoidance device for a mobile device provided in the above embodiment only uses the division of the above program modules as an example when the mobile device avoids obstacles. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the obstacle avoidance device for a mobile device provided in the above embodiment and the obstacle avoidance method embodiment for a mobile device belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0166] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a mobile device. Fig. 9 Only an exemplary structure of the mobile device is shown, not all structures, and it can be implemented as needed. Fig. 9 Partial or complete structure shown.

[0167] like Fig. 9 As shown, the mobile device 900 provided in the embodiment of the present application includes: at least one processor 901, a memory 902, a user interface 903 and at least one network interface 904. The various components in the mobile device 900 are coupled together through a bus system 905. It can be understood that the bus system 905 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 905 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Fig. 9 Various buses are labeled as bus system 905.

[0168] The user interface 903 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0169] The memory 902 in the embodiment of the present application is used to store various types of data to support the operation of the mobile device. Examples of such data include: any computer program used to operate on the mobile device.

[0170] The obstacle avoidance method for a mobile device disclosed in the embodiment of the present application can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the obstacle avoidance method for the mobile device can be completed by an integrated logic circuit of hardware or software instructions in the processor 901. The above-mentioned processor 901 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 901 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 902. The processor 901 reads the information in the memory 902 and completes the steps of the obstacle avoidance method for the mobile device provided in the embodiment of the present application in combination with its hardware.

[0171] In an exemplary embodiment, the mobile device may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0172] Exemplarily, the mobile device also includes: a device body, a light emitter and an image acquisition device, wherein the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit initial light, and the image acquisition device is used to capture a light spot image of a light spot formed by the initial light.

[0173] It can be understood that the memory 902 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0174] In an exemplary embodiment, the present application also provides a computer storage medium, which may be a computer-readable storage medium, for example, a memory 902 storing a computer program, and the computer program may be executed by a processor 901 of a mobile device to complete the steps described in the method of the present application. The computer-readable storage medium may be a memory such as a ROM, a PROM, an EPROM, an EEPROM, a Flash Memory, a magnetic surface memory, an optical disk, or a CD-ROM.

[0175] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, including a computer program, and the computer program can be executed by the processor 901 of the mobile device to complete the steps described in the method of the embodiment of the present application.

[0176] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0177] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0178] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An obstacle avoidance method for a mobile device, characterized in that: The mobile device comprises a device body, a light emitter and an image acquisition device, wherein the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit initial light, and the image acquisition device is used to acquire a light spot image of a light spot formed by the initial light. The method comprises: Acquire a first light spot image of the mobile device in an initial state, and determine a first pixel point position of the first light spot image; Acquire a second light spot image during the operation of the mobile device, and determine a second pixel point position of the second light spot image; Determine, based on the first pixel point position and the second pixel point position, whether the light spot is shifted in a height direction; Based on the offset result, it is determined whether there is an obstacle in front of the mobile device.

2. The method according to claim 1, characterized in that: The determining, based on the offset result, whether there is an obstacle in front of the mobile device includes: If it is determined that the offset result is the first result indicating no offset, it is determined that there is no obstacle in front of the mobile device; If it is determined that the displacement result is the second result indicating an upward displacement, it is determined that there is a first obstacle of a protruding type in front of the mobile device; If it is determined that the displacement result is the third result indicating a downward displacement, it is determined that a second obstacle of a concave type exists in front of the mobile device.

3. The method according to claim 2, characterized in that After determining that there is a first obstacle of the protruding type in front of the mobile device, the method further includes: Acquire a first offset of the light spot upwardly along a height direction; A first distance between the mobile device and the first obstacle is determined based on the first offset and a triangle similarity rule.

4. The method according to claim 3, characterized in that The image acquisition device is a binocular camera, and the step of obtaining a first offset of the light spot shifting upward in the height direction includes: Determine a first offset of the light spot upward in the height direction at the first obstacle position based on a ranging algorithm of the binocular camera; The determining, based on the first offset and a triangle similarity rule, a first distance between the mobile device and the first obstacle comprises: Determine a first distance between the mobile device and the first obstacle based on the first offset, initial calibration parameters and the triangle similarity law; Among them, the initial calibration parameters include at least: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device and a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state.

5. The method according to claim 2, characterized in that: After determining that there is a second obstacle of the concave type in front of the mobile device, the method further includes: Acquire a second offset of the light spot downwardly along a height direction; Based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a recess depth corresponding to the second obstacle is determined.

6. The method according to claim 5, characterized in that The image acquisition device is a binocular camera, and the step of obtaining a second offset of the light spot shifting downward in the height direction includes: In response to the binocular camera capturing a second light spot image of the light spot corresponding to the second obstacle, determining, based on a ranging algorithm of the binocular camera, a second offset amount by which the light spot is offset downward in a height direction at a position where the initial light rays intersect the horizontal ground; The determining, based on the second offset and the triangle similarity law, a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle includes: Determine a second distance between the mobile device and the second obstacle and / or a depression depth corresponding to the second obstacle based on the second offset, the initial calibration parameters and the triangle similarity law; Among them, the initial calibration parameters at least include: a first calibration parameter characterizing the distance between the light emitter and the image acquisition device, a second calibration parameter characterizing the horizontal emission distance of the initial light emitted by the light emitter on the mobile device in the initial state, and a third calibration parameter characterizing the height of the image acquisition device.

7. The method according to claim 3 or 5, characterized in that: The method further comprises: The operating state of the mobile device is controlled based on the first distance or based on at least one of the second distance and the recess depth.

8. The method according to claim 1, characterized in that The light emitter is located above the image acquisition device, and a line between the light emitter and the image acquisition device is perpendicular to the horizontal ground. The method further includes: Obtaining the inclination angle of the mobile device during operation; The posture of the mobile device is adjusted based on the tilt angle so that the connecting line is perpendicular to the horizontal ground.

9. An obstacle avoidance device for a mobile device, characterized in that: The mobile device comprises a device body, a light emitter and an image acquisition device, wherein the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit initial light, the image acquisition device is used to acquire a light spot image of a light spot formed by the initial light, and the obstacle avoidance device comprises: A first acquisition module, used to acquire a first light spot image of the mobile device in an initial state, and determine a first pixel point position of the first light spot image; A second acquisition module, used to acquire a second light spot image during the operation of the mobile device, and determine a second pixel point position of the second light spot image; A processing module, configured to determine a shift result of whether the light spot shifts along a height direction based on the first pixel point position and the second pixel point position; The determination module is used to determine whether there is an obstacle in front of the mobile device based on the offset result.

10. A mobile device, characterized in that: The mobile device comprises a device body, a light emitter and an image acquisition device, wherein the light emitter and the image acquisition device are arranged on the device body, the light emitter is used to emit initial light, and the image acquisition device is used to acquire a light spot image of a light spot formed by the initial light, and the mobile device further comprises: a processor and a memory for storing a computer program that can be run on the processor, wherein: The processor is used to execute the steps of the method according to any one of claims 1 to 8 when running a computer program.

11. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images