Obstacle avoidance method, device, drone and electronic equipment

Abstract

The present invention discloses an obstacle avoidance method, device, drone and electronic device. The method comprises: during the movement of the target device, detecting the target obstacle in the target area, wherein the target obstacle is an obstacle in a moving state; determining the moving trajectory of the target obstacle; and when it is determined that the moving trajectory overlaps with at least a part of the area in the safety area, and the distance between the target obstacle and the target device is not greater than the preset distance, controlling the target device to move in a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is the direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle. The present invention solves the technical problem that drones are easily damaged due to the inability of drones in related technologies to efficiently identify and avoid dynamic obstacles.

Description

Technical Field

[0001] The present invention relates to the field of automatic control, and in particular to an obstacle avoidance method, device, unmanned aerial vehicle and electronic equipment. Background Art

[0002] At present, the drones in the relevant technology have good recognition and avoidance capabilities for static obstacles when performing obstacle avoidance, but they cannot accurately avoid dynamic obstacles, especially when the drone is moving at high speed, which makes the drone easily damaged by the dynamic obstacles during movement.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide an obstacle avoidance method, device, drone and electronic equipment to at least solve the technical problem that the drone is easily damaged due to the inability of the drone in the related art to efficiently identify and avoid dynamic obstacles.

[0005] According to one aspect of an embodiment of the present invention, there is provided an obstacle avoidance method, comprising: during movement of a target device, detecting a target obstacle in a target area, wherein the target obstacle is an obstacle in a moving state; determining a moving trajectory of the target obstacle; and when it is determined that the moving trajectory overlaps with at least a portion of an area in a safety area and the distance between the target obstacle and the target device is not greater than a preset distance, controlling the target device to move in a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to a shortest avoidance time, and the avoidance time is a time consumed when the target device moves to a distance greater than the preset distance from the target obstacle.

[0006] Optionally, detecting a target obstacle in a target area includes: obtaining a light intensity parameter in the target area; when a change rate of the light intensity parameter is greater than a preset change rate, obtaining event information generated by a dynamic vision camera in a target device within a preset time period and a grayscale image corresponding to the event information, wherein the event information includes an event image; determining an obstacle area corresponding to the target obstacle and position information of the obstacle area in the grayscale image, wherein the obstacle area is a rectangular area containing the target obstacle; and, based on the event image, determining a first obstacle event image that only contains the event information corresponding to the target obstacle.

[0007] Optionally, the event image is composed of event points, and each pixel point in the event image corresponds to a timestamp. Based on the event image, determining a first obstacle event image that only contains time information corresponding to the target obstacle includes: determining an average timestamp of event points in the event image; determining a timestamp threshold; filtering event points in the event image based on the timestamp threshold and the average timestamp; and obtaining a first obstacle event image corresponding to the target obstacle based on the filtering result, wherein the first obstacle event image only contains event points corresponding to the target obstacle.

[0008] Optionally, screening the event points in the event image based on the timestamp threshold and the average timestamp includes: determining the event points whose average timestamps are greater than the timestamp threshold as the event points corresponding to the target obstacles.

[0009] Optionally, determining the moving trajectory of the target obstacle includes: determining multiple coordinate values ​​corresponding to the target obstacle in the world coordinate system based on the first obstacle event image; determining a change in the coordinate values ​​of the target obstacle in the world coordinate system based on the multiple coordinate values; determining a moving speed and a moving direction of the target obstacle based on the change; and determining a moving trajectory of the target obstacle within a target time period based on the multiple coordinate values, the moving speed and the moving direction, wherein the target time period is a time period starting from an end point of a preset time period.

[0010] Optionally, determining multiple coordinate values ​​corresponding to the target obstacle in the world coordinate system based on the first obstacle event image includes: clustering the first obstacle event image to obtain a second obstacle event image; determining a rotation matrix corresponding to the target device, and determining corner points and a center point of the rotation matrix based on the second obstacle event image; determining a depth value of the target obstacle in the camera reference system based on the corner points and the center point; and determining multiple coordinate values ​​of the target obstacle in the world coordinate system based on the depth value.

[0011] Optionally, determining the depth value of the target obstacle in the camera reference system based on the corner points and the center point includes: determining the size information of the target obstacle, the side length information of the obstacle area, and the focal length of the dynamic vision camera; determining the depth value of the target obstacle in the camera reference system based on the size information, side length information, focal length, corner points and center points.

[0012] Optionally, determining the rotation matrix corresponding to the target device includes: acquiring posture data of the target device; determining an average angular velocity of the target device based on the posture data; and establishing a rotation matrix according to the average angular velocity.

[0013] According to another aspect of an embodiment of the present invention, a drone is also provided. The drone includes a dynamic vision module, a processor, a flight control module, and an inertial measurement module, wherein the dynamic vision module is used to detect target obstacles in a target area during the movement of the drone, wherein the target obstacle is an obstacle in a moving state; the inertial measurement module is used to obtain the posture data of the drone; the processor is used to determine the moving trajectory of the target obstacle; the flight control module is used to determine whether the moving trajectory overlaps with a safe area, wherein the safe area is an area determined with the target device as the center; when it is determined that the moving trajectory overlaps with the safe area and the distance between the drone and the target device is not greater than a preset distance, the drone is controlled to move in a first direction, wherein the first direction is the direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the drone.

[0014] According to another aspect of an embodiment of the present invention, an obstacle avoidance device is also provided, including: a detection module, used to detect a target obstacle in a target area during the movement of a target device, wherein the target obstacle is an obstacle in a moving state; a calculation module, used to determine the moving trajectory of the target obstacle; and a processing module, used to control the target device to move along a first direction when it is determined that the moving trajectory overlaps with a safety area and the distance between the target obstacle and the target device is not greater than a preset distance, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle.

[0015] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is further provided. The non-volatile storage medium includes a stored program, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute an obstacle avoidance method.

[0016] According to another aspect of an embodiment of the present invention, an electronic device is further provided. The electronic device includes a processor, and the processor is used to run a program, wherein the obstacle avoidance method is executed when the program is run.

[0017] In an embodiment of the present invention, a target obstacle in a target area is detected during the movement of a target device, wherein the target obstacle is an obstacle in a moving state; a moving trajectory of the target obstacle is determined; and when it is determined that the moving trajectory overlaps with at least a portion of an area in a safety area and the distance between the target obstacle and the target device is not greater than a preset distance, the target device is controlled to move in a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to a shortest avoidance time, and the avoidance time is a time consumed when the target device moves to a distance between the target obstacle and the target obstacle greater than a preset distance. By determining the moving trajectory of the obstacle, the purpose of correcting the moving trajectory of the target device based on the moving trajectory of the obstacle is achieved, thereby realizing the technical effect of the drone identifying and avoiding dynamic obstacles during movement, thereby solving the technical problem of drone being easily damaged due to the inability of the drone in the related art to efficiently identify and avoid dynamic obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a flowchart of an obstacle avoidance method provided according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of the structure of a drone provided according to an embodiment of the present invention;

[0021] Figure 3 is a schematic structural diagram of an obstacle avoidance device provided according to an embodiment of the present invention;

[0022] Figure 4 It is a structural schematic diagram of a computer device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Example 1

[0026] According to an embodiment of the present invention, a method embodiment of an obstacle avoidance method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] Figure 1 is an obstacle avoidance method according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0028] Step S102, during the movement of the target device, detecting a target obstacle in the target area, wherein the target obstacle is an obstacle in a moving state;

[0029] As an optional implementation, a specific process for detecting a target obstacle in a target area includes the following steps: obtaining a light intensity parameter in the target area; when a change rate of the light intensity parameter is greater than a preset change rate, obtaining event information generated by a dynamic vision camera in the target device within a preset time period and a grayscale image corresponding to the event information, wherein the event information includes an event image; determining an obstacle area corresponding to the target obstacle and position information of the obstacle area in the grayscale image, wherein the obstacle area is a rectangular area containing the target obstacle; and, based on the event image, determining a first obstacle event image that only contains the event information corresponding to the target obstacle.

[0030] Specifically, in some embodiments of the present application, when a high-speed moving obstacle appears in the scene where the drone is located or the drone moves by itself, the light intensity in the scene will change, causing the dynamic vision camera to generate an event. In addition, the above-mentioned event image can be a four-channel event image, wherein the first two channels of the event image are used to encode the quantity information of positive and negative polarity events, and the last two channels are used to encode the timestamp information of positive and negative polarity events. It should be noted that in the present application, a positive polarity event refers to an event in which the light intensity increases within a preset time period, and a negative polarity event refers to an event in which the light intensity decreases within a preset time period.

[0031] In some embodiments of the present application, the event image is composed of event points, and each pixel in the event image corresponds to a timestamp. Based on the event image, determining a first obstacle event image that only contains time information corresponding to the target obstacle includes: determining an average timestamp of event points in the event image; determining a timestamp threshold; based on the timestamp threshold and the average timestamp, filtering the event points in the event image; and obtaining the first obstacle event image corresponding to the target obstacle based on the filtering result, wherein the first obstacle event image only contains event points corresponding to the target obstacle.

[0032] Specifically, based on the timestamp threshold and the average timestamp, screening the event points in the event image includes: determining the event points whose average timestamps are greater than the timestamp threshold as the event points corresponding to the target obstacle. In addition, by thresholding the average timestamp, an event image containing only dynamic obstacle event points can be obtained.

[0033] In some embodiments of the present application, when there are potential dynamic obstacles in the scene, the angular velocities on the drone's IMU (inertial measurement unit) can be averaged, the Rodrigues rotation algorithm can be applied to establish a rotation matrix, motion compensation can be performed, and a normalized average timestamp can be calculated for the pixels on the event image.

[0034] As an optional implementation method, after obtaining the above-mentioned grayscale image, the YOLO model detection algorithm can be used to detect whether there are potential dynamic obstacles in the grayscale image. When a potential dynamic obstacle is detected, the size and position information of the bounding box of the dynamic obstacle can be output.

[0035] Step S104, determining the moving trajectory of the target obstacle;

[0036] In some embodiments of the present application, the moving trajectory of the target obstacle can be determined in the following manner, specifically including: determining a plurality of coordinate values ​​corresponding to the target obstacle in the world coordinate system based on the first obstacle event image; determining a change in the coordinate values ​​of the target obstacle in the world coordinate system based on the plurality of coordinate values; determining a moving speed and a moving direction of the target obstacle based on the change; determining the moving trajectory of the target obstacle within a target time period based on the plurality of coordinate values, the moving speed and the moving direction, wherein the target time period is a time period starting from the end point of the preset time period.

[0037] Specifically, during clustering, the eight-way connected component clustering algorithm can be used to pre-cluster event points, and then the DBSCAN density clustering algorithm can be used for further clustering. The newly constructed clustering cost function is the Euclidean distance and the optical flow estimation result calculated by the Lucas-Kanade algorithm. After obtaining the clustering result, a matrix can be drawn around the result to obtain information such as the matrix corner points and the matrix center position.

[0038] As an optional implementation, determining multiple coordinate values ​​corresponding to the target obstacle in the world coordinate system based on the first obstacle event image includes: clustering the first obstacle event image to obtain a second obstacle event image; determining a rotation matrix corresponding to the target device, and determining the corner points and center points of the rotation matrix based on the second obstacle event image; determining a depth value of the target obstacle in the camera reference system based on the corner points and the center point; and determining multiple coordinate values ​​of the target obstacle in the world coordinate system based on the depth value.

[0039] The rotation matrix is ​​a minimum rectangle that can frame the clustering result. The center of the diagonal of the matrix is ​​determined by the center of the clustering point, and the corner points are the four corners of the matrix.

[0040] As an optional implementation, determining the depth value of the target obstacle in the camera reference system based on the corner point and the center point includes: determining the size information of the target obstacle, the side length information of the obstacle area, and the focal length of the dynamic vision camera; and determining the depth value of the target obstacle in the camera reference system based on the size information, the side length information, the focal length, the corner point and the center point.

[0041] Specifically, the known size of the dynamic obstacle (radius of the ball) and the focal length of the event camera can be used as prior conditions, combined with the coordinates of the corner points and center points obtained by clustering, to estimate the depth value of the dynamic obstacle in the camera reference system. As shown in the following formula:

[0042]

[0043] Where f is the focal length, ω real is the width of the object, is the measured side length of the fitted rectangle.

[0044] In some embodiments of the present application, the rotation matrix corresponding to the target device can be determined in the following manner: acquiring the posture data of the target device; determining the average angular velocity of the target device based on the posture data; and establishing the rotation matrix according to the average angular velocity.

[0045] In some embodiments of the present application, after determining the depth and size of the obstacle, the projection model in the homogeneous coordinates can be used to project the corner points and center points obtained by clustering into the 3D space to obtain the three-dimensional coordinates of the obstacle in the world coordinate system. The conversion formula between the pixel coordinate system and the world coordinate system (the conversion from two-dimensional to three-dimensional coordinates is completed by this formula):

[0046]

[0047] The homogeneous coordinate system refers to a coordinate system that always takes the camera as the origin. w ,Y w , Z w is the physical coordinate of a point in the world coordinate system, Z w , v is the pixel coordinate of the point in the pixel coordinate system, Z c is the scale factor. are the internal parameters of the camera. The internal parameter matrix depends on the internal parameters of the camera, where f is the image distance, dx and dy represent the physical length of a pixel on the camera's photosensitive plate in the X and Y directions respectively (that is, how many millimeters a pixel is on the photosensitive plate), u0 and v0 represent the coordinates of the center of the camera's photosensitive plate in the pixel coordinate system, and 0 represents the angle between the horizontal and vertical edges of the photosensitive plate (90 degrees means no error). are the external parameters of the camera. The external parameter matrix depends on the relative position of the camera coordinate system and the world coordinate system. R represents the rotation matrix and T represents the translation vector.

[0048] After obtaining the three-dimensional coordinates of the dynamic obstacle through the above formula, the instantaneous speed and direction of the obstacle are estimated by the coordinate change of the obstacle within the time window (that is, within the preset time period). The influence of noise is reduced by the mean filtering method to obtain the average speed and direction of the dynamic obstacle.

[0049] Step S106, when it is determined that the moving trajectory overlaps with at least a portion of the safe area and the distance between the target obstacle and the target device is not greater than a preset distance, control the target device to move in a first direction, wherein the safe area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle.

[0050] Specifically, after determining the moving trajectory of the obstacle, a safe range can be defined with the drone as the center. The safe range is a rectangular frame. The size of the rectangular frame is obtained based on the later parameter adjustment. At the same time, a safe distance is set for starting obstacle avoidance. Obstacle avoidance will only begin when the dynamic obstacle is within the safe distance from the drone.

[0051] As an optional implementation, since the preset time length is short, the movement of the dynamic obstacle can be temporarily regarded as uniform linear motion, and based on the speed and direction of the dynamic obstacle obtained previously, as well as the geometric relationship, it can be calculated whether it will intersect with the rectangular frame.

[0052] When a possible collision is detected, the drone's flight control device can send an obstacle avoidance signal to enable the drone to avoid the obstacle in the direction with the fastest avoidance time (up, down, and right, three directions for obstacle avoidance).

[0053] Through the above steps, it is possible to detect the target obstacle in the target area during the movement of the target device, wherein the target obstacle is an obstacle in a moving state; determine the moving trajectory of the target obstacle; and control the target device to move along the first direction when it is determined that the moving trajectory overlaps with at least a part of the area in the safety area and the distance between the target obstacle and the target device is not greater than the preset distance, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance between the target obstacle and the target obstacle greater than the preset distance. By determining the moving trajectory of the obstacle, the purpose of correcting the moving trajectory of the target device based on the moving trajectory of the obstacle is achieved, thereby achieving the technical effect of the drone identifying and avoiding dynamic obstacles during movement, and further solving the technical problem of drone being easily damaged due to the inability of the drone in the related technology to efficiently identify and avoid dynamic obstacles.

[0054] Example 2

[0055] According to an embodiment of the present invention, a drone is provided. Figure 2 is a schematic diagram of the structure of a drone provided according to an embodiment of the present invention. Figure 2As shown, the UAV includes: a dynamic vision module 20, a processor 22, a flight control module 24, and an inertial measurement module 26, wherein the dynamic vision module 20 is used to detect target obstacles in the target area during the movement of the UAV, wherein the target obstacles are obstacles in a moving state; the inertial measurement module 26 is used to obtain the posture data of the UAV; the processor 22 is used to determine the moving trajectory of the target obstacle; the flight control module 24 is used to determine whether the moving trajectory overlaps with the safety area, wherein the safety area is an area determined with the target device as the center; when it is determined that the moving trajectory overlaps with the safety area and the distance between the UAV and the target device is not greater than the preset distance, the UAV is controlled to move in a first direction, wherein the first direction is the direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the UAV.

[0056] In some embodiments of the present application, the above-mentioned drone also includes a TOF ranging module.

[0057] In some embodiments of the present application, the processor 22 is also used to obtain the attitude data of the drone from the inertial measurement module 26, and calculate the coordinate changes of the obstacle based on the attitude data, and use optical flow calculation to perform target segmentation on the event image, so as to obtain an event image containing only the obstacle, and send the position information and speed information of the obstacle to the flight control module 24.

[0058] It should be noted that the drone provided in this embodiment can be used to execute the obstacle avoidance method provided in Example 1. Therefore, the relevant explanations and descriptions of the obstacle avoidance method in Example 1 are also applicable to the drone in this embodiment and will not be repeated here.

[0059] Example 3

[0060] According to an embodiment of the present invention, an obstacle avoidance device is provided. Figure 3 is a schematic diagram of the structure of an obstacle avoidance device provided according to an embodiment of the present invention. Figure 3 As shown, the obstacle avoidance device includes: a detection module 30, used to detect a target obstacle in a target area during the movement of the target device, wherein the target obstacle is an obstacle in a moving state; a calculation module 32, used to determine the movement trajectory of the target obstacle; a processing module 34, used to control the target device to move along a first direction when it is determined that the movement trajectory overlaps with a safety area and the distance between the target obstacle and the target device is not greater than a preset distance, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle.

[0061] It should be noted that the obstacle avoidance device provided in this embodiment can be used to execute the obstacle avoidance method provided in Example 1. Therefore, the relevant explanations and descriptions of the obstacle avoidance method shown in Example 1 are also applicable to the embodiments of the present application and will not be repeated here.

[0062] According to an embodiment of the present invention, a non-volatile storage medium is also provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to perform the following obstacle avoidance method: during the movement of the target device, a target obstacle in the target area is detected, wherein the target obstacle is an obstacle in a moving state; the moving trajectory of the target obstacle is determined; when it is determined that the moving trajectory overlaps with at least part of the area in the safety area, and the distance between the target obstacle and the target device is not greater than the preset distance, the target device is controlled to move in a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is the direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle.

[0063] According to an embodiment of the present invention, an electronic device is also provided. The electronic device includes a processor, and the processor is used to run a program, wherein the program runs to execute the following obstacle avoidance method: during the movement of the target device, a target obstacle in a target area is detected, wherein the target obstacle is an obstacle in a moving state; the moving trajectory of the target obstacle is determined; when it is determined that the moving trajectory overlaps with at least a part of the area in the safety area, and the distance between the target obstacle and the target device is not greater than a preset distance, the target device is controlled to move in a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle.

[0064] According to an embodiment of the present invention, an embodiment of a computer terminal is also provided. Figure 4 FIG. 4 is a schematic diagram showing the structure of a computer device 400 according to an embodiment of the present invention.

[0065] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by the processor 402 of the device 400 to complete the following obstacle avoidance method: during the movement of the target device, detecting a target obstacle in the target area, wherein the target obstacle is an obstacle in a moving state; determining the movement trajectory of the target obstacle; and when it is determined that the movement trajectory overlaps with at least a part of the area in the safety area, and the distance between the target obstacle and the target device is not greater than a preset distance, controlling the target device to move along a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves to a distance greater than the preset distance from the target obstacle. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, an optical data storage device, etc.

[0066] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0067] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0069] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0070] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0071] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0072] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An obstacle avoidance method, characterized in that: include: During the movement of the target device, detecting a target obstacle in the target area, wherein the target obstacle is an obstacle in a moving state; Target obstacles within the detection target area include: Acquire a light intensity parameter in the target area; when the change rate of the light intensity parameter is greater than a preset change rate, acquire event information generated by a dynamic visual camera in the target device within a preset time period and a grayscale image corresponding to the event information, wherein the event information includes an event image, the event image is composed of event points, and each pixel in the event image corresponds to a timestamp; determine an obstacle area corresponding to the target obstacle and position information of the obstacle area in the grayscale image, wherein the obstacle area is a rectangular area containing the target obstacle; and, based on the event image, determine a first obstacle event image that only contains the event information corresponding to the target obstacle; Determining, based on the event image, a first obstacle event image that only includes time information corresponding to the target obstacle includes: determining an average timestamp of event points in the event image; determining a timestamp threshold; filtering the event points in the event image based on the timestamp threshold and the average timestamp; obtaining the first obstacle event image corresponding to the target obstacle according to the filtering result, wherein the first obstacle event image only includes event points corresponding to the target obstacle; Based on the timestamp threshold and the average timestamp, screening the event points in the event image includes: determining the event points whose average timestamps are greater than the timestamp threshold as the event points corresponding to the target obstacles; Determining a moving trajectory of the target obstacle; When it is determined that the movement trajectory overlaps with at least a portion of the safety area and the distance between the target obstacle and the target device is not greater than a preset distance, the target device is controlled to move in a first direction, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is the time consumed when the target device moves until the distance between it and the target obstacle is greater than the preset distance.

2. The obstacle avoidance method according to claim 1, characterized in that: Determining the moving trajectory of the target obstacle includes: Determine, based on the first obstacle event image, a plurality of coordinate values ​​corresponding to the target obstacle in a world coordinate system; Based on the multiple coordinate values, determining a change in the coordinate value of the target obstacle in the world coordinate system; Based on the change, determine the moving speed and moving direction of the target obstacle; Based on the multiple coordinate values, the moving speed and the moving direction, the moving trajectory of the target obstacle within a target time period is determined, wherein the target time period is a time period starting from the end point of the preset time period.

3. The obstacle avoidance method according to claim 2, characterized in that: Determining a plurality of coordinate values ​​corresponding to the target obstacle in the world coordinate system based on the first obstacle event image includes: Clustering the first obstacle event image to obtain a second obstacle event image; Determine a rotation matrix corresponding to the target device, and determine corner points and a center point of the rotation matrix according to the second obstacle event image; Determine the depth value of the target obstacle in the camera reference system according to the corner point and the center point; A plurality of coordinate values ​​of the target obstacle in the world coordinate system are determined according to the depth value.

4. The obstacle avoidance method according to claim 3, characterized in that: Determining the depth value of the target obstacle in the camera reference system according to the corner point and the center point includes: Determining the size information of the target obstacle, the side length information of the obstacle area, and the focal length of the dynamic vision camera; The depth value of the target obstacle in the camera reference system is determined according to the size information, the side length information, the focal length, the corner point and the center point.

5. The obstacle avoidance method according to claim 4, characterized in that: Determining the rotation matrix corresponding to the target device includes: Acquire the position and posture data of the target device; Based on the posture data, determining an average angular velocity of the target device; The rotation matrix is ​​established according to the average angular velocity.

6. A drone, applicable to the obstacle avoidance method according to any one of claims 1 to 5, the drone comprising a dynamic vision module, a processor, a flight control module, and an inertial measurement module, wherein: The dynamic vision module is used to detect target obstacles in a target area during the movement of the drone, wherein the target obstacles are obstacles in a moving state; The inertial measurement module is used to obtain the position and posture data of the drone; The processor is used to determine the moving trajectory of the target obstacle; The flight control module is used to determine whether the moving trajectory overlaps with a safe area, wherein the safe area is an area determined with a target device as the center; when it is determined that the moving trajectory overlaps with the safe area and the distance between the drone and the target device is not greater than a preset distance, control the drone to move in a first direction, wherein the first direction is a direction corresponding to a shortest avoidance time, and the avoidance time is a time consumed when the target device moves to a distance greater than the preset distance from the drone.

7. An obstacle avoidance device, characterized in that: include: A detection module, used to detect a target obstacle in a target area during movement of the target device, wherein the target obstacle is an obstacle in a moving state; Target obstacles within the detection target area include: Acquire a light intensity parameter in the target area; when the change rate of the light intensity parameter is greater than a preset change rate, acquire event information generated by a dynamic visual camera in the target device within a preset time period and a grayscale image corresponding to the event information, wherein the event information includes an event image, the event image is composed of event points, and each pixel in the event image corresponds to a timestamp; determine an obstacle area corresponding to the target obstacle and position information of the obstacle area in the grayscale image, wherein the obstacle area is a rectangular area containing the target obstacle; and, based on the event image, determine a first obstacle event image that only contains the event information corresponding to the target obstacle; Determining, based on the event image, a first obstacle event image that only includes time information corresponding to the target obstacle includes: determining an average timestamp of event points in the event image; determining a timestamp threshold; filtering the event points in the event image based on the timestamp threshold and the average timestamp; obtaining the first obstacle event image corresponding to the target obstacle according to the filtering result, wherein the first obstacle event image only includes event points corresponding to the target obstacle; Based on the timestamp threshold and the average timestamp, screening the event points in the event image includes: determining the event points whose average timestamps are greater than the timestamp threshold as the event points corresponding to the target obstacles; A calculation module, used to determine the moving trajectory of the target obstacle; A processing module, used to control the target device to move along a first direction when it is determined that the movement trajectory overlaps with a safety area and the distance between the target obstacle and the target device is not greater than a preset distance, wherein the safety area is an area determined with the target device as the center, the first direction is a direction corresponding to the shortest avoidance time, and the avoidance time is a time consumed when the target device moves to a distance between it and the target obstacle greater than the preset distance.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the obstacle avoidance method described in any one of claims 1 to 5.

9. An electronic device, comprising a processor, characterized in that: The processor is used to run a program, wherein the program executes the obstacle avoidance method described in any one of claims 1 to 5 when running.

Images