Map processing method and device, terminal and storage medium

Abstract

The embodiment of the invention provides a map processing method and device, a terminal and a storage medium. The map processing method comprises the following steps: acquiring a baseline length and a parallax corresponding to a planned path of a target object; determining depth information according to the baseline length and the parallax; determining point cloud data under a predetermined coordinate system according to the depth information; performing ellipsoid expansion processing on the point cloud data based on the length information of the target object in each direction to obtain obstacle point data; wherein the obstacle point data is in the ellipsoid; inputting the obstacle point data into an octree node for information updating so as to construct an octree map; wherein nodes corresponding to the obstacle point data in the octree map are represented by first indication information, and nodes corresponding to the non-obstacle point data are represented by second indication information. Therefore, according to the embodiment of the invention, expansion can be carried out according to the ellipsoid and the octree map can be constructed, so that the flight area of the target object can be maximized while maps with different resolutions can be determined.

Description

Technical Field

[0001] The present invention relates to the field of map construction technology, but is not limited to it, and in particular to a map processing method and device, a terminal and a storage medium. Background Art

[0002] In recent years, drones have been increasingly studied and / or applied in various industries. For example, drones can perform tasks such as target tracking, image stitching, power line inspection, island monitoring, coastline inspection, post-disaster monitoring, and / or river flood season monitoring. However, when drones complete tasks, they need to have the ability to fly autonomously, that is, in any complex environment, the drone can complete the task independently without external assistance. Among them, the ability to build maps is a key part of unmanned autonomous flight.

[0003] Currently, drones often construct maps by expanding a circle or a sphere, which results in a waste of flight area and poor performance of the planned path. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a map processing method and device, a terminal and a storage medium to solve the above technical problems.

[0005] The technical solution of the present invention is achieved in this way:

[0006] In a first aspect, an embodiment of the present invention provides a map processing method, comprising:

[0007] A map processing method, the method comprising:

[0008] Get the baseline length and disparity of the target object on the planned path;

[0009] Determining depth information according to the baseline length and the parallax;

[0010] Determining point cloud data in a predetermined coordinate system according to the depth information;

[0011] Based on the length information of the target object in each direction, the point cloud data is subjected to ellipsoid expansion processing to obtain obstacle point data; wherein the obstacle point data is within the ellipsoid;

[0012] The obstacle point data is input into the octree node for information update to construct an octree map; wherein the nodes corresponding to the obstacle point data in the octree map are represented by first indication information and the nodes corresponding to the non-obstacle point data are represented by second indication information.

[0013] In the above scheme, the method further includes:

[0014] Determining the positioning accuracy, maximum size and safety margin of the target object in each direction;

[0015] The length information of the target object in each direction is determined based on the positioning accuracy of the target object in each direction, the sum of the maximum size and the safety margin.

[0016] In the above solution, the step of performing ellipsoid expansion processing on the point cloud data based on the length information of the target object in each direction to obtain obstacle point data includes:

[0017] The length information and the point cloud data are input into an ellipsoid equation to obtain the obstacle point data, wherein the obstacle point data is within the ellipsoid indicated by the ellipsoid equation.

[0018] In the above solution, the first part of the ellipsoid equation is the sum of the first values ​​in various directions, and the second part of the ellipsoid equation is a predetermined value; the first part is less than or equal to the second part;

[0019] Among them, the first value is the ratio of the second value to the third value in the same direction; the second value is the square of the difference between the obstacle point data and the point cloud data in one direction among all directions, and the third value is the square of the length information in one direction among all directions.

[0020] In the above solution, determining the depth information according to the baseline length and the parallax includes:

[0021] Determine a first parameter based on the product of the baseline length and the camera focal length; wherein the camera focal length is the focal length of the binocular camera set for the target object;

[0022] The depth information is determined based on the ratio of the first parameter and the disparity.

[0023] In the above solution, the step of obtaining the baseline length and disparity of the target object corresponding to the planned path includes:

[0024] The first baseline length is determined based on a reflective device provided for the target object; the first baseline length is greater than a second baseline length, wherein the second baseline length is a baseline length determined when the target object is not provided with a reflective device and an image is collected;

[0025] A reflection device is set based on the target object, and a parallax determined by collecting images of the target object on a planned path is determined.

[0026] In a second aspect, an embodiment of the present invention provides a drone, including:

[0027] A binocular camera, wherein the binocular camera comprises two cameras;

[0028] The corresponding reflecting devices are respectively arranged at predetermined angles with the two cameras so that the distance between the focal points of the two cameras is the second baseline length and the distance between the two virtual focal points of the reflecting devices is the first baseline length; wherein the first baseline length is greater than the second baseline length.

[0029] In a third aspect, an embodiment of the present invention provides a map processing device, including:

[0030] An acquisition module is used to obtain the baseline length and parallax corresponding to the target object on the planned path;

[0031] A determination module, configured to determine depth information according to the baseline length and the parallax;

[0032] The determination module is further used to determine point cloud data in a predetermined coordinate system according to the depth information;

[0033] A processing module, configured to perform ellipsoid expansion processing on the point cloud data based on the length information of the target object in various directions to obtain obstacle point data; wherein the obstacle point data is within the ellipsoid;

[0034] The processing module is also used to input the obstacle point data into the octree node to update the information so as to construct an octree map; wherein the nodes corresponding to the obstacle point data in the octree map are represented by the first indication information and the nodes corresponding to the non-obstacle point data are represented by the second indication information.

[0035] In a fourth aspect, an embodiment of the present invention provides a terminal, comprising a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the map processing method described in any embodiment of the present invention is implemented.

[0036] In a fifth aspect, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium contains computer executable instructions, and the computer executable instructions are executed by a processor to implement the map processing method described in any embodiment of the present invention.

[0037] In an embodiment of the present invention, the baseline length and disparity corresponding to the target object in the planned path are obtained; the depth information is determined according to the baseline length and the disparity; the point cloud data in a predetermined coordinate system is determined according to the depth information; the point cloud data is expanded according to the length information of the target object in each direction to obtain obstacle point data, wherein the obstacle point data is within the ellipsoid; the obstacle point data is input into the octree node for information update to construct an octree map; wherein the node corresponding to the obstacle point data in the octree map is represented by the first indication information and the node corresponding to the non-obstacle point data is represented by the second indication information. In this way, the embodiment of the present invention can construct a map by expanding the ellipsoid, so that the points inside the ellipsoid are virtual obstacles. Compared with constructing a map by expanding a circle or a sphere to determine obstacles, the expanded map can be made more reasonable and the occurrence of excessive map expansion can be reduced; in this way, the path of the target object can be reasonably planned and the waste of the flight area of ​​the target object can be reduced. Furthermore, the present invention can also input obstacle point data (i.e., data expanded from cloud data) into the octree node to construct an octree map, so that maps of different resolutions can be determined while facilitating maximizing the flight area of ​​the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of a map processing method provided by an embodiment of the present invention.

[0039] Figure 2 A schematic diagram of an octree model is provided for an embodiment of the present invention.

[0040] Figure 3 A schematic diagram of installing a reflection device on a target object provided by an embodiment of the present invention.

[0041] Figure 4 A schematic diagram of a binocular camera depth calculation model provided by an embodiment of the present invention.

[0042] Figure 5 A schematic flow chart of another map processing method provided by an embodiment of the present invention.

[0043] Figure 6 A schematic diagram of a camera coordinate system provided by an embodiment of the present invention.

[0044] Figure 7 A schematic diagram of the structure of a map processing device provided by an embodiment of the present invention.

[0045] Figure 8 A schematic diagram of the hardware structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to indicate elements is only for the purpose of facilitating the description of the present invention, and has no specific meaning in itself. Therefore, "module", "component" or "unit" can be used in a mixed manner. In addition, in the subsequent description, the use of prefixes such as "first" or "second" to identify information is only for the purpose of facilitating the description of the present invention, and has no specific meaning in itself. In addition, in the subsequent description, "multiple" refers to two or more; "multiple" refers to two or more.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a map processing method, comprising the following steps:

[0049] Step S11: obtaining the baseline length and disparity corresponding to the target object on the planned path;

[0050] Step S12: determining depth information according to the baseline length and the parallax;

[0051] Step S13: determining point cloud data in a predetermined coordinate system according to the depth information;

[0052] Step S14: Based on the length information of the target object in each direction, the point cloud data is expanded into an ellipsoid to obtain obstacle point data; wherein the obstacle point data is within the ellipsoid;

[0053] Step S15: input the obstacle point data into the octree node to update the information to construct an octree map; wherein the nodes corresponding to the obstacle point data in the octree map are represented by the first indication information and the nodes corresponding to the non-obstacle point data are represented by the second indication information.

[0054] The map processing method provided in the embodiment of the present invention may be executed by a terminal; the terminal may be any mobile terminal or fixed terminal. For example, the terminal may be, but is not limited to, at least one of the following: a drone, a mobile communication device, a computer, a server, a medical device, an intelligent office device, an industrial device, and / or a wearable device, etc.

[0055] The map processing method provided in the embodiment of the present invention may be executed by a terminal that establishes a communication connection with the drone.

[0056] In some embodiments, the drone may be provided with a binocular camera; the binocular camera includes two cameras. Exemplarily, the binocular camera is provided on both sides of the drone, or the binocular camera may be two modules or components or cameras of the drone with image acquisition functions.

[0057] In some embodiments, the drone may be equipped with two additional reflective devices. The two reflective devices may be arranged on both sides of the binocular camera; one reflective device is arranged at a preset angle to one of the binocular cameras of a drone. Here, the binocular camera may receive visual information of the drone on the planned path through the reflective device to generate an image; the image is used to determine the parallax. The drone may obtain the baseline length based on the virtual focus of the reflective device.

[0058] In other embodiments, the drone may not be equipped with a reflector. Here, the binocular camera in the drone receives visual information to generate an image. The drone can obtain the baseline length based on the focus of the binocular camera.

[0059] In some embodiments, step S11 may be one of the following:

[0060] Obtaining a first baseline length and / or parallax corresponding to the planned path of the UAV when the reflective device is not added;

[0061] Obtaining a first baseline length and / or parallax corresponding to a planned path of the drone when a reflective device is added;

[0062] Obtaining a first baseline length from a terminal or a server;

[0063] The first baseline length is determined based on historical experience information.

[0064] In some embodiments, the target object may be a drone.

[0065] In other embodiments, the target object may be any flying device or aircraft, etc.

[0066] In some embodiments, the parallax may be: the parallax of two images captured by the drone through a binocular camera on the planned path. Alternatively, the parallax may be the parallax corresponding to each pixel in the image. Alternatively, the parallax may be the displacement of corresponding pixels in the two images.

[0067] In some embodiments, the depth information may be depth information corresponding to each pixel.

[0068] In some embodiments, the predetermined coordinate system may be a three-dimensional space coordinate system; for example, the predetermined coordinate system may be an XYZ coordinate system.

[0069] Of course, the predetermined coordinate system may also be any other coordinate system that can represent a position space or a map, etc. For example, the predetermined coordinate system may be a camera coordinate system, an inertial coordinate system, a body coordinate system, or a pixel coordinate system.

[0070] In some embodiments, the point cloud data is used to describe the depth information mapped to the coordinate points in the predetermined coordinate system. One point cloud data corresponds to one pixel point; the point cloud data in step S13 may be the point cloud data corresponding to each pixel point.

[0071] In some embodiments, the length information of the target object in various directions may be: the length information of the target object in various directions or axial directions of a predetermined coordinate system, etc. For example, the length information of the target object in the directions of the X-axis, Y-axis and Z-axis of a three-dimensional space coordinate system.

[0072] In some embodiments, the length information includes: positioning accuracy, maximum size and / or safety margin.

[0073] Optionally, the length information of the target object in each direction may include: the maximum size of the target object in each direction; or, the length information of the target object in each direction may include: the maximum size and positioning accuracy of the target object in each direction; or, the length information of the target object in each direction may include: the maximum size and safety margin of the target object in each direction; or, the length information of the target object in each direction may include: the maximum size, positioning accuracy and safety margin of the target object in each direction; and so on.

[0074] Optionally, the maximum size of the target object in each direction refers to the maximum length of the target object (e.g., drone) body in each direction. The positioning accuracy of the target object in each direction may refer to the error range of the target object (e.g., drone) body in each direction, etc. The safety margin of the target object in each direction may refer to the amount reserved for the safety indicators of the target object (e.g., drone) body in each direction.

[0075] In some embodiments, the obstacle point data may be virtual point cloud data.

[0076] In some embodiments, step S14 includes: constructing an ellipsoid equation based on the length information and the point cloud data. Here, the coordinate point data within the ellipse can be determined as obstacle point data or virtual point cloud data, and the coordinate point data outside the ellipse can be determined as non-obstruction point data or non-virtual point cloud data.

[0077] Thus, in the embodiment of the present invention, the map can be constructed by expanding the ellipsoid so that the points inside the ellipsoid are virtual obstacles. Compared with constructing the map by expanding a circle or a sphere to determine the obstacles, the expanded map can be made more reasonable and the occurrence of excessive expansion of the map can be reduced. In this way, the path of the target object can be reasonably planned and the waste of the flight area of ​​the target object can be reduced.

[0078] Of course, in other embodiments, the map in step S15 may be any three-dimensional space map including obstacle point data.

[0079] In some embodiments, both the first indication information and the second indication information may be one or more bits. Exemplarily, the first indication information is "1" and the second indication information is "0". Exemplarily, the first indication information is used to represent blank or unoccupied, that is, the first indication information can be used to represent that the area corresponding to the node is a fault area; the second indication is used to represent that it is occupied, that is, the second indication information can be used to represent that the area corresponding to the node is a non-fault area.

[0080] In some embodiments, the octree map includes a root node and at least one layer of leaf nodes; one root node corresponds to eight leaf nodes in the first layer, and one leaf node in the previous layer corresponds to eight leaf nodes in the next layer. The above-mentioned octree node or nodes may include a root node and a leaf node. The octree node can represent a map; the leaf node in the lowest layer indicates the smallest area and the resolution of the map.

[0081] For example, Figure 2 As shown, an octree model is disclosed; the octree model can be used to construct an octree map; the octree model includes a root node, a first-layer leaf node, a second-layer leaf node, ... and a bottom-layer leaf node. The octree map is a flexible, compressed map that can be updated at any time. Here, the octree map can be a large cube that is continuously and evenly divided into eight blocks until it becomes the smallest block. The entire large block can be regarded as a root node, and the smallest block can be regarded as a "bottom-layer leaf node" or "leaf node". In the octree map, when going up one layer of leaf nodes from the next layer of leaf nodes, the volume of the map can be expanded eight times. When all the leaf nodes of a block are occupied or unoccupied, there is no need to expand the upper layer of nodes corresponding to this leaf node (including the upper layer of leaf nodes or the root node). For example, the octree node stores information on whether it is occupied. "0" can be used to indicate blank, and "1" can be used to indicate occupation; the point cloud data of each pixel point obtained in step S13 can be substituted into the octree node to update the occupation information, and the obstacle point data (such as virtual point cloud data) is 1 to obtain the expanded octree map.

[0082] Thus, in an embodiment of the present invention, the obstacle point data (i.e., virtual point cloud data after the point cloud data is expanded) can be used to construct an octree map; thus, maps of different resolutions can be determined based on the characteristics of the octree nodes, thereby facilitating maximizing the flight area of ​​the target object (e.g., a drone) while determining maps of different resolutions.

[0083] In some embodiments, step S11 includes:

[0084] Based on a reflective device provided on the target object, a first baseline length is determined; the first baseline length is greater than a second baseline length, wherein the second baseline length is a baseline length determined when the target object is not provided with a reflective device and an image is collected;

[0085] A reflection device is set based on the target object, and a parallax determined by collecting images of the target object on a planned path is determined.

[0086] For example, Figure 3 As shown, fixed reflective devices are installed on both sides of the binocular camera; for example, reflective devices 1 and 2 are installed on both sides of camera 1 and camera 2; reflective device 1 is at a predetermined angle with camera 1, and reflective device 2 is at a predetermined angle with camera 2; camera 1 and camera 2 receive visual information of the external environment through corresponding reflective devices to form an image. The distance between the focal points of camera 1 and camera 2 is the second baseline length; the distance between reflective device 1 and transmitting device 2 is the first baseline length. Optionally, the focus can be the focal center, and the virtual focus can be the virtual focal center. Optionally, the first baseline length can be a virtual baseline; the second baseline length can be a baseline.

[0087] Exemplarily, the predetermined angle may be 90 degrees. Of course, the predetermined angle may be any other angle, for example, the predetermined angle may be 110 degrees or 120 degrees, as long as the distance between the two reflective devices is greater than the distance between the two cameras.

[0088] For example, Figure 4 As shown in FIG. 1 , assume that P is a point of the target object on the planned path; the imaging points of point P on the two camera sensors are p and p′ respectively; the terminal can determine that P is X based on the parallax of the two images. R -X T ; Among them, X R , X T is the displacement of P in the left and right images collected by the binocular camera. Here, the terminal can determine the disparity corresponding to each pixel point in the image collected on the planned path based on the above method.

[0089] In an embodiment of the present invention, the first baseline length can be determined based on a reflective device set on the target object; this can indirectly increase the baseline length of the binocular camera, thereby increasing the ranging range of the binocular camera, making the acquired point cloud data more extensive, which is conducive to determining a more reasonable flight area.

[0090] Furthermore, the reflective devices fixed on both sides of the binocular camera are not added with a servo system, so that the cost of adding the reflective devices is relatively low and the importance of adding the reflective devices can be within an acceptable range, which facilitates the flight of the target object.

[0091] Of course, in other embodiments, the baseline length obtained in step S11 may also be the second baseline length. In this way, the baseline length actually corresponding to the focus of the binocular camera can be obtained without adding a reflection device.

[0092] In the embodiment of the present invention, the target object may continuously pass through two cameras on the planned path to collect images, and the parallax of each pixel point on each of the two corresponding images may be obtained.

[0093] In some embodiments, step S12 includes:

[0094] Determine a first parameter based on the product of the baseline length and the camera focal length; wherein the camera focal length is the focal length of the binocular camera set for the target object;

[0095] The depth information is determined based on the ratio of the first parameter and the disparity.

[0096] For example, Figure 4 As shown, if the terminal determines that the focal length of the binocular camera is f and the baseline length of the binocular camera is B; according to the principle of similar triangles Depth information available Here, the depth information is positively correlated with the baseline length.

[0097] In the embodiment of the present invention, the focal length (f) and / or the baseline length (B) of the camera can be obtained by calibration; the parallax (X R -X T ) can be obtained through the binocular stereo matching algorithm. It can be seen from the above formula of depth information (Z) that when the baseline length increases, the depth information of the binocular camera increases, that is, the binocular camera can perceive the surrounding environment farther away, making the obtained point cloud data richer.

[0098] like Figure 5 As shown, in some embodiments, the method further includes:

[0099] Step S16: determining the positioning accuracy, maximum size and safety margin of the target object in each direction;

[0100] Step S17: Determine the length information of the target object in each direction based on the positioning accuracy of the target object in each direction, the sum of the maximum size and the safety margin.

[0101] In some embodiments, step S17 precedes step S14.

[0102] In some embodiments, the directions in step S16 and step S17 refer to directions on a predetermined coordinate system.

[0103] Exemplarily, the predetermined coordinate system may be a camera coordinate system. Figure 6 The Oxyz coordinate system shown in the figure, wherein the z-axis refers to the front of the camera, which is one of the binocular cameras; the y-axis is the bottom of the camera; obeying the right-hand principle, the x-axis is the right of the camera; the origin O is the optical center of the camera (such as a video camera).

[0104] Exemplarily, the predetermined coordinate system may be an inertial coordinate system. Here, the north-east coordinate system is adopted, that is, due north is the x-axis, due east is the y-axis, and the z-axis is established by the right-hand principle. The origin of the inertial coordinate system is the coordinate point of the target object (such as a drone) before takeoff; the inertial coordinate system is fixedly connected to the earth and moves with the rotation of the earth.

[0105] Exemplarily, the predetermined coordinate system may be a body coordinate system. The longitudinal axis (front) of the target object (such as a drone) is the x-axis, the right side is the y-axis, and the z-axis is established by the right-hand principle; the origin of the body coordinate system is the instantaneous center of gravity of the target object. The conversion relationship between the body coordinate system and the camera coordinate system is only related to the dark rotation position of the camera, so it can be obtained by prior calibration; it is considered that the transformation matrix (T bc ) is known.

[0106] Exemplarily, the predetermined coordinate system may be a pixel coordinate system. The origin (o') of the pixel coordinate system is located at the upper left corner of the image, the u axis is parallel to the x axis of the camera coordinate system to the right, and the v axis is parallel to the y axis of the camera coordinate system downward. The conversion matrix between the pixel coordinate system and the camera coordinate system is also called an intrinsic parameter; the intrinsic parameter matrix of the camera is fixed after leaving the factory and will not change during use, and its intrinsic parameter matrix (K) is considered known here.

[0107] Exemplarily, first, the binocular camera can be calibrated to obtain the internal and external parameters of each camera in the binocular camera. The internal parameters include parameters such as the focal length, principal point and / or distortion of the camera; the external parameters include parameters such as the position and / or posture of the camera. Then, the left and right images captured simultaneously by the binocular camera are matched to find the corresponding feature points (such as corner points, scale-invariant feature transform (SIFT) feature points, etc.), and their disparity is calculated, that is, the pixel displacement of the corresponding points in the left and right images is calculated. Using the principle of similar triangulation, the disparity is converted into depth information, for example, according to the relationship between the baseline length, the focal length of the camera and the disparity, the depth information (such as the depth value) of each pixel point is determined. According to the depth information, each pixel point can be converted into a coordinate point in three-dimensional space, thereby generating point cloud data. Point cloud data is usually represented as a series of three-dimensional points (x, y, z), each of which corresponds to a point in three-dimensional space. Finally, the point cloud data can be converted into a navigation coordinate system, which can be the inertial coordinate system mentioned above and does not change with the movement of the target object (such as a drone).

[0108] Exemplarily, the directions of the x-axis, y-axis and / or z-axis in the above coordinate system can be various directions of the target object, or the directions parallel to the z-axis, y-axis and / or z-axis can be various directions of the target object.

[0109] Exemplarily, the terminal obtains the positioning accuracy of the target object in each direction on the predetermined coordinate system, such as obtaining the positioning accuracy of the target object in the x-axis, y-axis and z-axis directions as d x d y and z The terminal obtains the maximum size of the target object in each direction on the predetermined coordinate system, such as obtaining the maximum size of the target object in the x-axis, y-axis and z-axis directions. x , l y and z The terminal obtains the safety margin of the target object in each direction on the predetermined coordinate system, such as obtaining the safety margin of the target object in the x-axis, y-axis and z-axis directions as s x 、s y and z The terminal determines the length information of the target object in each direction on the predetermined coordinate system based on the positioning accuracy, maximum size and safety margin of the target object in each direction on the predetermined coordinate system; for example, the length information of the target object on the x-axis, y-axis and z-axis is determined to be: a x =d x +l x +s x 、a y =d y +l y +sy and a z =d z +l z +s z .

[0110] In an embodiment of the present invention, the length information of the target object can be determined through the positioning accuracy, maximum size and safety margin of the target object in various directions. The actual length of the target object body in various directions, the error of the length in various directions and the distance considered for safety can be considered, so that more reasonable length information of the target object can be obtained; and based on the length information obtained in this way, it is helpful to obtain more comprehensive obstacle points.

[0111] Of course, in other embodiments, step S17 may be: based on the maximum size of the target object in each direction, determine the length information of the target object in each direction; or, step S17 may be: based on the maximum size of the target object in each direction and the positioning accuracy, determine the length information of the target object in each direction; or, step S17 may be: based on the maximum size of the target object in each direction and the safety margin, determine the length information of the target object in each direction.

[0112] In the embodiment of the present invention, the real scene in which the size of the target object may be inconsistent in various directions is taken into consideration, instead of directly taking the maximum size information of the target object body, so that the subsequent expanded map is more reasonable and the map expansion is avoided from being too large.

[0113] In some embodiments, in step S14, based on the length information of the target object in each direction, the point cloud data is subjected to ellipsoid expansion processing to obtain obstacle point data, including:

[0114] The length information and the point cloud data are input into an ellipsoid equation to obtain the obstacle point data, wherein the obstacle point data is within the ellipsoid indicated by the ellipsoid equation.

[0115] In some embodiments, the ellipsoid equation includes a first part and a second part; wherein the first part is the sum of first values ​​in each direction, and the second part is a predetermined value; the first part is less than or equal to the second part; wherein the first value is the ratio of the second value to the third value in the same direction; the second value is the square of the difference between the obstacle point data and the point cloud data in one direction in each direction, and the third value is the square of the length information in one direction in each direction.

[0116] Exemplarily, the predetermined value may be 1. Of course, the predetermined value may also be other values, for example, a certain constant.

[0117] For example, the ellipsoid equation can be Among them, (xyz) is the obstacle point data; (x n y n z n ) is point cloud data; (a x a y a z ) is the length information of the target object in each direction. In the ellipsoid equation, (x n y n z n ) and (a x a y a z ) is a known quantity; by solving the ellipsoid equation, the unknown obstacle point data (xyz) can be obtained.

[0118] In other embodiments, the ellipsoid equation includes a third part and a fourth part; the third part is the sum of fourth values ​​in each direction, and the second part is a predetermined value; the third part is equal to the fourth part; wherein the square of the ratio of the point cloud data in one direction in each direction of the fourth value to the positioning information; the coordinate data within the ellipsoid indicated by the ellipsoid equation can all be obstacle point data.

[0119] For example, the ellipsoid equation can be Among them, (x n y n z n ) is point cloud data; (a x a y a z ) is the length information of the target object in each direction; the coordinate data within the ellipsoid indicated by the ellipsoid method can all be obstacle point data.

[0120] Thus, in an embodiment of the present invention, the point cloud data can be adaptively expanded according to an ellipsoid. Compared with the related art in which the point cloud data is expanded according to a circle or a sphere, the popular area of ​​the target object can be maximized, thereby improving the performance of the path planned for the target object.

[0121] In an embodiment of the present invention, the octree map is used for an obstacle avoidance system of a target object (e.g., an obstacle avoidance system for a drone), so that the value of the obstacle avoidance system for a drone mainly includes the following:

[0122] (1) Improved safety. The drone obstacle avoidance system can help the drone avoid collisions with obstacles; thereby reducing the risk of accidents and damage and improving the safety of drone operations. This is very important for many industries, such as aerial photography, logistics and / or monitoring, to avoid property damage and personal injury caused by collisions.

[0123] (2) Improve work efficiency. The obstacle avoidance system of drones can enable drones to automatically avoid obstacles in complex environments, thereby reducing the need for operator intervention and improving the efficiency of task execution. This can significantly save time and human resources in many industries, such as in the agricultural field for plant protection, in the industrial field for inspection, and / or in the field of logistics for cargo transportation.

[0124] (3) Expanding the application areas. The obstacle avoidance system of UAVs can allow UAVs to operate in complex and crowded environments, such as urban areas, forests and / or mountainous areas. This will expand the application areas of UAVs to include urban planning, disaster monitoring and rescue, and / or environmental monitoring. This provides new business opportunities for many industries and fields.

[0125] (4) Providing value-added services. The drone obstacle avoidance system can provide value-added services to users, such as avoiding damage to the drone and improving the quality and / or stability of aerial photography. This will enable drone operators to provide higher quality services to customers and obtain higher profits from them. In addition, the drone obstacle avoidance system can also be integrated with other technologies, such as cameras and / or sensors, to provide richer data and information, thereby providing more commercial value.

[0126] (5) Reduce costs. The drone obstacle avoidance system can reduce the repair and replacement costs caused by the collision between the drone and obstacles, thereby reducing operating costs. In addition, it can also reduce insurance costs caused by accidents and damages, and reduce economic losses caused by shutdowns and production interruptions caused by unexpected events.

[0127] It should be noted that the following description of the drone is similar to the description of the map processing method, and the beneficial effects of the method are not described in detail. For technical details not disclosed in the drone embodiment of the present invention, please refer to the description of the map processing method embodiment of the present invention.

[0128] An embodiment of the present invention provides a drone, including:

[0129] A binocular camera, wherein the binocular camera comprises two cameras;

[0130] The corresponding reflecting devices are respectively arranged at predetermined angles with the two cameras so that the distance between the focal points of the two cameras is the second baseline length and the distance between the two virtual focal points of the reflecting devices is the first baseline length; wherein the first baseline length is greater than the second baseline length.

[0131] Alternatively, if Figure 3 As shown, the two cameras mentioned above may be camera 1 and camera 2 ; the reflecting device mentioned above may include a transmitting device 1 and a reflecting device 2 .

[0132] In some embodiments, two cameras in the binocular camera respectively receive visual information of the drone on the planned path through the corresponding reflection device to generate two images; the images are used to determine the parallax of the drone on the planned path.

[0133] It should be noted that the following description of the map processing device is similar to the description of the map processing method, and the beneficial effects of the same method are not described in detail. For technical details not disclosed in the map processing device embodiment of the present invention, please refer to the description of the map processing method embodiment of the present invention.

[0134] like Figure 7 As shown, an embodiment of the present invention provides a map processing device, comprising

[0135] An acquisition module 21 is used to acquire the baseline length and parallax corresponding to the target object on the planned path;

[0136] A determination module 22, configured to determine depth information according to the baseline length and the parallax;

[0137] The determination module 22 is further used to determine point cloud data in a predetermined coordinate system according to the depth information;

[0138] A processing module 23 is used to perform ellipsoid expansion processing on the point cloud data based on the length information of the target object in various directions to obtain obstacle point data; wherein the obstacle point data is within the ellipsoid;

[0139] The processing module 23 is used to input the obstacle point data into the octree node to update the information so as to construct an octree map; wherein the nodes corresponding to the obstacle point data in the octree map are represented by the first indication information and the nodes corresponding to the non-obstacle point data are represented by the second indication information.

[0140] In some embodiments, the determination module 22 is used to determine the positioning accuracy, maximum size and safety margin of the target object in various directions;

[0141] The processing module 23 is used to determine the length information of the target object in each direction based on the positioning accuracy of the target object in each direction, the sum of the maximum size and the safety margin.

[0142] In some embodiments, the processing module 23 is used to input the length information and the point cloud data into an ellipsoid equation to obtain the obstacle point data, wherein the obstacle point data is within the ellipsoid indicated by the ellipsoid equation.

[0143] In some embodiments, the first part of the ellipsoid equation is the sum of first values ​​in various directions, and the second part of the ellipsoid equation is a predetermined value; the first part is less than or equal to the second part;

[0144] Among them, the first value is the ratio of the second value to the third value in the same direction; the second value is the square of the difference between the obstacle point data and the point cloud data in one direction among all directions, and the third value is the square of the length information in one direction among all directions.

[0145] In some embodiments, the determination module 22 is used to determine the first parameter based on the product of the baseline length and the camera focal length; wherein the camera focal length is the focal length of the binocular camera set for the target object;

[0146] The determination module 22 is further configured to determine the depth information based on the ratio of the first parameter and the disparity.

[0147] In some embodiments, the acquisition module 21 is used to determine the first baseline length based on a reflective device set on the target object; the first baseline length is greater than the second baseline length, wherein the second baseline length is a baseline length determined when the target object is not set with a reflective device to collect an image;

[0148] The acquisition module 21 is used to set a reflection device based on the target object and determine the parallax determined by collecting images of the target object on the planned path.

[0149] like Figure 8 As shown, an embodiment of the present invention further provides a terminal, which includes a processor 31 and a memory 32 for storing a computer program that can be run on the processor 31; wherein, when the processor 31 is used to run the computer program, the map processing method of any embodiment of the present invention is implemented.

[0150] In some embodiments, the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0151] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor may be a microprocessor or the processor may also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0152] In some embodiments, the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the invention, or a combination thereof.

[0153] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0154] An embodiment of the present invention provides a computer storage medium, wherein the computer-readable storage medium stores an executable program. When the executable program is executed by a processor, the steps of the map processing method of any embodiment of the present invention can be implemented.

[0155] In some embodiments, the computer storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0156] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0157] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A map processing method, characterized in that: The method comprises: Get the baseline length and disparity of the target object on the planned path; Determining depth information according to the baseline length and the parallax; Determining point cloud data in a predetermined coordinate system according to the depth information; Based on the length information of the target object in each direction, the point cloud data is subjected to ellipsoid expansion processing to obtain obstacle point data; wherein the obstacle point data is within the ellipsoid; The obstacle point data is input into the octree node for information update to construct an octree map; wherein the nodes corresponding to the obstacle point data in the octree map are represented by first indication information and the nodes corresponding to the non-obstacle point data are represented by second indication information.

2. The method according to claim 1, characterized in that The method further comprises: Determining the positioning accuracy, maximum size and safety margin of the target object in each direction; The length information of the target object in each direction is determined based on the positioning accuracy of the target object in each direction, the sum of the maximum size and the safety margin.

3. The method according to claim 2, characterized in that The step of performing ellipsoid expansion processing on the point cloud data based on the length information of the target object in each direction to obtain obstacle point data includes: The length information and the point cloud data are input into an ellipsoid equation to obtain the obstacle point data, wherein the obstacle point data is within the ellipsoid indicated by the ellipsoid equation.

4. The method according to claim 3, characterized in that The first part of the ellipsoid equation is the sum of first values ​​in various directions, and the second part of the ellipsoid equation is a predetermined value; the first part is less than or equal to the second part; Among them, the first value is the ratio of the second value to the third value in the same direction; the second value is the square of the difference between the obstacle point data and the point cloud data in one direction among all directions, and the third value is the square of the length information in one direction among all directions.

5. The method according to claim 1, characterized in that: The determining of depth information according to the baseline length and the parallax includes: Determine a first parameter based on the product of the baseline length and the camera focal length; wherein the camera focal length is the focal length of the binocular camera set for the target object; The depth information is determined based on the ratio of the first parameter and the disparity.

6. The method according to claim 1 or 5, characterized in that: The obtaining of the baseline length and disparity corresponding to the target object on the planned path includes: Based on a reflective device provided on the target object, a first baseline length is determined; the first baseline length is greater than a second baseline length, wherein the second baseline length is a baseline length determined when the target object is not provided with a reflective device and an image is collected; A reflection device is set based on the target object, and a parallax determined by collecting images of the target object on a planned path is determined.

7. A drone, characterized in that: include: A binocular camera, wherein the binocular camera comprises two cameras; The corresponding reflecting devices are respectively arranged at predetermined angles with the two cameras so that the distance between the focal points of the two cameras is the second baseline length and the distance between the two virtual focal points of the reflecting devices is the first baseline length; wherein the first baseline length is greater than the second baseline length.

8. A map processing device, characterized in that: include: An acquisition module is used to obtain the baseline length and parallax corresponding to the target object on the planned path; A determination module, configured to determine depth information according to the baseline length and the parallax; The determination module is further used to determine point cloud data in a predetermined coordinate system according to the depth information; A processing module, configured to perform ellipsoid expansion processing on the point cloud data based on the length information of the target object in various directions to obtain obstacle point data; wherein the obstacle point data is within the ellipsoid; The processing module is also used to input the obstacle point data into the octree node to update the information so as to construct an octree map; wherein the nodes corresponding to the obstacle point data in the octree map are represented by the first indication information and the nodes corresponding to the non-obstacle point data are represented by the second indication information.

9. A terminal, characterized in that: The terminal includes a processor and a memory for storing a computer program that can be run on the processor; wherein the processor, when running the computer program, implements the map processing method according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The computer storage medium contains computer executable instructions, wherein the computer executable instructions are executed by a processor to implement the map processing method according to any one of claims 1 to 6.

Images