Obstacle detection and re-identification method, device, movable platform and storage medium

Abstract

An obstacle detection and re-identification method, device, movable platform and storage medium, the method comprising: obtaining at least two images respectively captured by at least two shooting devices arranged on the movable platform, and obtaining a three-dimensional grid model generated based on the position information of the movable platform (S201); determining one or more three-dimensional anchor points from the three-dimensional grid model according to the at least two images, the three-dimensional anchor points being used to indicate the positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located (S202); extracting feature information at the positions indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information (S203); and generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images (S204). The method implements both obstacle detection and obstacle re-identification in an overall framework.

Description

Technical Field

[0001] The present application relates to the technical field of object recognition, and in particular to an obstacle detection and re-recognition method, device, movable platform and storage medium. Background Art

[0002] In the fields of autonomous driving, drone automatic flight, and robot navigation, obstacle recognition is one of the key factors for the safe and reliable operation of mobile platforms. Usually, various sensors (such as millimeter wave radar, lidar, camera, or satellite navigation components, etc.) installed on the mobile platform are used to sense the surrounding environment during the mobile movement process, collect perception data, and identify, detect, and track obstacles (static or dynamic objects) based on the perception data, predict possible dangers, and ensure that the mobile platform can operate safely and reliably.

[0003] Among them, in the process of identifying obstacles based on perception data, if there are multiple sensors, their perception ranges may overlap and the same obstacle may be perceived. Therefore, it is also necessary to associate the same obstacle perceived by multiple sensors, which is the process of target re-identification.

[0004] In related technologies, obstacle detection and obstacle re-identification are usually divided into two processes and processed independently. First, obstacle detection is performed based on the perception data obtained by the sensor, such as obtaining obstacle detection information based on the features extracted from the perception data. Then, the same obstacle detected by each sensor is associated, and the re-identification result is generally obtained by matching the position and features of the obstacle. Among them, the independent processing of obstacle detection and obstacle re-identification is usually time-consuming and cannot meet the real-time requirements. Summary of the invention

[0005] In view of this, one of the objects of the present application is to provide an obstacle detection and re-identification method, device, movable platform and storage medium.

[0006] In a first aspect, an embodiment of the present application provides an obstacle detection and re-identification method, including:

[0007] Acquire at least two images respectively captured by at least two shooting devices disposed on a movable platform, and acquire a three-dimensional grid model generated based on position information of the movable platform;

[0008] Determine one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located;

[0009] Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information;

[0010] And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0011] In a second aspect, an embodiment of the present application provides an obstacle detection and re-identification method, including:

[0012] Acquire a three-dimensional grid model generated based on the position information of the movable platform, and determine one or more three-dimensional anchor points from the three-dimensional grid model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located;

[0013] Acquire at least two images respectively captured by at least two shooting devices disposed on the movable platform;

[0014] Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information;

[0015] And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0016] In a third aspect, an embodiment of the present application provides an obstacle detection and re-identification device, including a processor and a memory storing a computer program;

[0017] When executing the computer program, the processor implements the following steps:

[0018] Acquire at least two images respectively captured by at least two shooting devices disposed on a movable platform, and acquire a three-dimensional grid model generated based on position information of the movable platform;

[0019] Determine one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located;

[0020] Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information;

[0021] And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0022] In a fourth aspect, an embodiment of the present application provides an obstacle detection and re-identification device, including a processor and a memory storing a computer program;

[0023] When executing the computer program, the processor implements the following steps:

[0024] Acquire a three-dimensional grid model generated based on the position information of the movable platform, and determine one or more three-dimensional anchor points from the three-dimensional grid model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located;

[0025] Acquire at least two images respectively captured by at least two shooting devices disposed on the movable platform;

[0026] Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information;

[0027] And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0028] In a fifth aspect, an embodiment of the present application provides a movable platform, including:

[0029] Body;

[0030] A power system, installed in the body, for providing power to the movable platform;

[0031] At least two camera devices for capturing images; and

[0032] The obstacle detection and re-identification device as described in the third aspect or the fourth aspect.

[0033] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0034] An obstacle detection and re-identification method provided in an embodiment of the present application first obtains a three-dimensional grid model generated based on the position information of a movable platform, and determines one or more three-dimensional anchor points from the three-dimensional grid model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located; and obtains at least two images respectively captured by at least two shooting devices arranged on the movable platform, extracts feature information at the positions indicated by each of the three-dimensional anchor points from each of the images, and performs obstacle detection based on the feature information to obtain obstacle detection information. At the same time, if the feature information corresponding to the at least two images respectively points to the three-dimensional anchor points The same naturally indicates that the obstacle detection information obtained by the feature information corresponding to the at least two images respectively belongs to the same obstacle, and the purpose of re-identification is also achieved; it can be seen that in this embodiment, the three-dimensional anchor point is used to locate the position where the obstacle may appear around the movable platform, and then based on the three-dimensional anchor point and the feature information corresponding to the three-dimensional anchor point in each image, both the obstacle detection process and the obstacle re-identification process are implemented in an overall framework. While obtaining the obstacle detection information, the re-identification result is also determined based on the three-dimensional anchor point. The obstacle detection process and the obstacle re-identification process are completed at the same time, which is beneficial to improve the detection and identification efficiency of the obstacle, thereby meeting the real-time requirements in certain scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0037] Figure 2 is a structural schematic diagram of a vehicle provided by an embodiment of the present application;

[0038] Figure 3 is a schematic diagram of an image acquisition device on a vehicle provided by an embodiment of the present application;

[0039] Figure 4 as well as Figure 7 is a different schematic diagram showing obstacle detection information and re-identification results provided by an embodiment of the present application;

[0040] Figure 5 as well as Figure 81 is a schematic diagram of different flow charts of an obstacle detection and re-identification method provided by an embodiment of the present application;

[0041] Figure 6 is a schematic diagram of a three-dimensional grid model provided by an embodiment of the present application;

[0042] Fig. 9 is a structural schematic diagram of an obstacle detection and re-identification device provided by an embodiment of the present application;

[0043] Fig.10 It is a structural schematic diagram of a movable platform provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] In related technologies, obstacle detection and obstacle re-identification are usually divided into two processes and processed independently. First, obstacle detection is performed based on the perception data obtained by the sensor, such as obtaining obstacle detection information based on the features extracted from the perception data. Then, the same obstacle detected by each sensor is associated, generally by matching the position and features of the obstacle, to obtain the re-identification result. Independently processing the obstacle detection process and the obstacle re-identification process is usually time-consuming and cannot meet the real-time requirements.

[0046] Based on this, an embodiment of the present application provides an obstacle detection and re-identification method, which realizes both obstacle detection and obstacle re-identification in an overall framework, which is beneficial to improving the detection and identification efficiency of obstacles and can meet the real-time requirements in certain scenarios; the embodiment of the present application first obtains a three-dimensional grid model generated based on the position information of a movable platform, and determines one or more three-dimensional anchor points from the three-dimensional grid model, and the three-dimensional anchor points are used to indicate the position where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located; and obtains at least two images respectively captured by at least two shooting devices arranged on the movable platform, extracts feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performs obstacle detection based on the feature information to obtain obstacle information. At the same time, if the feature information corresponding to the at least two images respectively points to the same three-dimensional anchor point, it naturally indicates that the obstacle detection information obtained by the feature information corresponding to the at least two images respectively belongs to the same obstacle, and the purpose of re-identification is also achieved; it can be seen that in this embodiment, the position where obstacles may appear around the movable platform is located by the three-dimensional anchor point, and then based on the three-dimensional anchor point and the feature information corresponding to the three-dimensional anchor point in each image, both the obstacle detection process and the obstacle re-identification process are implemented in an overall framework, and the obstacle detection information is obtained while the re-identification result is determined based on the three-dimensional anchor point. The obstacle detection process and the obstacle re-identification process are completed at the same time, which is conducive to improving the detection and identification efficiency of obstacles, thereby being able to meet the real-time requirements in certain scenarios.

[0047] The obstacle detection and re-identification method of the embodiment of the present application can be implemented by an obstacle detection and re-identification device. In a possible implementation, the obstacle detection and re-identification device can be a computer chip or integrated circuit with data processing capabilities, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). The obstacle detection and re-identification device can be installed in a movable platform; the movable platform of the embodiment of the present application can include: a car, an unmanned aerial vehicle, an unmanned ship or a robot, wherein the car can be an unmanned vehicle or a manned vehicle, and the unmanned aerial vehicle can be a four-rotor drone, a six-rotor drone or an eight-rotor drone. In another implementation, the obstacle detection and re-identification device can also be a movable platform, etc.; the movable platform at least includes a car, an unmanned aerial vehicle, an unmanned ship or a robot, wherein the car can be an unmanned vehicle or a manned vehicle.

[0048] In an exemplary application scenario, see Figure 1 as well as Figure 2 , taking the movable platform as a vehicle, and the vehicle including the obstacle detection and re-identification device as an example for explanation, Figure 1 shows a driving scene of a vehicle 100, Figure 1 It is assumed that vehicle 100 is the host vehicle and other vehicles are obstacles. Figure 2 The structure of the vehicle 100 is shown, and the vehicle 100 may also be equipped with at least two camera devices 10 and a satellite positioning device 20 ( Figure 2 Taking two photographing devices 10 as an example), the images respectively captured by the at least two photographing devices 10 have a preset overlap ratio, Figure 2 Two photographing devices 10 are used as an example for description. Optionally, this embodiment does not impose any restrictions on the installation position of the photographing position on the vehicle 100. For example, see Figure 3 , one of the camera devices 10 can be installed on the front of the vehicle 100, and the other camera device 10 can be installed on the side of the vehicle 100. Figure 3As shown, the fields of view of the two camera devices 10 partially overlap, so that there are overlapping parts in the two images captured by the two camera devices 10. For example, image 11 is an image captured by the camera device 10 installed on the front of the vehicle, and image 12 is an image captured by the camera device 10 installed on the side of the vehicle body. Both capture image data of the same vehicle, thereby achieving redundancy. Optionally, the at least two camera devices 10 can operate at the same frame rate. Optionally, the shooting parameters or lens types of at least two camera devices 10 can be the same or different, and can be specifically set according to the actual application scenario. For example, if the at least two camera devices 10 are installed on Figure 3 The lens of the camera 10 at the front of the vehicle 100 shown may be a standard lens, and the lens of the camera 10 installed on the side of the vehicle 100 may be a fisheye lens.

[0049] During the driving process of the vehicle 100, the at least two shooting devices 10 transmit the images respectively collected to the obstacle detection and re-identification device 30, and the satellite positioner 20 also transmits the obtained position information of the vehicle 100 to the obstacle detection and re-identification device 30. Optionally, the frequency of the satellite positioner 20 acquiring the position information is synchronized with the frame rate of the at least two shooting devices 10. The obstacle detection and re-identification device 30 can generate a three-dimensional grid model based on the position information of the vehicle 100, and determine one or more three-dimensional anchor points from the three-dimensional grid model, and the three-dimensional anchor points are used to indicate the position where the obstacle may exist in the three-dimensional space where the three-dimensional grid model is located, and then obtain obstacle detection information and re-identification results according to the three-dimensional anchor points and the at least two images.

[0050] Further, after obtaining the obstacle detection information and the re-identification result, the obstacle detection information and the re-identification result can be used to make obstacle avoidance decisions or perform route planning. Figure 4 , FIG110 is the obstacle detection information obtained according to the image 11, and FIG120 is the obstacle detection information obtained according to the image 12. The obstacle detection information and the re-identification result can be displayed on the interface of the vehicle 100 or the interface of the terminal connected to the vehicle 100 in communication, so that the user can understand the driving condition of the vehicle 100 and the road condition around the vehicle 100. Alternatively, the obstacle detection information and the re-identification result can be transmitted to other components in the vehicle 100, so that the other components can control the vehicle 100 to work safely and reliably based on the obstacle detection information and the re-identification result.

[0051] Next, the obstacle detection and re-identification method provided in the embodiment of the present application is described: Please refer to Figure 5 , Figure 5A flowchart of an obstacle detection and re-identification method provided in an embodiment of the present application is provided. The method can be implemented by an obstacle detection and re-identification device; the obstacle detection and re-identification device can be a movable platform, or the obstacle detection and re-identification device can be installed as a chip in a movable platform; the method includes:

[0052] In step S101, a three-dimensional mesh model generated based on the position information of the movable platform is obtained, and one or more three-dimensional anchor points are determined from the three-dimensional mesh model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located.

[0053] In step S102, at least two images respectively captured by at least two shooting devices disposed on the movable platform are acquired.

[0054] In step S103, feature information at the position indicated by each of the three-dimensional anchor points is extracted from each of the images, and obstacle detection is performed based on the feature information to obtain obstacle detection information.

[0055] In step S104, a re-identification result is generated according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0056] In some embodiments, the location information of the movable platform can be obtained by a satellite locator installed on the movable platform; wherein the satellite locator can be a locator based on the Global Navigation Satellite System (GNSS), and the GNSS includes but is not limited to GPS (Global Positioning System), GLONASS system (GLONASS satellite navigation system), Galileo system (Galileo satellite navigation system) and Beidou satellite navigation system, etc.

[0057] In some embodiments, the shooting device includes but is not limited to a visible light camera, a grayscale camera, and an infrared camera. The shooting device can capture a sequence of images at a specified frame rate. The shooting device can have adjustable shooting parameters. Under different shooting parameters, the shooting device can capture different images despite being subjected to exactly the same external conditions (e.g., position, lighting). Shooting parameters may include exposure (e.g., exposure time, shutter speed, aperture, film speed), gain, gamma, region of interest, binning / subsampling, pixel clock, offset, trigger, ISO, etc. Parameters related to exposure can control the amount of light reaching the image sensor in the shooting device. For example, the shutter speed can control the amount of time for light to reach the image sensor and the aperture can control the amount of light reaching the image sensor within a given time. Parameters related to gain can control the amplification of the signal from the optical sensor. ISO can control the sensitivity level of the camera to available light. The shooting device has a lens, which includes but is not limited to a fixed focus lens, a standard lens, a telephoto lens, a wide-angle lens, a fisheye lens, and a zoom lens.

[0058] Exemplarily, the at least two cameras may operate at the same frame rate. Exemplarily, the at least two cameras may be cameras of the same type, such as visible light cameras; or they may be cameras of different types, such as one camera being a visible light camera and the other being a grayscale camera. Exemplarily, the shooting parameters and lens types of the at least two cameras may be the same or different. Exemplarily, the fields of view of the at least two cameras have a preset overlap rate, so that at least two images respectively captured by the at least two cameras also have overlapping parts, thereby achieving redundancy.

[0059] In some embodiments, the frequency at which the satellite locator acquires position information is synchronized with the frame rate of the at least two shooting devices, that is, while the shooting device captures the image, the satellite locator also acquires the position information of the movable platform, thereby ensuring a one-to-one correspondence between the position information of the movable platform and the image, indicating that the image is captured when the movable platform is at the position indicated by the position information, thereby ensuring the accuracy of the subsequently obtained obstacle detection information and re-identification results.

[0060] In step S101, after obtaining the position information of the movable platform, the obstacle detection and re-identification device can generate a three-dimensional grid model based on the position information of the movable platform and preset grid parameters, and the grid parameters at least indicate the size and / or shape of the three-dimensional grid in the three-dimensional grid model (for example, the three-dimensional grid can be a cuboid or a cube, etc.). In an example, refer to Figure 6, taking the movable platform as vehicle 100 as an example, for example, the origin of the three-dimensional coordinate system where the three-dimensional mesh model is located can be determined based on the position information of the movable platform, the moving direction of the movable platform is the X-axis direction, the direction perpendicular to the X-axis is the Y-axis direction (for example, the plane where the X-axis direction and the Y-axis direction are located can be parallel to the ground), and the direction perpendicular to the ground is the Z-axis direction. Then, the three-dimensional mesh model can be generated based on the above parameters of the three-dimensional coordinate system where the three-dimensional mesh model is located (origin, three-axis directions, etc.) and based on preset mesh parameters.

[0061] After acquiring the 3D mesh model, the obstacle detection and re-identification device may acquire one or more 3D anchors from the 3D mesh model, wherein the 3D anchors indicate positions where obstacles may exist in the 3D space where the 3D mesh model is located. Exemplarily, the 3D anchors may be acquired in the following possible implementations:

[0062] In a first possible implementation, corresponding three-dimensional anchor points may be determined according to respective positions in the three-dimensional mesh model. That is, assuming that obstacles may exist at all positions in the three-dimensional mesh model, three-dimensional anchor points corresponding to all positions in the three-dimensional mesh model may be obtained.

[0063] In a second possible implementation, the probability of obstacles existing at various positions in the three-dimensional grid model can be statistically calculated based on historical data. The historical data can be historical detection information about obstacles obtained by the obstacle detection and re-identification device within a historical time period. The historical detection information can include the position information of obstacles (such as three-dimensional position information, etc.). Then, the obstacle detection and re-identification device can determine the one or more three-dimensional anchor points based on the statistically obtained probability of obstacles existing at various positions in the three-dimensional grid model. Exemplarily, in order to save computing resources, the three-dimensional anchor points can be determined based on the positions where the probability of obstacles existing in the three-dimensional grid model is greater than a preset threshold, and the positions where the probability of obstacles existing in the three-dimensional grid model is not greater than the preset threshold are excluded. The preset threshold can be flexibly set according to the actual application scenario. Compared with the first implementation, the number of three-dimensional anchor points is significantly reduced, which is conducive to reducing the complexity of calculation (for example, the amount of calculation in the subsequent obstacle detection process is reduced), thereby achieving the purpose of saving computing resources.

[0064] In a third possible implementation, one or more three-dimensional anchor points may be determined from the three-dimensional grid model according to the attribute information of the movable platform and / or the attribute information of the obstacle. The attribute information of the movable platform includes one or more of the following: the movement mode, category, size of the movable platform and the installation position of the shooting device on the movable platform; the attribute information of the obstacle includes but is not limited to the category, size and shape of the obstacle. This embodiment determines the three-dimensional anchor points that meet the attribute information of the movable platform and / or the attribute information of the obstacle from the three-dimensional grid model. Compared with the first implementation, the number of three-dimensional anchor points is significantly reduced, which is conducive to reducing the complexity of calculation (for example, the amount of calculation in the subsequent obstacle detection process is reduced), and the purpose of saving computing resources is achieved.

[0065] Exemplarily, based on the attribute information of the movable platform and / or the attribute information of the obstacle, the positions in the three-dimensional grid model where there are no obstacles or the positions in the three-dimensional grid model where there is no need to detect obstacles can be excluded, and then the three-dimensional anchor points can be determined based on the remaining positions in the three-dimensional grid model. Here, taking the movable platform as an unmanned vehicle as an example, based on the attribute information of the unmanned vehicle and / or the attribute information of the obstacle, it can be determined that there are no obstacles in the part below the ground, and there is no need to detect obstacles in the part in the sky, so the positions in the three-dimensional grid model below the ground or in the sky can be excluded.

[0066] Exemplarily, based on the attribute information of the movable platform and / or the attribute information of the obstacle, the position where the obstacle may exist in the three-dimensional grid model can be determined, and the three-dimensional anchor point can be determined according to the position where the obstacle may exist in the three-dimensional grid model. Here, taking the movable platform as an unmanned vehicle as an example, based on the attribute information of the unmanned vehicle and / or the attribute information of the obstacle, it can be determined that there is a high probability of an obstacle at the position on the ground, thereby affecting the driving safety of the unmanned vehicle, and therefore the three-dimensional anchor point can be determined according to the position on the ground in the three-dimensional grid model.

[0067] In a fourth possible implementation, at least two images captured by at least two shooting devices respectively arranged on the movable platform can be obtained. Exemplarily, the frequency at which the satellite locator acquires the position information is synchronized with the frame rate of the at least two shooting devices, that is, the at least two images are captured when the movable platform is at the position indicated by the position information, thereby ensuring the correspondence between the three-dimensional grid model generated based on the position information and the at least two images. Then, one or more three-dimensional anchor points can be determined from the three-dimensional grid model based on the at least two images. Since the at least two images contain two-dimensional information of the obstacles to be identified, the accuracy of the determined three-dimensional anchor points can be further improved. Compared with the first possible implementation, the number of three-dimensional anchor points determined based on the at least two images is further reduced and more accurate, which is conducive to reducing the complexity of calculation (for example, the amount of calculation in the subsequent obstacle detection process is reduced), and the purpose of saving computing resources is achieved.

[0068] In the first example, the at least two images and the three-dimensional network model may be input into a pre-trained first model, so that the first model can extract features from the at least two images, and predict the positions where obstacles may exist in the three-dimensional mesh model based on the extracted features, so as to determine the one or more three-dimensional anchor points. In this embodiment, considering that the at least two images contain two-dimensional information of obstacles to be identified, predicting the positions where obstacles may exist in the three-dimensional mesh model based on the at least two images is conducive to improving the accuracy of the determined three-dimensional anchor points.

[0069] Among them, the first model can be trained based on several training samples, and the training samples include image samples and three-dimensional mesh model samples based on a movable platform. Here, supervised training is taken as an example. The training samples also correspond to the true values ​​of three-dimensional anchor points. During the training process, the training samples can be input into the first model so that the first model can predict the positions where obstacles may exist in the three-dimensional mesh model according to the extracted features of the image samples, thereby outputting three-dimensional anchor point prediction values, and then adjusting the parameters of the first model according to the difference between the three-dimensional anchor point prediction values ​​and the true values ​​of the three-dimensional anchor points, thereby obtaining the trained first model. The first model is used to predict the positions where obstacles may exist in the three-dimensional mesh model according to the image, so as to determine the three-dimensional anchor points.

[0070] In the second example, one or more first two-dimensional anchors (2D Anchors) can be generated based on each of the images obtained by each of the shooting devices, and the first two-dimensional anchors are used to indicate the positions where obstacles may exist in the two-dimensional space where the images are located; as an example, feature extraction can be performed on each of the images, and the one or more first two-dimensional anchors can be determined based on the extracted features; wherein the features include but are not limited to: SIFT (scale-invariant feature transform) features, HOG (histogram of oriented gradients) features, LBP (Local Binary Pattern) features, SURF (Speeded Up Robust Features) features, ORB (Oriented Fast and Rotated Brief) features, HAAR features, or image features extracted based on feature extraction neural networks, etc.

[0071] After obtaining the one or more first two-dimensional anchor points, the one or more first two-dimensional anchor points can be projected to the three-dimensional space where the three-dimensional mesh model is located according to the projection relationship between the image and the three-dimensional mesh model to obtain the one or more three-dimensional anchor points. In this embodiment, considering that the at least two images contain two-dimensional information of obstacles to be identified, the position where the obstacle may exist in the two-dimensional space based on the two-dimensional image is determined with high accuracy, and the three-dimensional anchor point obtained based on the mapping of the one or more first two-dimensional anchor points also has a high accuracy, and there is a great possibility that an obstacle exists at the position indicated by the three-dimensional anchor point.

[0072] Among them, the projection relationship between the image and the three-dimensional mesh model can be determined based on the positional relationship between the movable platform and the shooting device, and the external parameters and internal parameters of the shooting device; for example, the external parameters of the camera coordinate system of the shooting device to the three-dimensional coordinate system where the three-dimensional mesh model is located can be determined according to the positional relationship between the movable platform and the shooting device, and the external parameters of the shooting device, and then the projection relationship between the image and the three-dimensional mesh model is obtained based on the external parameters of the camera coordinate system of the shooting device to the three-dimensional coordinate system where the three-dimensional mesh model is located and the internal parameters of the shooting device.

[0073] In the third example, the obstacle detection and re-identification device can obtain multiple images captured by the at least two shooting devices in a continuous time series; then perform moving target detection based on the multiple images to obtain the movement information of the obstacle; and obtain the movement information of the movable platform in the continuous time series; finally, determine one or more three-dimensional anchor points from the three-dimensional grid model based on the movement information of the obstacle and the movement information of the movable platform. In this embodiment, the three-dimensional anchor point is predicted based on the movement information of the obstacle and the movement information of the movable platform, and the obstacle is tracked and monitored in time series, which is conducive to ensuring the accuracy of the determined three-dimensional anchor point.

[0074] In one example, there are two shooting devices, one of which captures images a, b and c in a continuous time series, and the other captures images A, B and C in a continuous time series. The obstacle detection and re-identification device can perform moving target detection based on images a, b and c to obtain first motion information of the obstacle; and perform moving target detection based on images A, B and C to obtain second motion information of the obstacle; then predict one or more three-dimensional anchor points from the three-dimensional grid model based on the first motion information, the second motion information and the motion information of the movable platform in the continuous time series.

[0075] After obtaining one or more three-dimensional anchor points of the three-dimensional model and at least two images respectively captured by at least two shooting devices arranged on the movable platform, in step S103, the obstacle detection and re-identification device can extract feature information at the position indicated by each three-dimensional anchor point from each image, and the feature information can be two-dimensional feature information and / or three-dimensional feature information, and then obstacle detection is performed based on the feature information to obtain obstacle detection information.

[0076] When extracting the feature information, the two-dimensional feature information and / or the three-dimensional feature information may be obtained by any combination of the following two possible implementations:

[0077] In a first possible implementation, the obstacle detection and re-identification device may project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located based on the projection relationship between the image and the three-dimensional grid model, and obtain one or more second two-dimensional anchor points (2D Anchors), wherein the second two-dimensional anchor points are used to indicate the positions where obstacles may exist in the two-dimensional space where the image is located, and then extract the two-dimensional feature information at the positions indicated by each of the second two-dimensional anchor points from the image. In this embodiment, by mapping the three-dimensional anchor points into the two-dimensional space to obtain the second two-dimensional anchor points, the image information at the positions where obstacles may exist indicated by the three-dimensional anchor points in the image can be obtained. Since there is a high probability that there is an obstacle at the position indicated by the three-dimensional anchor points, obstacle detection is performed based on the image information, which is conducive to improving the accuracy of obstacle detection, and the image information of other positions in the image that are not indicated by the three-dimensional anchor points does not need to be processed, which is conducive to improving the efficiency of obstacle detection.

[0078] The projection relationship between the image and the three-dimensional grid model may be determined based on the positional relationship between the movable platform and the shooting device, and the external parameters and internal parameters of the shooting device.

[0079] Exemplarily, when extracting the two-dimensional feature information at the position indicated by each second two-dimensional anchor point, the obstacle detection and re-identification device may first perform feature extraction on the image to obtain the two-dimensional feature information of the image, and then obtain the two-dimensional feature information at the position indicated by each second two-dimensional anchor point from the two-dimensional feature information of the image according to each second two-dimensional anchor point. In one example, the two-dimensional feature information of the image includes but is not limited to: SIFT (Scale Invariant Feature Transform) feature, HOG (Histogram of Oriented Gradients) feature, LBP (Local Binary Pattern) feature, SURF (Speeded Up Robust Features) feature, ORB (Oriented Fast and Rotated Brief) feature, HAAR feature, or image features extracted based on feature extraction neural network, etc.

[0080] In a second possible implementation, the at least two shooting devices may be devices with a depth acquisition function, such as a depth camera; the obstacle detection and re-identification device may perform three-dimensional reconstruction based on the image and the corresponding depth information to obtain a three-dimensional model, and then extract the three-dimensional feature information at the position indicated by each three-dimensional anchor point from the three-dimensional model. In this embodiment, the image is mapped to a three-dimensional space through three-dimensional reconstruction, so that the three-dimensional information corresponding to the image at the position where the obstacle may exist indicated by the three-dimensional anchor point can be obtained. Since the position indicated by the three-dimensional anchor point is very likely to have an obstacle, obstacle detection is performed based on the three-dimensional information of the image, which is conducive to improving the accuracy of obstacle detection, and the three-dimensional information of the image at other positions not indicated by the three-dimensional anchor point does not need to be processed, which is conducive to improving the efficiency of obstacle detection.

[0081] Exemplarily, when performing three-dimensional reconstruction, the obstacle detection and re-identification device can determine the three-dimensional coordinates of each pixel in the image mapped to the three-dimensional space based on the projection relationship between the image and the three-dimensional mesh model, and the depth information, and then obtain the three-dimensional model based on the three-dimensional coordinates.

[0082] Exemplarily, when obtaining the three-dimensional feature information at the position indicated by each of the three-dimensional anchor points, the obstacle detection and re-identification device can first perform feature extraction on the three-dimensional model to obtain the three-dimensional feature information of the three-dimensional model, and then obtain the three-dimensional feature information at the position indicated by each of the three-dimensional anchor points from the three-dimensional feature information of the three-dimensional model according to each of the three-dimensional anchor points.

[0083] As an example, the three-dimensional feature information includes but is not limited to contour shape features, topological shape features or visual shape features, etc. For example, the contour shape features may include features of vertices and meshes, etc.; the topological shape features include but are not limited to branches or connectivity features of the three-dimensional model, etc.; the visual shape features may include features of the visual image of the three-dimensional model in all directions. As an example, a neural network model for extracting three-dimensional features may be used to extract features of the three-dimensional model.

[0084] After acquiring the feature information, the obstacle detection and re-identification device may perform obstacle detection based on the feature information. In a possible implementation, the obstacle detection and re-identification device may input the feature information into a pre-trained second model, perform obstacle detection using the second model, and acquire obstacle detection information; wherein the obstacle detection information includes at least one or more of the following: obstacle category, size, orientation, two-dimensional coordinates, and three-dimensional coordinates, etc.

[0085] The second model can be trained based on a number of training samples, and the training samples include feature information extracted from image samples at locations where obstacles may exist. Here, supervised training is taken as an example, and the training samples also correspond to real information of obstacles. During the training process, the training samples can be input into the second model so that the second model can identify obstacles according to the feature information, thereby outputting obstacle prediction information, and then adjusting the parameters of the second model according to the difference between the obstacle prediction information and the real obstacle information, thereby obtaining a trained second model, and the second model is used to identify obstacles according to the feature information of the image.

[0086] Exemplarily, the first model is used to predict a three-dimensional anchor point from a three-dimensional grid model according to an image, and the second model is used to detect obstacles according to feature information at a position indicated by the three-dimensional anchor point extracted from the image. The first model and the second model are associated with each other. Therefore, in order to further improve the accuracy of obstacle detection, a joint training method can be used to train the first model and the second model with an associated relationship. For example, the training samples include image samples, three-dimensional grid model samples based on a movable platform, and feature information at a position indicated by a three-dimensional anchor point extracted from the image sample (the three-dimensional anchor point is obtained from the three-dimensional grid model sample). Here, a supervised training method is taken as an example, and the training samples also correspond to the true value of the three-dimensional anchor point and the true information of the obstacle. During the training process, the image samples and the three-dimensional grid model samples based on a movable platform can be trained together. The three-dimensional mesh model samples of the platform are input into the first model, so that the first model can predict the position where the obstacle may exist in the three-dimensional mesh model samples according to the image samples, thereby outputting the three-dimensional anchor point prediction value; and the feature information at the position indicated by the three-dimensional anchor point extracted from the image sample is input into the second model, so that the second model can identify the obstacle according to the feature information, thereby outputting the obstacle prediction information; and then the parameters of the first model and the second model can be jointly adjusted based on the difference between the three-dimensional anchor point prediction value and the true value of the three-dimensional anchor point, and the difference between the obstacle prediction information and the true obstacle information, so that the first model and the second model with correlation can assist and influence each other, so as to achieve the purpose of joint training, which is helpful to further improve the accuracy of obstacle detection.

[0087] While obtaining the obstacle detection information, in step S104, the obstacle detection and re-identification device can generate a re-identification result based on the three-dimensional anchor point pointed to by the feature information corresponding to at least two images. It can be said that if the at least two feature information corresponding to the at least two images point to the same three-dimensional anchor point, indicating that they point to the same three-dimensional spatial position, then it is determined that the obstacle detection information obtained by the at least two feature information belongs to the same obstacle. In this embodiment, both the obstacle detection process and the obstacle re-identification process are performed in an overall framework. While obtaining the obstacle detection information, the re-identification result is also determined based on the three-dimensional anchor point. The obstacle detection process and the obstacle re-identification process are completed at the same time, which is conducive to improving the detection and identification efficiency of obstacles, thereby meeting the real-time requirements in certain scenarios.

[0088] Furthermore, in order to improve the accuracy of the re-identification result, after obtaining the obstacle detection information, the obstacle detection and re-identification device can generate a re-identification result based on the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images and the obstacle detection information; specifically, if the at least two feature information corresponding to the at least two images point to the same three-dimensional anchor point, indicating that they point to the same three-dimensional spatial position, and the difference between the obstacle detection information obtained by the at least two feature information satisfies a preset condition, it can be further accurately determined that the obstacle detection information obtained by the at least two feature information belongs to the same obstacle. This embodiment is conducive to improving the accuracy of the re-identification result through the joint judgment of the three-dimensional anchor point and the obstacle detection information.

[0089] It can be understood that the preset condition can be specifically set according to the actual application scenario. For example, the preset condition can be that the similarity between at least two obstacle detection information obtained respectively by the at least two feature information is greater than a preset threshold. The preset threshold can be flexibly set according to the actual application scenario.

[0090] Exemplarily, after obtaining the obstacle detection information and the re-identification result, the obstacle detection information and the re-identification result may be used to make an obstacle avoidance decision or perform route planning.

[0091] Exemplarily, the obstacle detection and re-identification device may display the obstacle detection information and the re-identification result on the interface of the mobile platform or the interface of a terminal communicatively connected to the mobile platform, so as to allow the user to understand the environment of the mobile platform.

[0092] For further information, see Figure 7 , Figure 7 by Figure 3Taking the obtained images 11 and 12 as an example, the at least two images can be displayed on the interface of the mobile platform or the interface of the terminal device connected to the mobile platform for communication; wherein the obstacle detection information and the re-identification result are displayed in the at least two images, see Figure 7 , the identified obstacles are marked with white candidate boxes, and the re-identification results of the same obstacle in the marked images 11 and 12 are combined with the obstacle detection information and the re-identification results with the actual scene, so that the user can further understand the movement of the movable platform in the actual scene.

[0093] Exemplarily, the obstacle detection and re-identification device may transmit the obstacle detection information and the re-identification result to other components in the movable platform, so that the other components control the movable platform to operate safely and reliably based on the obstacle detection information and the re-identification result.

[0094] Accordingly, see Figure 8 , the embodiment of the present application also provides an obstacle detection and re-identification method, including:

[0095] In step S201, at least two images respectively captured by at least two shooting devices disposed on a movable platform are obtained, and a three-dimensional mesh model generated based on position information of the movable platform is obtained.

[0096] In step S202, one or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the at least two images, and the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located.

[0097] In step S203, feature information at the position indicated by each of the three-dimensional anchor points is extracted from each of the images, and obstacle detection is performed based on the feature information to obtain obstacle detection information.

[0098] In step S204, a re-identification result is generated according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0099] In one embodiment, determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises:

[0100] The at least two images and the three-dimensional network model are input into a pre-trained first model, and the one or more three-dimensional anchor points are obtained using the first model; the first model is used to predict the positions of possible obstacles in the three-dimensional mesh model based on the at least two images to determine the three-dimensional anchor points.

[0101] In one embodiment, determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises:

[0102] generating one or more first two-dimensional anchor points based on each of the images; the first two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located;

[0103] According to the projection relationship between the image and the three-dimensional mesh model, the one or more first two-dimensional anchor points are projected to the three-dimensional space where the three-dimensional mesh model is located to obtain the one or more three-dimensional anchor points.

[0104] In one embodiment, the projection relationship between the image and the three-dimensional grid model is determined based on the positional relationship between the movable platform and the shooting device, and the external parameters and internal parameters of the shooting device.

[0105] In one embodiment, generating one or more first two-dimensional anchor points based on each of the images includes:

[0106] Feature extraction is performed on each of the images, and the one or more first two-dimensional anchor points are determined according to the extracted features.

[0107] In one embodiment, the step of acquiring at least two images respectively captured by at least two shooting devices disposed on the movable platform includes:

[0108] Acquire a plurality of images captured by the at least two photographing devices in a continuous time sequence;

[0109] The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises:

[0110] Performing moving target detection according to the plurality of images to obtain movement information of obstacles; and obtaining movement information of the movable platform in the continuous time series;

[0111] One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the motion information of the obstacle and the motion information of the movable platform.

[0112] In one embodiment, the feature information includes two-dimensional feature information and / or three-dimensional feature information.

[0113] In one embodiment, extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images includes:

[0114] Based on the projection relationship between the image and the three-dimensional grid model, project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located to obtain one or more second two-dimensional anchor points; the second two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located;

[0115] The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is extracted from the image.

[0116] In one embodiment, extracting the two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points from the image includes:

[0117] Performing feature extraction on the image to obtain two-dimensional feature information of the image;

[0118] The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is acquired from the two-dimensional feature information of the image according to each of the second two-dimensional anchor points.

[0119] In one embodiment, the at least two shooting devices are devices with a depth acquisition function.

[0120] In one embodiment, extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images includes:

[0121] Perform three-dimensional reconstruction based on the image and the corresponding depth information to obtain a three-dimensional model;

[0122] The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is extracted from the three-dimensional model.

[0123] In one embodiment, extracting the three-dimensional feature information at the position indicated by each three-dimensional anchor point from the three-dimensional model includes:

[0124] Performing feature extraction on the three-dimensional model to obtain three-dimensional feature information of the three-dimensional model;

[0125] The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is acquired from the three-dimensional feature information of the three-dimensional model according to each of the three-dimensional anchor points.

[0126] In one embodiment, performing obstacle detection according to the feature information to obtain obstacle detection information includes: inputting the feature information into a pre-trained second model, performing obstacle detection using the second model, and obtaining obstacle detection information.

[0127] In one embodiment, the obstacle detection information includes at least one or more of the following: obstacle type, size, orientation, two-dimensional coordinates, and three-dimensional coordinates.

[0128] In one embodiment, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively includes:

[0129] If the at least two pieces of feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, it is determined that the obstacle detection information respectively obtained from the at least two pieces of feature information belongs to the same obstacle.

[0130] In one embodiment, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively includes:

[0131] A re-identification result is generated according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images respectively and the obstacle detection information.

[0132] In one embodiment, generating a re-identification result according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images and the obstacle detection information includes:

[0133] If the at least two feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, and the difference between the obstacle detection information respectively obtained from the at least two feature information satisfies a preset condition, it is determined that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

[0134] In one embodiment, the obstacle detection information and the re-identification result are used to make obstacle avoidance decisions or move route planning for the movable platform.

[0135] In one embodiment, the further embodiment includes: displaying the obstacle detection information and the re-identification result on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform.

[0136] In one embodiment, it also includes: displaying the at least two images on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform; wherein the obstacle detection information and the re-identification result are displayed in the at least two images.

[0137] In one embodiment, the at least two images respectively captured by at least two photographing devices have a preset overlap rate.

[0138] In one embodiment, the position information of the movable platform is obtained by a satellite locator disposed on the movable platform.

[0139] In one embodiment, the three-dimensional grid model is generated based on the position information of the movable platform and preset grid parameters; the grid parameters at least indicate the size and / or shape of the three-dimensional grid in the three-dimensional grid model.

[0140] Accordingly, see Fig. 9 , the embodiment of the present application further provides an obstacle detection and re-identification device 30, including a processor 31 and a memory 32 storing a computer program;

[0141] The processor 31 implements the following steps when executing the computer program:

[0142] Acquire at least two images respectively captured by at least two shooting devices disposed on a movable platform, and acquire a three-dimensional grid model generated based on position information of the movable platform;

[0143] Determine one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located;

[0144] Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information;

[0145] And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0146] In one embodiment, the processor 31 is specifically used to: input the at least two images and the three-dimensional network model into a pre-trained first model, and use the first model to obtain the one or more three-dimensional anchor points; the first model is used to predict the location of possible obstacles in the three-dimensional grid model based on the at least two images to determine the three-dimensional anchor points.

[0147] In one embodiment, the processor 31 is specifically used to: generate one or more first two-dimensional anchor points based on each of the images; the first two-dimensional anchor points are used to indicate the locations where obstacles may exist in the two-dimensional space where the image is located; and according to the projection relationship between the image and the three-dimensional mesh model, project the one or more first two-dimensional anchor points to the three-dimensional space where the three-dimensional mesh model is located to obtain the one or more three-dimensional anchor points.

[0148] In one embodiment, the projection relationship between the image and the three-dimensional grid model is determined based on the positional relationship between the movable platform and the shooting device, and the external parameters and internal parameters of the shooting device.

[0149] In one embodiment, the processor 31 is further configured to: extract features from each of the images, and determine the one or more first two-dimensional anchor points according to the extracted features.

[0150] In one embodiment, the processor 31 is further configured to:

[0151] Acquire a plurality of images captured by the at least two photographing devices in a continuous time sequence;

[0152] Performing moving target detection according to the plurality of images to obtain movement information of obstacles; and obtaining movement information of the movable platform in the continuous time series;

[0153] One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the motion information of the obstacle and the motion information of the movable platform.

[0154] In one embodiment, the feature information includes two-dimensional feature information and / or three-dimensional feature information.

[0155] In one embodiment, the processor 31 is also used to: based on the projection relationship between the image and the three-dimensional mesh model, project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located to obtain one or more second two-dimensional anchor points; the second two-dimensional anchor points are used to indicate the positions where obstacles may exist in the two-dimensional space where the image is located; and extract the two-dimensional feature information at the positions indicated by each of the second two-dimensional anchor points from the image.

[0156] In one embodiment, the processor 31 is further used to: perform feature extraction on the image to obtain two-dimensional feature information of the image; and obtain two-dimensional feature information at the position indicated by each second two-dimensional anchor point from the two-dimensional feature information of the image according to each second two-dimensional anchor point.

[0157] In one embodiment, the at least two shooting devices are devices with a depth acquisition function.

[0158] In one embodiment, the processor 31 is further used to: perform three-dimensional reconstruction according to the image and the corresponding depth information to obtain a three-dimensional model; and extract three-dimensional feature information at the position indicated by each three-dimensional anchor point from the three-dimensional model.

[0159] In one embodiment, the processor 31 is further used to: perform feature extraction on the three-dimensional model to obtain three-dimensional feature information of the three-dimensional model; and obtain three-dimensional feature information at the position indicated by each three-dimensional anchor point from the three-dimensional feature information of the three-dimensional model according to each three-dimensional anchor point.

[0160] In one embodiment, the processor 31 is further used to: input the feature information into a pre-trained second model, perform obstacle detection using the second model, and obtain obstacle detection information.

[0161] In one embodiment, the processor 31 is further configured to: the obstacle detection information includes at least one or more of the following: obstacle category, size, orientation, two-dimensional coordinates, and three-dimensional coordinates.

[0162] In one embodiment, the processor 31 is further used to: if the at least two feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, determine that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

[0163] In one embodiment, the processor 31 is further configured to generate a re-identification result according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images and the obstacle detection information.

[0164] In one embodiment, the processor 31 is specifically used to: if the at least two feature information respectively corresponding to the at least two images point to the same three-dimensional anchor point, and the difference between the obstacle detection information respectively obtained from the at least two feature information satisfies a preset condition, determine that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

[0165] In one embodiment, the obstacle detection information and the re-identification result are used to make obstacle avoidance decisions or move route planning for the movable platform.

[0166] In one embodiment, the processor 31 is further configured to: display the obstacle detection information and the re-identification result on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform.

[0167] In one embodiment, the processor 31 is further used to: display the at least two images on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform; wherein the obstacle detection information and the re-identification result are displayed in the at least two images.

[0168] In one embodiment, the at least two images respectively captured by at least two photographing devices have a preset overlap rate.

[0169] In one embodiment, the position information of the movable platform is obtained by a satellite locator disposed on the movable platform.

[0170] In one embodiment, the three-dimensional grid model is generated based on the position information of the movable platform and preset grid parameters; the grid parameters at least indicate the size and / or shape of the three-dimensional grid in the three-dimensional grid model.

[0171] Accordingly, an embodiment of the present application further provides an obstacle detection and re-identification device, comprising a processor and a memory storing a computer program;

[0172] When executing the computer program, the processor implements the following steps:

[0173] Acquire a three-dimensional grid model generated based on the position information of the movable platform, and determine one or more three-dimensional anchor points from the three-dimensional grid model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located;

[0174] Acquire at least two images respectively captured by at least two shooting devices disposed on the movable platform;

[0175] Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information;

[0176] And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

[0177] In one embodiment, the processor is further configured to: calculate the probability of obstacles existing at various positions in the three-dimensional grid model according to historical data to determine the one or more three-dimensional anchor points.

[0178] In one embodiment, the three-dimensional anchor point indicates a position in the three-dimensional grid model where the probability of an obstacle existing therein is greater than a preset threshold.

[0179] In one embodiment, the processor is further configured to: determine one or more three-dimensional anchor points from the three-dimensional mesh model according to attribute information of the movable platform and / or attribute information of the obstacle.

[0180] In one embodiment, the attribute information of the movable platform includes one or more of the following: the movement mode, type, size of the movable platform and the installation position of the shooting device on the movable platform; and / or, the attribute information of the obstacle includes one or more of the following: the type, size and shape of the obstacle.

[0181] In one embodiment, the processor is further configured to: determine corresponding three-dimensional anchor points according to respective positions in the three-dimensional mesh model.

[0182] In one embodiment, the processor is further configured to: determine one or more three-dimensional anchor points from the three-dimensional mesh model based on the at least two images.

[0183] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0184] Various embodiments described herein can be implemented using computer readable media such as computer software, hardware or any combination thereof. For hardware implementation, the embodiments described herein can be implemented using at least one of application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and electronic units designed to perform the functions described herein. For software implementation, embodiments such as processes or functions can be implemented with separate software modules that allow execution of at least one function or operation. Software code can be implemented by a software application (or program) written in any appropriate programming language, and the software code can be stored in a memory and executed by a controller.

[0185] Accordingly, see Fig.10 The embodiment of the present application also provides a movable platform 100, including: a body 101; a power system 102, installed in the body 101, for providing power for the movable platform 100; and the above-mentioned obstacle detection and re-identification device 30.

[0186] Optionally, the movable platform 100 is a vehicle, a drone, an unmanned ship or a movable robot.

[0187] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0188] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the above method.

[0189] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0190] The method and device provided in the embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An obstacle detection and re-identification method, characterized in that: include: Acquire at least two images respectively captured by at least two shooting devices disposed on a movable platform, and acquire a three-dimensional grid model generated based on position information of the movable platform; Determine one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located; Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information; And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

2. The method according to claim 1, characterized in that The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises: The at least two images and the three-dimensional mesh model are input into a pre-trained first model, and the one or more three-dimensional anchor points are obtained using the first model; the first model is used to predict the locations of possible obstacles in the three-dimensional mesh model based on the at least two images to determine the three-dimensional anchor points.

3. The method according to claim 1, characterized in that The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises: generating one or more first two-dimensional anchor points based on each of the images; the first two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; According to the projection relationship between the image and the three-dimensional mesh model, the one or more first two-dimensional anchor points are projected to the three-dimensional space where the three-dimensional mesh model is located to obtain the one or more three-dimensional anchor points.

4. The method according to claim 3, characterized in that: The projection relationship between the image and the three-dimensional grid model is determined based on the positional relationship between the movable platform and the shooting device, and the external parameters and internal parameters of the shooting device.

5. The method according to claim 3, characterized in that: The generating one or more first two-dimensional anchor points based on each of the images comprises: Feature extraction is performed on each of the images, and the one or more first two-dimensional anchor points are determined according to the extracted features.

6. The method according to claim 1, characterized in that The step of acquiring at least two images respectively captured by at least two shooting devices disposed on the movable platform comprises: Acquire a plurality of images captured by the at least two photographing devices in a continuous time sequence; The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises: Performing moving target detection according to the plurality of images to obtain movement information of obstacles; and obtaining movement information of the movable platform in the continuous time series; One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the motion information of the obstacle and the motion information of the movable platform.

7. The method according to claim 1, characterized in that The feature information includes two-dimensional feature information and / or three-dimensional feature information.

8. The method according to claim 1, characterized in that: The step of extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images comprises: Based on the projection relationship between the image and the three-dimensional grid model, project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located to obtain one or more second two-dimensional anchor points; the second two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is extracted from the image.

9. The method according to claim 8, characterized in that The extracting the two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points from the image includes: Performing feature extraction on the image to obtain two-dimensional feature information of the image; The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is acquired from the two-dimensional feature information of the image according to each of the second two-dimensional anchor points.

10. The method according to claim 1, characterized in that The at least two shooting devices are devices with depth acquisition function.

11. The method according to claim 10, characterized in that The step of extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images comprises: Perform three-dimensional reconstruction based on the image and the corresponding depth information to obtain a three-dimensional model; The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is extracted from the three-dimensional model.

12. The method according to claim 11, characterized in that The extracting the three-dimensional feature information at the position indicated by each three-dimensional anchor point from the three-dimensional model includes: Performing feature extraction on the three-dimensional model to obtain three-dimensional feature information of the three-dimensional model; The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is acquired from the three-dimensional feature information of the three-dimensional model according to each of the three-dimensional anchor points.

13. The method according to claim 1, characterized in that The performing obstacle detection according to the feature information to obtain obstacle detection information includes: The feature information is input into a pre-trained second model, and the second model is used to perform obstacle detection to obtain obstacle detection information.

14. The method according to claim 1, characterized in that The obstacle detection information includes at least one or more of the following: obstacle type, size, direction, two-dimensional coordinates, and three-dimensional coordinates.

15. The method according to claim 1, characterized in that The generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively includes: If the at least two pieces of feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, it is determined that the obstacle detection information respectively obtained from the at least two pieces of feature information belongs to the same obstacle.

16. The method according to claim 1, characterized in that The generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively includes: A re-identification result is generated according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images respectively and the obstacle detection information.

17. The method according to claim 16, characterized in that The generating a re-identification result according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images and the obstacle detection information includes: If the at least two feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, and the difference between the obstacle detection information respectively obtained from the at least two feature information satisfies a preset condition, it is determined that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

18. The method according to claim 1, characterized in that The obstacle detection information and the re-identification result are used to make obstacle avoidance decisions or move route planning for the movable platform.

19. The method according to claim 1, characterized in that Also includes: The obstacle detection information and the re-identification result are displayed on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform.

20. The method according to claim 1 or 19, characterized in that Also includes: The at least two images are displayed on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform; wherein the obstacle detection information and the re-identification result are displayed in the at least two images.

21. The method according to claim 1, characterized in that The at least two images respectively captured by at least two shooting devices have a preset overlap rate.

22. The method according to claim 1, characterized in that The position information of the movable platform is obtained by a satellite locator arranged on the movable platform.

23. The method according to claim 1, characterized in that The three-dimensional grid model is generated based on the position information of the movable platform and preset grid parameters; the grid parameters at least indicate the size and / or shape of the three-dimensional grid in the three-dimensional grid model.

24. An obstacle detection and re-identification method, characterized in that: include: Acquire a three-dimensional grid model generated based on the position information of the movable platform, and determine one or more three-dimensional anchor points from the three-dimensional grid model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located; Acquire at least two images respectively captured by at least two shooting devices disposed on the movable platform; Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information; And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

25. The method according to claim 24, characterized in that The step of determining one or more three-dimensional anchor points from the three-dimensional mesh model comprises: The probability of obstacles existing at various positions in the three-dimensional grid model is statistically analyzed according to historical data to determine the one or more three-dimensional anchor points.

26. The method according to claim 25, characterized in that The three-dimensional anchor point indicates a position in the three-dimensional grid model where the probability of an obstacle existing therein is greater than a preset threshold.

27. The method according to claim 24, characterized in that The step of determining one or more three-dimensional anchor points from the three-dimensional mesh model comprises: One or more three-dimensional anchor points are determined from the three-dimensional grid model according to the attribute information of the movable platform and / or the attribute information of the obstacle.

28. The method according to claim 27, characterized in that The attribute information of the movable platform includes one or more of the following: the moving mode, type, size of the movable platform and the installation position of the shooting device on the movable platform; And / or, the attribute information of the obstacle includes one or more of the following: type, size and shape of the obstacle.

29. The method according to claim 24, characterized in that The step of determining one or more three-dimensional anchor points from the three-dimensional mesh model comprises: The corresponding three-dimensional anchor point is determined according to each position in the three-dimensional grid model.

30. The method according to claim 24, characterized in that The step of determining one or more three-dimensional anchor points from the three-dimensional mesh model comprises: One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the at least two images.

31. The method according to claim 30, characterized in that The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises: The at least two images and the three-dimensional mesh model are input into a pre-trained first model, and the one or more three-dimensional anchor points are obtained using the first model; the first model is used to predict the locations of possible obstacles in the three-dimensional mesh model based on the at least two images to determine the three-dimensional anchor points.

32. The method according to claim 30, characterized in that The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises: generating one or more first two-dimensional anchor points based on each of the images; the first two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; According to the projection relationship between the image and the three-dimensional mesh model, the one or more first two-dimensional anchor points are projected to the three-dimensional space where the three-dimensional mesh model is located to obtain the one or more three-dimensional anchor points.

33. The method according to claim 30, characterized in that The step of acquiring at least two images respectively captured by at least two shooting devices disposed on the movable platform comprises: Acquire a plurality of images captured by the at least two photographing devices in a continuous time sequence; The determining one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images comprises: Performing moving target detection according to the plurality of images to obtain movement information of obstacles; and obtaining movement information of the movable platform in the continuous time series; One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the motion information of the obstacle and the motion information of the movable platform.

34. The method according to claim 24, characterized in that The feature information includes two-dimensional feature information and / or three-dimensional feature information.

35. The method according to claim 24, characterized in that The step of extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images comprises: Based on the projection relationship between the image and the three-dimensional grid model, project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located to obtain one or more second two-dimensional anchor points; the second two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is extracted from the image.

36. The method according to claim 24, characterized in that The at least two shooting devices are devices with depth acquisition function.

37. The method according to claim 36, characterized in that The step of extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images comprises: Perform three-dimensional reconstruction based on the image and the corresponding depth information to obtain a three-dimensional model; The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is extracted from the three-dimensional model.

38. The method according to claim 24, characterized in that The performing obstacle detection according to the feature information to obtain obstacle detection information includes: The feature information is input into a pre-trained second model, and the second model is used to perform obstacle detection to obtain obstacle detection information.

39. The method according to claim 24, characterized in that The obstacle detection information includes at least one or more of the following: obstacle type, size, direction, two-dimensional coordinates, and three-dimensional coordinates.

40. The method according to claim 24, characterized in that The generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively includes: If the at least two pieces of feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, it is determined that the obstacle detection information respectively obtained from the at least two pieces of feature information belongs to the same obstacle.

41. The method according to claim 24, characterized in that The generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively includes: A re-identification result is generated according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images respectively and the obstacle detection information.

42. The method according to claim 41, characterized in that The generating a re-identification result according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images and the obstacle detection information includes: If the at least two feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, and the difference between the obstacle detection information respectively obtained from the at least two feature information satisfies a preset condition, it is determined that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

43. The method according to claim 24, characterized in that The obstacle detection information and the re-identification result are used to make obstacle avoidance decisions or move route planning for the movable platform.

44. The method according to claim 24, characterized in that Also includes: The obstacle detection information and the re-identification result are displayed on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform.

45. The method according to claim 24 or 44, characterized in that Also includes: The at least two images are displayed on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform; wherein the obstacle detection information and the re-identification result are displayed in the at least two images.

46. ​​The method according to claim 24, characterized in that The position information of the movable platform is obtained by a satellite locator arranged on the movable platform.

47. An obstacle detection and re-identification device, characterized in that: comprising a processor and a memory storing a computer program; When executing the computer program, the processor implements the following steps: Acquire at least two images respectively captured by at least two shooting devices disposed on a movable platform, and acquire a three-dimensional grid model generated based on position information of the movable platform; Determine one or more three-dimensional anchor points from the three-dimensional mesh model according to the at least two images, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional mesh model is located; Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information; And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

48. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is specifically configured to: input the at least two images and the three-dimensional mesh model into a pre-trained first model, and obtain the one or more three-dimensional anchor points using the first model; The first model is used to predict the location of possible obstacles in the three-dimensional grid model based on the at least two images to determine the three-dimensional anchor point.

49. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is specifically used to: generate one or more first two-dimensional anchor points based on each of the images; the first two-dimensional anchor points are used to indicate the locations where obstacles may exist in the two-dimensional space where the image is located; and project the one or more first two-dimensional anchor points to the three-dimensional space where the three-dimensional mesh model is located according to the projection relationship between the image and the three-dimensional mesh model to obtain the one or more three-dimensional anchor points.

50. The obstacle detection and re-identification device according to claim 49, characterized in that: The projection relationship between the image and the three-dimensional grid model is determined based on the positional relationship between the movable platform and the shooting device, and the external parameters and internal parameters of the shooting device.

51. The obstacle detection and re-identification device according to claim 49, characterized in that: The processor is further configured to: extract features from each of the images, and determine the one or more first two-dimensional anchor points according to the extracted features.

52. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is further configured to: Acquire a plurality of images captured by the at least two photographing devices in a continuous time sequence; Performing moving target detection according to the multiple images to obtain movement information of obstacles; and, obtaining motion information of the movable platform in the continuous time series; One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the motion information of the obstacle and the motion information of the movable platform.

53. The obstacle detection and re-identification device according to claim 47, characterized in that: The feature information includes two-dimensional feature information and / or three-dimensional feature information.

54. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is further configured to: Based on the projection relationship between the image and the three-dimensional grid model, project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located to obtain one or more second two-dimensional anchor points; the second two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is extracted from the image.

55. The obstacle detection and re-identification device according to claim 54, characterized in that: The processor is further configured to: perform feature extraction on the image to obtain two-dimensional feature information of the image; and obtain two-dimensional feature information at positions indicated by each of the second two-dimensional anchor points from the two-dimensional feature information of the image according to each of the second two-dimensional anchor points.

56. The obstacle detection and re-identification device according to claim 47, characterized in that: The at least two shooting devices are devices with depth acquisition function.

57. The obstacle detection and re-identification device according to claim 56, characterized in that: The processor is further configured to: Perform three-dimensional reconstruction based on the image and the corresponding depth information to obtain a three-dimensional model; The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is extracted from the three-dimensional model.

58. The obstacle detection and re-identification device according to claim 57, characterized in that: The processor is further configured to: Performing feature extraction on the three-dimensional model to obtain three-dimensional feature information of the three-dimensional model; The three-dimensional feature information at the position indicated by each of the three-dimensional anchor points is acquired from the three-dimensional feature information of the three-dimensional model according to each of the three-dimensional anchor points.

59. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is also used to: input the feature information into a pre-trained second model, use the second model to perform obstacle detection, and obtain obstacle detection information.

60. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is further configured to: the obstacle detection information includes at least one or more of the following: obstacle category, size, orientation, two-dimensional coordinates, and three-dimensional coordinates.

61. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is further configured to: if the at least two pieces of feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, determine that the obstacle detection information respectively obtained from the at least two pieces of feature information belongs to the same obstacle.

62. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is further configured to generate a re-identification result according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images respectively and the obstacle detection information.

63. The obstacle detection and re-identification device according to claim 62, characterized in that: The processor is specifically used to: if at least two feature information corresponding to the at least two images respectively point to the same three-dimensional anchor point, and the difference between the obstacle detection information respectively obtained from the at least two feature information satisfies a preset condition, determine that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

64. The obstacle detection and re-identification device according to claim 47, characterized in that: The obstacle detection information and the re-identification result are used to make obstacle avoidance decisions or move route planning for the movable platform.

65. The obstacle detection and re-identification device according to claim 47, characterized in that: The processor is further configured to display the obstacle detection information and the re-identification result on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform.

66. The obstacle detection and re-identification device according to claim 47 or 65, characterized in that: The processor is also used to: display the at least two images on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform; wherein the obstacle detection information and the re-identification result are displayed in the at least two images.

67. The obstacle detection and re-identification device according to claim 47, characterized in that: The at least two images respectively captured by at least two shooting devices have a preset overlap rate.

68. The obstacle detection and re-identification device according to claim 47, characterized in that: The position information of the movable platform is obtained by a satellite locator arranged on the movable platform.

69. The obstacle detection and re-identification device according to claim 47, characterized in that: The three-dimensional grid model is generated based on the position information of the movable platform and preset grid parameters; the grid parameters at least indicate the size and / or shape of the three-dimensional grid in the three-dimensional grid model.

70. An obstacle detection and re-identification device, characterized in that: comprising a processor and a memory storing a computer program; When executing the computer program, the processor implements the following steps: Acquire a three-dimensional grid model generated based on the position information of the movable platform, and determine one or more three-dimensional anchor points from the three-dimensional grid model, wherein the three-dimensional anchor points are used to indicate positions where obstacles may exist in the three-dimensional space where the three-dimensional grid model is located; Acquire at least two images respectively captured by at least two shooting devices disposed on the movable platform; Extracting feature information at the position indicated by each of the three-dimensional anchor points from each of the images, and performing obstacle detection according to the feature information to obtain obstacle detection information; And, generating a re-identification result according to the three-dimensional anchor points pointed to by the feature information corresponding to the at least two images respectively.

71. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to: statistically analyze the probability of obstacles existing at various positions in the three-dimensional grid model according to historical data, so as to determine the one or more three-dimensional anchor points.

72. The obstacle detection and re-identification device according to claim 71, characterized in that: The three-dimensional anchor point indicates a position in the three-dimensional grid model where the probability of an obstacle existing therein is greater than a preset threshold.

73. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to determine one or more three-dimensional anchor points from the three-dimensional grid model according to the attribute information of the movable platform and / or the attribute information of the obstacle.

74. The obstacle detection and re-identification device according to claim 73, characterized in that: The attribute information of the movable platform includes one or more of the following: the moving mode, type, size of the movable platform and the installation position of the shooting device on the movable platform; And / or, the attribute information of the obstacle includes one or more of the following: type, size and shape of the obstacle.

75. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to determine corresponding three-dimensional anchor points according to respective positions in the three-dimensional grid model.

76. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to determine one or more three-dimensional anchor points from the three-dimensional mesh model based on the at least two images.

77. The obstacle detection and re-identification device according to claim 76, characterized in that: The processor is further configured to: input the at least two images and the three-dimensional mesh model into a pre-trained first model, and obtain the one or more three-dimensional anchor points using the first model; The first model is used to predict the location of possible obstacles in the three-dimensional grid model based on the at least two images to determine the three-dimensional anchor point.

78. The obstacle detection and re-identification device according to claim 76, characterized in that: The processor is further configured to: generating one or more first two-dimensional anchor points based on each of the images; the first two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; According to the projection relationship between the image and the three-dimensional mesh model, the one or more first two-dimensional anchor points are projected to the three-dimensional space where the three-dimensional mesh model is located to obtain the one or more three-dimensional anchor points.

79. The obstacle detection and re-identification device according to claim 76, characterized in that: The processor is further configured to: Acquire a plurality of images captured by the at least two photographing devices in a continuous time sequence; Performing moving target detection according to the multiple images to obtain movement information of obstacles; and, obtaining motion information of the movable platform in the continuous time series; One or more three-dimensional anchor points are determined from the three-dimensional mesh model according to the motion information of the obstacle and the motion information of the movable platform.

80. The obstacle detection and re-identification device according to claim 70, characterized in that: The feature information includes two-dimensional feature information and / or three-dimensional feature information.

81. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to: Based on the projection relationship between the image and the three-dimensional grid model, project one or more of the three-dimensional anchor points into the two-dimensional space where the image is located to obtain one or more second two-dimensional anchor points; the second two-dimensional anchor points are used to indicate positions where obstacles may exist in the two-dimensional space where the image is located; The two-dimensional feature information at the position indicated by each of the second two-dimensional anchor points is extracted from the image.

82. The obstacle detection and re-identification device according to claim 70, characterized in that: The at least two shooting devices are devices with depth acquisition function.

83. The obstacle detection and re-identification device according to claim 82, characterized in that: The processor is also used to: perform three-dimensional reconstruction according to the image and the corresponding depth information to obtain a three-dimensional model; and extract three-dimensional feature information at the position indicated by each three-dimensional anchor point from the three-dimensional model.

84. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is also used to: input the feature information into a pre-trained second model, use the second model to perform obstacle detection, and obtain obstacle detection information.

85. The obstacle detection and re-identification device according to claim 70, characterized in that: The obstacle detection information includes at least one or more of the following: obstacle type, size, direction, two-dimensional coordinates, and three-dimensional coordinates.

86. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to: if the at least two pieces of feature information respectively corresponding to the at least two images both point to the same three-dimensional anchor point, determine that the obstacle detection information respectively obtained from the at least two pieces of feature information belongs to the same obstacle.

87. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to generate a re-identification result according to the three-dimensional anchor point pointed to by the feature information corresponding to the at least two images respectively and the obstacle detection information.

88. The obstacle detection and re-identification device according to claim 87, characterized in that: The processor is also used to: if the at least two feature information corresponding to the at least two images respectively point to the same three-dimensional anchor point, and the difference between the obstacle detection information respectively obtained from the at least two feature information satisfies a preset condition, determine that the obstacle detection information respectively obtained from the at least two feature information belongs to the same obstacle.

89. The obstacle detection and re-identification device according to claim 70, characterized in that: The obstacle detection information and the re-identification result are used to make obstacle avoidance decisions or move route planning for the movable platform.

90. The obstacle detection and re-identification device according to claim 70, characterized in that: The processor is further configured to display the obstacle detection information and the re-identification result on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform.

91. The obstacle detection and re-identification device according to claim 70 or 90, characterized in that: The processor is also used to: display the at least two images on an interface of the mobile platform or an interface of a terminal device that is communicatively connected to the mobile platform; wherein the obstacle detection information and the re-identification result are displayed in the at least two images.

92. The obstacle detection and re-identification device according to claim 70, characterized in that: The position information of the movable platform is obtained by a satellite locator arranged on the movable platform.

93. A movable platform, characterized in that: include: Body; A power system, installed in the body, for providing power to the movable platform; At least two camera devices for capturing images; as well as, An obstacle detection and re-identification device as described in any one of claims 47 to 69 or any one of claims 70 to 92.

94. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 23 or any one of 24 to 46 is implemented.

Images