Obstacle grounding position identification method and device, equipment and storage medium

Abstract

The invention provides an obstacle grounding position identification method and device, a vehicle and a storage medium. The method comprises the following steps: extracting image features from a panoramic image; a coordinate mapping relation matrix is constructed, the coordinate mapping relation refers to the mapping relation between a first coordinate system and a second coordinate system, the first coordinate system is used for representing the position features from the first dimension, and the second coordinate system is used for representing the position features from the second dimension; generating coordinate features of the panoramic image according to the image features and the coordinate mapping relation matrix; and identifying the position information of the obstacle grounding area from the coordinate features, wherein the position information comprises the position information of the obstacle grounding area in the first dimension and the position information of the obstacle grounding area in the second dimension. According to the technical scheme provided by the embodiment of the invention, the multi-dimensional position characterization of the obstacle grounding area can be obtained, so that the obtained position characterization is finer in granularity and relatively higher in accuracy.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a method, device, equipment and storage medium for identifying the ground position of an obstacle. Background Art

[0002] During the automatic parking process, the boundary of the vehicle's drivable area (free space) can be identified based on the panoramic image of the vehicle. The boundary of the drivable area is composed of the grounding positions of obstacles in the area where the vehicle is located. Therefore, the main process of identifying the boundary of the drivable area can be to identify the grounding positions of obstacles in the area where the vehicle is located.

[0003] A conventional method for identifying the grounding position of an obstacle can be implemented by mapping a panoramic image to a coordinate system, extracting features from the image that establishes a mapping relationship with the coordinate system, and obtaining image features including coordinate position representations. Thereafter, the grounding position of the obstacle can be determined based on the image features including coordinate position representations.

[0004] The dimension represented by the obstacle grounding position feature depends on the coordinate system of the panoramic image mapping. However, this recognition method makes the dimension represented by the obstacle grounding position feature relatively single, for example, it represents the dimension of the relative angle with the vehicle, resulting in the identified obstacle grounding position feature with relatively large granularity and relatively low accuracy. Summary of the invention

[0005] The present application proposes a method, device, equipment and storage medium for identifying the ground position of an obstacle, which can deploy two coordinate systems, associate image features with the two coordinate systems, and map the image features to different position representation dimensions, so that the obtained position representation has finer granularity and relatively higher accuracy.

[0006] The first embodiment of the present application provides a method for identifying a grounding position of an obstacle, including:

[0007] Extracting image features from the panoramic image, wherein the image features include image features of the obstacle contact area;

[0008] Constructing a coordinate mapping relationship matrix, wherein the coordinate mapping relationship refers to a mapping relationship between a first coordinate system and a second coordinate system, wherein the coordinate mapping relationship matrix includes coordinate mapping relationships of a plurality of position points, wherein the plurality of position points are associated with the first coordinate system, wherein the first coordinate system is used to characterize position features from a first dimension, and wherein the second coordinate system is used to characterize position features from a second dimension;

[0009] Generate coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix, the coordinate features including pixel features of the multiple location points and coordinate mapping relationship features of the multiple location points;

[0010] The position information of the obstacle contact area is identified from the coordinate features, where the position information includes the position information of the obstacle contact area in the first dimension and the position information of the second dimension.

[0011] In some embodiments of the present application, extracting image features of the object to be identified from the panoramic image includes:

[0012] Feature extraction is performed on the panoramic image to obtain at least one image sub-feature, wherein the at least one image sub-feature constitutes the image feature; and the at least one image sub-feature represents a feature of the panoramic image at at least one size.

[0013] In some embodiments of the present application, the step of constructing a coordinate mapping relationship matrix includes:

[0014] Constructing the first coordinate system according to the size of the image feature, the first coordinate system includes M directions, the multiple position points are distributed in the M directions, wherein M is an integer greater than 3;

[0015] Corresponding to any one of the multiple position points, calculating the coordinates of the position point in the first coordinate system to obtain a first coordinate of the position point;

[0016] Calculating the coordinates of the position point in the second coordinate system according to the first coordinates and a mapping relationship between the first coordinate system and the second coordinate system to obtain the second coordinates of the position point;

[0017] The coordinate mapping relationship matrix is ​​generated by taking the second coordinates of the plurality of position points as eigenvalues.

[0018] In some embodiments of the present application, generating the coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix includes:

[0019] Obtain pixel features corresponding to the multiple position points in the image features to obtain pixel features of the multiple position points;

[0020] According to the correspondence between the multiple position points and the eigenvalues ​​in the coordinate mapping relationship matrix, the sorting of each feature point in the pixel feature is deployed so that each feature point is associated with the corresponding eigenvalue in the coordinate mapping relationship matrix, and the sorted pixel feature is the coordinate feature.

[0021] In some embodiments of the present application, identifying the location information of the obstacle touchdown area from the coordinate features includes:

[0022] Encoding the coordinate features, wherein the encoded features include position features of the obstacle contact area, and the position features include position features of the first dimension and position features of the second dimension;

[0023] The position information of the obstacle contact area in the first dimension and the position information of the obstacle contact area in the second dimension are identified according to the position feature of the obstacle contact area.

[0024] In some embodiments of the present application, the encoded feature corresponds to the multiple position points, the multiple position points are respectively deployed in M ​​directions, the M directions are directions included in the first coordinate system, and after encoding the coordinate feature, it also includes:

[0025] Determine the type of the object to be identified corresponding to each direction of the M directions according to the encoded features, the type of the object to be identified being the ground contact area of ​​the obstacle or the edge of the panoramic image;

[0026] If the type of the object to be identified corresponding to any direction is the obstacle touchdown area, the position information of the first dimension and the position information of the second dimension of the obstacle touchdown area are identified.

[0027] In some embodiments of the present application, the first coordinate system is a polar coordinate system, and M is 360; the second coordinate system is a Cartesian coordinate system.

[0028] In some embodiments of the present application, the position information of the first dimension is the angle between the obstacle grounding area and the center position of the panoramic image; the position information of the second dimension is the distance between the obstacle grounding area and the center of the panoramic image.

[0029] In some embodiments of the present application, after identifying the location information of the obstacle touchdown area from the coordinate features, the method further includes:

[0030] Determining, in the panoramic image, an outline of the obstacle contact area component;

[0031] An area enclosed by the outline formed by the obstacle contact area is determined as a drivable area of ​​a vehicle, and the vehicle is a vehicle that collects the panoramic image.

[0032] In some embodiments of the present application, a first convolutional network is called to extract image features from a panoramic image; a second convolutional network is called to construct a coordinate mapping relationship matrix, and coordinate features of the panoramic image are generated based on the image features and the coordinate mapping relationship matrix; and a third convolutional network is called to identify the location information of the obstacle ground area from the coordinate features.

[0033] The embodiment of the second aspect of the present application provides a device for identifying a grounding position of an obstacle, including:

[0034] A feature extraction module, used to extract image features from the panoramic image, wherein the image features include image features of the obstacle ground contact area;

[0035] A construction module, used to construct a coordinate mapping relationship matrix, wherein the coordinate mapping relationship refers to a mapping relationship between a first coordinate system and a second coordinate system, wherein the coordinate mapping relationship matrix includes coordinate mapping relationships between a plurality of position points, wherein the plurality of position points are associated with the first coordinate system, wherein the first coordinate system is used to characterize position features from a first dimension, and wherein the second coordinate system is used to characterize position features from a second dimension;

[0036] A generating module, configured to generate coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix, wherein the coordinate features include pixel features of the plurality of position points and coordinate mapping relationship features of the plurality of position points;

[0037] An identification module is used to identify the position information of the obstacle touchdown area from the coordinate features, wherein the position information includes the position information of the obstacle touchdown area in the first dimension and the position information of the second dimension.

[0038] An embodiment of the third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0039] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the method described in the first aspect above.

[0040] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0041] In an embodiment of the present application, image features can be extracted from a panoramic image, and a coordinate mapping relationship matrix can be constructed. The coordinate mapping relationship matrix represents the mapping relationship between the first coordinate system and the second coordinate system, the first coordinate system is used to represent the position feature from the first dimension, and the second coordinate system is used to represent the position feature from the second dimension. The coordinate mapping relationship matrix can be a coordinate mapping relationship including multiple position points (i.e., sampling points). Afterwards, the coordinate features are generated by the correlation between the image features and the coordinate mapping relationship matrix, and the coordinate features include the pixel features of the multiple position points, and the coordinate mapping relationship features of the multiple position points. That is, the coordinate features include the pixel features of each position point, and the first dimension position features and the second dimension position features of the corresponding pixels. In this way, the position information of the obstacle grounding area identified from the coordinate features can include the position information including the first dimension position information and the second dimension position information of the obstacle grounding area. It can be seen that in the embodiment of the present application, the image features are mapped to different position representation dimensions, so that the position representation of multiple dimensions of the obstacle grounding area can be obtained, so that the obtained position representation has a finer granularity and a relatively higher accuracy.

[0042] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0044] In the attached picture:

[0045] Figure 1 A method flow chart of a method for identifying a grounding position of an obstacle provided by an embodiment of the present application is shown;

[0046] Figure 2A A schematic diagram of the structure of an algorithm network provided by an embodiment of the present application is shown;

[0047] Figure 2B A schematic diagram of data flow based on an algorithm network provided by an embodiment of the present application is shown;

[0048] Figure 3 A flowchart of a method for coordinate system conversion provided by an embodiment of the present application is shown;

[0049] Figure 4 A schematic diagram of the structure of an obstacle grounding position identification device provided by an embodiment of the present application is shown;

[0050] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0051] Figure 6 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0053] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0054] The embodiments of the present application relate to an automatic parking scenario. In the automatic parking scenario, after a vehicle enters an area to be parked, the boundary of the vehicle's drivable area (free space) in the area to be parked can be identified, and a parking control scheme can be planned based on the boundary of the free space. In some embodiments, the automatic parking system can identify the grounding position of an obstacle on an image of an Around View Monitor (AVM), and the boundary formed by the contour line of the grounding position of the obstacle can be considered as the free space boundary.

[0055] Among them, the automatic parking system can call the pre-deployed algorithm network to process the AVM image to obtain the characteristics of the obstacle grounding image contained in the AVM image, and then, the position of the obstacle grounding area can be identified according to the characteristics of the obstacle grounding image.

[0056] It should be noted that the "position" described in the embodiments of the present application may be the relative position of the obstacle contact area and the vehicle, for example, the angle between the obstacle contact area and the vehicle in the front direction of the vehicle.

[0057] In some conventional obstacle grounding position recognition methods, after obtaining the AVM image, for example, the AVM image is mapped to a polar coordinate system so that the pixels in the AVM image have polar coordinate position features. Afterwards, feature extraction is performed on the AVM image to obtain image features that can characterize the obstacle grounding area and polar coordinate features of related image features, based on which the angle between the obstacle grounding area and the vehicle can be predicted.

[0058] It should be understood that the above is an explanation based on mapping the AVM image to polar coordinates. The polar coordinate system can represent the position from the angle dimension. In this way, the above can predict the angle between the obstacle touchdown area and the vehicle. In other embodiments, if the AVM image is mapped to a position representation system of other dimensions, the position characteristics of the obstacle touchdown area in the corresponding dimension can be obtained.

[0059] In this way, conventional methods are used to map the AVM image to a coordinate system, so that the dimension represented by the characteristics of the obstacle grounding position is relatively single, resulting in the identified obstacle grounding position characteristics having relatively large granularity and relatively low accuracy.

[0060] In view of this, the technical solution provided in the embodiment of the present application can first extract image features from the panoramic image after acquiring the panoramic image, and then construct a mapping relationship matrix between the first coordinate system and the second coordinate system. Among them, the first coordinate system is used to represent the position features from the first dimension, and the second coordinate system is used to represent the position features from the second dimension. Afterwards, the coordinate features are generated according to the image features and the coordinate mapping relationship matrix. In this way, the coordinate features can include the position features of the first dimension and the position features of the second dimension, so that the position information of the obstacle grounding area identified from the coordinate features can include the position information of the first dimension and the position information of the second dimension. It can be seen that by adopting this implementation method, the image features are converted and mapped by the coordinate system, and the image features are mapped to different position representation dimensions, so that the position representation of multiple dimensions of the obstacle grounding area can be obtained, so that the obtained position representation has a finer granularity and relatively higher accuracy.

[0061] The execution subject of this technical solution can be any device that supports the obstacle grounding position recognition function, including vehicles, ships, aircraft or robots. Such devices can support the obstacle grounding position recognition of the embodiment of this application through the automatic parking system. The automatic parking system can deploy an algorithm network for obstacle grounding position recognition.

[0062] The following describes a method, device, equipment and storage medium for identifying the ground position of an obstacle proposed according to an embodiment of the present application in conjunction with the accompanying drawings.

[0063] See also Figure 1 , Figure 1 A flow chart of a method for identifying a grounding position of an obstacle provided by an embodiment of the present application is shown, and the method specifically comprises the following steps:

[0064] Step S101: extracting image features from a panoramic image, wherein the image features include image features of an obstacle contact area.

[0065] The panoramic image may be a bird's eye view (BEV) panoramic image of the area surrounding the vehicle, which may be acquired in real time through a pre-deployed AVM system.

[0066] In some embodiments, the automatic parking system may call a pre-deployed algorithm network to pre-process the panoramic image to obtain image data of the panoramic image, and then perform computer vision operations on the image data to extract image features from the image data. Pre-processing the panoramic image may include performing noise reduction, filtering, normalization, and other processing on the pixels of the panoramic image to obtain red, green, and blue (RGB) three-channel data of the pixels of the panoramic image. For example, the RGB three-channel data of the panoramic image may include the RGB mean of the pixels in the panoramic image and the RGB variance of the pixels in the panoramic image.

[0067] Furthermore, the algorithm network can operate on the image data, and the operation result obtained can be an image feature that characterizes the panoramic image. In some embodiments, the algorithm network can operate on the image data once to obtain the image feature of a panoramic image of a certain size, and the image feature of the panoramic image of a certain size is the image feature. In other embodiments, the algorithm network can operate on the image data at least twice, and each operation can obtain an image sub-feature of a panoramic image of a certain size, and the sizes of the panoramic images corresponding to the at least two operations are different. The at least two image sub-features constitute the image feature.

[0068] It should be understood that the size of the panoramic image described in the embodiments of the present application may refer to the width (pixel (pixel, P))*height (P) of the panoramic image, or the width (centimeter (cm))*height (cm) of the panoramic image.

[0069] Exemplarily, in an embodiment of the present application, if the panoramic image includes an obstacle, the image features corresponding to the panoramic image may include image features of the obstacle grounding area. If the image features include image sub-features of two sizes, the image features may characterize features of obstacle grounding areas of different sizes. That is, by adopting this implementation, panoramic images of different sizes or features of obstacle grounding areas of different sizes in the panoramic image may be extracted, thereby improving the accuracy of identifying the obstacle grounding area.

[0070] Step S102, constructing a coordinate mapping relationship matrix.

[0071] The coordinate mapping relationship refers to the mapping relationship between the first coordinate system and the second coordinate system, and the first coordinate system and the second coordinate system can be deployed according to the size of the panoramic image. In an embodiment of the present application, the first coordinate system can be used to characterize position features from a first dimension, and the second coordinate system can be used to characterize position features from a second dimension.

[0072] In some embodiments, the coordinate mapping relationship matrix includes coordinate mapping relationships of multiple position points, and the multiple position points are associated with the first coordinate system. Exemplarily, the multiple position points can be sampling points pre-set in the process of constructing the first coordinate system, and these sampling points can be used to sample and arrange pixels corresponding to image features in a certain order.

[0073] Exemplarily, a first coordinate system can be constructed according to the size of the image feature, the first coordinate system includes M directions, and the multiple position points are distributed in the M directions, wherein M is an integer greater than 3. For any position point among the multiple position points, the coordinates of the position point in the first coordinate system can be calculated to obtain the first coordinate of the position point. Furthermore, the coordinates of the position point in the second coordinate system can be calculated according to the first coordinate and the mapping relationship between the first coordinate system and the second coordinate system to obtain the second coordinate of the position point. In some embodiments, the second coordinate of the position point is obtained by converting the first coordinate and the mapping relationship, and the second coordinate can characterize the mapping relationship between the first coordinate system and the second coordinate system of the position point. Based on this, after obtaining the first coordinates and the second coordinates of all position points, the second coordinates of the multiple position points can be used as eigenvalues ​​to generate the coordinate mapping relationship matrix.

[0074] In some embodiments, the first coordinate system may be, for example, a polar coordinate system, M may be, for example, 360, and 20 position points may be deployed in each of the 360 ​​directions, for example. The 20 position points in each direction may be sampling points in that direction. The first coordinate system may be, for example, a Cartesian coordinate system. In this way, the first coordinate system may represent the angle in the position information, and the second coordinate system may represent the distance in the position information. Furthermore, for any position point X, the coordinate value of the position point X in the polar coordinate system may be calculated to obtain the first coordinate of the position point X. Afterwards, combining the mapping relationship between the polar coordinate system and the Cartesian coordinate system, the first coordinate is converted into the coordinate value of the position point X in the Cartesian coordinate system to obtain the second coordinate of the position point X. After obtaining the second coordinates of 20*360=7200 position points, the second coordinates of the 7200 position points may be used as a coordinate mapping relationship matrix between the polar coordinate system and the Cartesian coordinate system.

[0075] Exemplarily, a feature column in the coordinate mapping relationship matrix may correspond to one of the 360 ​​directions, and any two adjacent columns are two adjacent directions in the polar coordinate system. A feature row is a position point in the polar coordinate system from the pole to the radiation direction. That is, the matrix may be a feature matrix of 360 columns*20 rows.

[0076] It should be understood that the above introduction to the polar coordinate system is a schematic description and does not constitute a limitation on the first coordinate system involved in the embodiment of the present application. In other embodiments, the polar coordinate system can be deployed in 270 or 180 directions, and the position points in each direction can be deployed more or less. This is not limited here.

[0077] In addition, the polar coordinate system and Cartesian coordinate system are examples for explaining the present application and do not constitute a limitation. In another embodiment, the first coordinate system and the second coordinate system may be other coordinate systems that can respectively characterize position features from two dimensions. Examples are not given one by one here.

[0078] It should be pointed out that in the embodiment of the present application, after acquiring the panoramic image, the execution of extracting image features in S101 and the execution of S102 can be executed in parallel or in any time sequence, which is not limited here.

[0079] Step S103: generating coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix.

[0080] The coordinate features include pixel features of the multiple location points and coordinate mapping relationship features of the multiple location points.

[0081] In some embodiments, the pixel features corresponding to the multiple position points in the image features can be obtained, that is, the image features are sampled according to the pre-deployed multiple position points to obtain the pixel features of the multiple position points. Combined with the aforementioned relationship between the multiple position points and the polar coordinates, it can be known that the multiple position points are in a one-to-one correspondence with the positions indicated by the eigenvalues ​​in the coordinate mapping relationship matrix, then the image features of each position point correspond to the corresponding eigenvalues ​​in the coordinate mapping relationship matrix. To this end, the sorting of the feature points in the pixel features can be deployed according to the correspondence between the multiple position points and the eigenvalues ​​in the coordinate mapping relationship matrix, so that each feature point is associated with the corresponding eigenvalue in the coordinate mapping relationship matrix. The sorted pixel features can include the feature points, as well as the position features of the feature points in the first coordinate system and the second coordinate system, and the sorted pixel features can be considered as the coordinate features.

[0082] Still taking the case where the first coordinate system is a polar coordinate system, 20 position points (i.e., sampling points) are deployed in each of the 360 ​​directions, and the second coordinate system is a Cartesian coordinate system as an example. In this example, the image features can be sampled according to the 20 position points in each direction, and the pixel feature points corresponding to the 20 position points in each direction are sampled to obtain pixel features composed of 7200 pixel feature points. Afterwards, the 7200 pixel feature points can be arranged correspondingly according to the distribution of each direction and the 20 position points in each direction to form the sampled pixel features.

[0083] Exemplarily, corresponding to the aforementioned coordinate mapping relationship matrix, the pixel feature can be a feature matrix of 360 columns * 20 rows, wherein each pixel feature point corresponds to a coordinate mapping relationship of a position point. In view of this, the position feature of each feature point of the pixel feature can be characterized by the corresponding coordinate mapping relationship.

[0084] It can be seen that by adopting this implementation method, two coordinate systems can be constructed for image features, and the two coordinate systems represent position features from different dimensions respectively, and the two coordinate systems can be mapped to each other. After establishing the correspondence between the sampling points of the image features and the two coordinate systems, the sampled pixel features can have the position features corresponding to the two coordinate systems, so that the position representation of the obstacle grounding area in multiple dimensions can be obtained.

[0085] Step S104: identifying the location information of the obstacle touchdown area from the coordinate features.

[0086] The location information may include but is not limited to the location information of the obstacle touchdown area in the first dimension and the location information of the second dimension.

[0087] For example, if the first coordinate system is a polar coordinate system, the position information of the first dimension may be the angle between the obstacle contact area and the center position of the panoramic image (also referred to as angle information). If the second coordinate system is a Cartesian coordinate system, the position information of the second dimension is the distance between the obstacle contact area and the center of the panoramic image.

[0088] In this way, the location information extracted from the feature data containing location features of two dimensions can contain location information of two dimensions, so that the obtained location representation has finer granularity and relatively higher accuracy.

[0089] In some embodiments, in order to obtain more obvious characterization features of the obstacle touchdown area and the corresponding position information, the coordinate features may be further encoded, and then the first dimension position information and the second dimension position information of the obstacle touchdown area are identified according to the position features of the obstacle touchdown area. It should be understood that the encoded features include the position features of the obstacle touchdown area, and the position features include the first dimension position features and the second dimension position features.

[0090] It should be pointed out that, affected by the field of view of the image acquisition device, the AVM image may present the scenery within a certain range around the vehicle. In some implementation scenarios, the grounding area of ​​the obstacle may intersect with the boundary of the AVM image, or the grounding area of ​​the obstacle may overlap with the edge of the AVM image. Based on this, in other embodiments, for any of the M directions, the type of object to be identified corresponding to the direction can be determined based on the encoded features. The type of object to be identified here is the grounding area of ​​the obstacle or the edge of the panoramic image. If the type of object to be identified corresponding to the direction is the grounding area of ​​the obstacle, the location information of the grounding area of ​​the obstacle can be further identified.

[0091] It can be seen that, by adopting the implementation method, the embodiment of the present application also has the function of identifying the obstacle touchdown area, and then, when it is determined that it is the obstacle touchdown area, the position information of the obstacle touchdown area can be predicted, thereby further improving the accuracy of the position information of the obstacle touchdown area.

[0092] Exemplarily, the above encoding process may include, but is not limited to, converting the pixel features and coordinate mapping relationship matrix obtained in S103 into feature data of the same data dimension, further extracting features from the feature data to obtain features with higher dimensions or more accurate representations, and then predicting the obstacle grounding area and the location features of the obstacle grounding area based on the extracted features. Optionally, multiple feature extraction operations may be performed on the feature data, and the parameters related to each feature extraction operation may be different, so as to extract features from multiple angles.

[0093] Taking the convolution algorithm as an example, multiple convolution operations may be performed, and at least one of the convolution kernel and the number of channels of each convolution operation may be different.

[0094] In one implementation scenario, after identifying the positions of all obstacle touchdown areas in a panoramic image, the outline of the obstacle touchdown areas can be determined in the panoramic image. The outline can be considered as the boundary of the vehicle's drivable area. Then, the area enclosed by the outline of the obstacle touchdown areas can be determined as the vehicle's drivable area.

[0095] In an embodiment of the present application, image features can be extracted from a panoramic image, and a coordinate mapping relationship matrix can be constructed. The coordinate mapping relationship matrix represents the mapping relationship between the first coordinate system and the second coordinate system, the first coordinate system is used to represent the position feature from the first dimension, and the second coordinate system is used to represent the position feature from the second dimension. The coordinate mapping relationship matrix can be a coordinate mapping relationship including multiple position points (i.e., sampling points). Afterwards, the coordinate features are generated by the correlation between the image features and the coordinate mapping relationship matrix, and the coordinate features include the pixel features of the multiple position points, and the coordinate mapping relationship features of the multiple position points. That is, the coordinate features include the pixel features of each position point, and the first dimension position features and the second dimension position features of the corresponding pixels. In this way, the position information of the obstacle grounding area identified from the coordinate features can include the position information including the first dimension position information and the second dimension position information of the obstacle grounding area. It can be seen that in the embodiment of the present application, the image features are mapped to different position representation dimensions, so that the position representation of multiple dimensions of the obstacle grounding area can be obtained, so that the obtained position representation has a finer granularity and a relatively higher accuracy.

[0096] Combined with the above description, Figure 1 Some or all of the steps of the illustrated method embodiment may be executed by a pre-deployed algorithm network.

[0097] An exemplary algorithm network is as follows Figure 2A As shown, according to the data transmission direction, the algorithm network may include a backbone network, a feature processing network and a prediction network, wherein the feature processing network may be the neck network of the algorithm network, and the prediction network may be the head network of the algorithm network. In some embodiments, the backbone network may be used to perform the above-mentioned processing of S101, the feature processing network may be used to perform the above-mentioned processing of S102 and S103, and the prediction network may be used to perform the above-mentioned processing of S104.

[0098] It should be understood that Figure 2A The illustrated structure does not constitute a specific limitation on the algorithm network of the present application. In other embodiments of the present application, the algorithm network may include more or fewer component networks than shown, or combine certain networks, or split certain networks, or arrange different networks. It will be apparent to those skilled in the art that many modifications and variations are possible based on the above teachings.

[0099] Among them, the backbone network can be implemented as any algorithm model that supports extracting image features from panoramic images, such as but not limited to convolutional neural network (CNN), EfficientNetLite network, etc. The feature processing network can be implemented as any algorithm network that supports coordinate system mapping, image encoding and decoding, or a combination of algorithm networks. For example, the feature processing network can be implemented as a multi-layer transformer network; for example, a multi-layer convolution network, different parts of the multi-layer convolution network can perform different processing, for example, a part of the multi-layer convolution network is used to perform the above-mentioned S102 processing, and a part of the multi-layer convolution network is used to perform the above-mentioned S103 processing. The prediction network can be a network that matches the feature processing network, and is used to process the feature data obtained by the feature processing network and output the prediction result. The prediction result output by the prediction network is the recognition result of the obstacle grounding area location information. The prediction network can include, for example, a feed forward net (FFN), a fully connected network, and a convolution network.

[0100] The following is an introduction to the obstacle grounding location identification method of the embodiment of the present application using an algorithm network including multiple convolutional layers and examples.

[0101] See also Figure 2B , Figure 2B An exemplary data flow diagram of an obstacle ground position identification method is shown. Figure 2B The algorithm network used to perform the obstacle ground position identification method includes, for example: EfficientNetLite network, Feature Pyramid Networks (FPN), coordinate system mapping network, encoding network, and prediction network. Among them, the EfficientNetLite network, Feature Pyramid Networks (FPN), coordinate system mapping network, encoding network, and prediction network can all include multiple convolutional layers. In some other embodiments, the prediction network can be a fully connected layer.

[0102] Exemplarily, the EfficientNetLite network may include multiple convolutional layers, each of which may be loaded with pre-trained weights, so that the corresponding convolutional layer has the ability to extract image features. In this example, the weights loaded by the multiple convolutional layers are different, so that different convolutional layers can be used to extract features of images of different sizes (e.g., resolutions). The pre-trained weights can be obtained by pre-training on the imagenet dataset.

[0103] The EfficientNetLite network can be Figure 2B The backbone network of the algorithm network shown, FPN, coordinate system mapping network and encoding network can be composed Figure 2B The neck network of the algorithm network shown, the prediction network can be Figure 2B The head network of the algorithm network shown.

[0104] In some embodiments, if the EfficientNetLite network and FPN are regarded as the first convolutional network, the coordinate system mapping network is regarded as the second convolutional network, and the encoding network and the prediction network are regarded as the third convolutional network, it can be considered that the automatic parking system calls the first convolutional network to extract image features from the panoramic image (such as S101 above); calls the second convolutional network to construct a coordinate mapping relationship matrix (such as S102 and S103 above); and identifies the location information of the obstacle grounding area from the coordinate features according to the third convolutional network (such as S104 above).

[0105] The following examples are combined with Figure 2B The obstacle grounding position identification method of the embodiment of the present application is introduced from the perspective of the actions performed by each network.

[0106] For example, the AVM image of the vehicle is an image with a resolution of 896*896P. After the algorithm network receives the AVM image, the preprocessing module ( Figure 2B (not shown) 896*896P pixels can be processed according to the RGB three channels, for example, image data with RGB mean values ​​of [123.675, 116.28, 103.53] and RGB variances of [58.395, 57.12, 57.375] are obtained. Then, the RGB mean value [123.675, 116.28, 103.53] and the RGB variance [58.395, 57.12, 57.375] are input into the EfficientNetLite network.

[0107] The multiple convolutional layers in the EfficientNetLite network can process the image data separately, and each convolutional layer can extract an image sub-feature of the AVM image. The image sub-features extracted by the multiple convolutional layers correspond to AVM images of different resolutions. In this way, features of various sizes of the AVM image can be obtained. Furthermore, the EfficientNetLite network inputs the features of various sizes of the AVM image into the FPN.

[0108] FPN can fuse features of various sizes of AVM images, and the fused features are, for example, image features of AVM images. Afterwards, FPN transmits the image features to the coordinate system mapping network. In this example, the image features include features of AVM images at various resolutions, thereby supporting the recognition of grounded areas of obstacles of various sizes. The size of the image features is, for example, 64 channels, and each channel has a resolution of 224*224: [64,224,224].

[0109] It should be noted that in some other embodiments, if the EfficientNetLite network includes a convolution layer, then the image features output by the EfficientNetLite network are features of an AVM image of one size, and feature fusion is not required. The algorithm network corresponding to this embodiment may not include FPN, and the EfficientNetLite network may input the extracted image features into the coordinate system mapping network.

[0110] The coordinate system mapping network can, for example, deploy polar coordinates and Cartesian coordinates according to image features, and establish a coordinate mapping relationship matrix between the polar coordinates and Cartesian coordinates. Then, coordinate features are obtained according to the coordinate mapping relationship matrix and image features. Figure 3 As shown, the coordinate system mapping network can obtain coordinate features through the implementation process of S301-S305.

[0111] S301, constructing features of a polar coordinate system according to the size of the image features.

[0112] The characteristic of the polar coordinates is, for example, [64, 20, 360], that is, there are also 64 channels, deployed in 360 radial directions, and 20 position points (also called sampling points) are sampled in each radial direction.

[0113] S302, setting the pole and the length of the radius r of the polar coordinate system.

[0114] The pole may be, for example, the center point of the polar coordinate system. The length of r may be, for example, set to w / 2, where w refers to half of the polar coordinate angle.

[0115] S303, calculating the Cartesian coordinates of the corresponding position point according to the polar coordinates of each position point.

[0116] In the Cartesian coordinates of any point, X=r.cos(θ)+W / 2, Y=r.sin(θ)+H / 2, where θ is the polar angle of the point, that is, the angle w between the direction of the point and the polar axis. W refers to the width of the image feature, H is the height of the image feature, r.cos(θ) refers to the coordinate value of the polar coordinate of the point mapped to the x-axis, and r.sin(θ) refers to the coordinate value of the polar coordinate of the point mapped to the y-axis.

[0117] S304, obtaining a coordinate mapping relationship matrix M according to the Cartesian coordinates of each position point.

[0118] For example, the coordinate system mapping network can normalize the Cartesian coordinates of each position point. The normalized coordinates of any position point can be expressed as x=X / W / 2, y=Y / H / 2. The normalized coordinates of each position point form a matrix M.

[0119] Among them, the coordinate values ​​of each position point in the matrix M and the coordinate system mapping network can be arranged according to the direction of the polar coordinate system and the sampling order in each direction.

[0120] S305, processing the coordinate mapping relationship matrix M and the image features to obtain coordinate features.

[0121] For example, the coordinate system mapping network can sample image features according to 360 directions and 20 position points in each direction to obtain 7200 pixel feature points, and arrange the 7200 pixel feature points accordingly according to the distribution of the position points in the coordinate mapping relationship matrix M to obtain coordinate features.

[0122] After obtaining the coordinate features, the coordinate system mapping network can transfer the coordinate features to the encoding network.

[0123] The encoding network can include multiple convolutional layers, so that multiple rounds of convolution operations can be performed on the coordinate features, and richer features of the AVM image can be extracted. In this step, the features extracted by the encoding network are called encoding features. Encoding features include, for example, pixel features and position features of obstacle areas. One encoding feature is implemented as [32, 20, 360], where 32 refers to the number of channels, 360 refers to the direction, and 20 refers to the number of sampling points in each direction, which is also a height feature.

[0124] Furthermore, the encoding network transmits the encoding features to the prediction network so that the prediction network predicts the location information of the obstacle grounding area.

[0125] It should be pointed out that the prediction network can be implemented as an algorithm network with different functions according to the different types of prediction results.

[0126] For example, in one implementation scenario, the prediction network is used to predict the location information of the obstacle grounding area. After receiving the coded features [32, 20, 360], the channel features and height features can be merged to retain the angle features corresponding to 360 directions. Then, based on the angle features corresponding to the 360 ​​directions, the distance between each obstacle grounding area and the center point and the angle in polar coordinates can be predicted.

[0127] For example, in another implementation scenario, after merging the channel features and height features of the encoded features, the prediction network inputs the angle features corresponding to 360 directions into the classification model to predict whether the boundary pixels in each direction are the ground area of ​​the obstacle or the edge of the panoramic image.

[0128] It should be understood that Figure 2B The illustrated algorithm network and the processing of each network model in the algorithm network are schematic descriptions and do not limit the algorithm network involved in the embodiments of the present application. In other embodiments, each network model in the algorithm network can also be implemented based on other algorithm mechanisms other than the convolution algorithm. Accordingly, the operation process of each network model and the output feature data can be different from the presentation of the above embodiment. This is not limited here.

[0129] The present application also provides an obstacle grounding position identification device, which can be deployed in an automatic parking system. The obstacle grounding position identification device modularizes the functions of the network model in the above algorithm network through software, hardware, or a combination of software and hardware, and can be used to execute the obstacle grounding position identification method provided in any of the above embodiments. Figure 4 As shown, the device comprises:

[0130] A feature extraction module 401 is used to extract image features from a panoramic image, wherein the image features include image features of an obstacle grounding area; a construction module 402 is used to construct a coordinate mapping relationship matrix, wherein the coordinate mapping relationship refers to a mapping relationship between a first coordinate system and a second coordinate system, wherein the coordinate mapping relationship matrix includes coordinate mapping relationships of multiple position points, wherein the multiple position points are associated with the first coordinate system, wherein the first coordinate system is used to characterize position features from a first dimension, and wherein the second coordinate system is used to characterize position features from a second dimension; a generation module 403 is used to generate coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix, wherein the coordinate features include pixel features of the multiple position points and coordinate mapping relationship features of the multiple position points; an identification module 404 is used to identify position information of the obstacle grounding area from the coordinate features, wherein the position information includes position information of the first dimension and position information of the second dimension of the obstacle grounding area.

[0131] Optionally, the feature extraction module 401 is further used to extract features from the panoramic image to obtain at least one image sub-feature, and the at least one image sub-feature constitutes the image feature; the at least one image sub-feature represents the feature of the panoramic image at at least one size.

[0132] Optionally, the construction module 402 is further configured to construct the first coordinate system according to the size of the image feature, the first coordinate system including M directions, the multiple position points being distributed in the M directions, wherein M is an integer greater than 3;

[0133] Corresponding to any position point among the multiple position points, calculating the coordinates of the position point in the first coordinate system to obtain a first coordinate of the position point;

[0134] Calculating the coordinates of the position point in the second coordinate system according to the first coordinates and a mapping relationship between the first coordinate system and the second coordinate system to obtain the second coordinates of the position point;

[0135] The coordinate mapping relationship matrix is ​​generated by taking the second coordinates of the plurality of position points as eigenvalues.

[0136] Optionally, the generating module 403 is further configured to obtain pixel features corresponding to the plurality of position points in the image features, so as to obtain pixel features of the plurality of position points;

[0137] According to the correspondence between the multiple position points and the eigenvalues ​​in the coordinate mapping relationship matrix, the sorting of each feature point in the pixel feature is deployed so that each feature point is associated with the corresponding eigenvalue in the coordinate mapping relationship matrix, and the sorted pixel feature is the coordinate feature.

[0138] Optionally, the identification module 404 is further configured to encode the coordinate features, where the encoded features include position features of the obstacle touchdown area, where the position features include position features of the first dimension and position features of the second dimension;

[0139] The position information of the obstacle contact area in the first dimension and the position information of the obstacle contact area in the second dimension are identified according to the position feature of the obstacle contact area.

[0140] Optionally, the encoded feature corresponds to the multiple position points, the multiple position points are respectively deployed in M ​​directions, the M directions are directions included in the first coordinate system, and the identification module 404 is further used to determine the type of object to be identified corresponding to each direction of the M directions according to the encoded feature, and the type of object to be identified is the obstacle grounding area or the edge of the panoramic image;

[0141] If the type of the object to be identified corresponding to any direction is the obstacle touchdown area, the position information of the first dimension and the position information of the second dimension of the obstacle touchdown area are identified.

[0142] Optionally, the device also includes a boundary determination module, which is used to determine the outline of the obstacle ground contact area in the panoramic image, and determine the area enclosed by the outline of the obstacle ground contact area as a drivable area for a vehicle, and the vehicle is the vehicle that captures the panoramic image.

[0143] Optionally, the device also includes a calling module, which is used to call the first convolutional network to extract image features from the panoramic image; call the second convolutional network to construct a coordinate mapping relationship matrix, and generate coordinate features of the panoramic image based on the image features and the coordinate mapping relationship matrix; and identify the location information of the obstacle ground area from the coordinate features according to the third convolutional network.

[0144] The obstacle grounding position identification device provided in the embodiment of the present application and the obstacle grounding position identification method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0145] The present application also provides an electronic device, which is applied to an automatic parking system to execute the above-mentioned obstacle grounding position identification method. Figure 5 It shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 5 As shown, the electronic device 5 includes: a processor 500, a memory 501, a bus 502 and a communication interface 503, wherein the processor 500, the communication interface 503 and the memory 501 are connected via the bus 502; the memory 501 stores a computer program that can be run on the processor 500, and when the processor 500 runs the computer program, the obstacle grounding position identification method provided in any of the aforementioned embodiments of the present application is executed.

[0146] The memory 501 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 503 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0147] The bus 502 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 501 is used to store a program, and the processor 500 executes the program after receiving an execution instruction. The obstacle grounding position identification method disclosed in any implementation of the embodiment of the present application may be applied to the processor 500, or implemented by the processor 500.

[0148] The processor 500 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 500. The above processor 500 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 501, and the processor 500 reads the information in the memory 501 and completes the steps of the above method in combination with its hardware.

[0149] The electronic device provided in the embodiment of the present application and the method for identifying the ground position of an obstacle provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0150] The present application also provides a computer-readable storage medium corresponding to the obstacle grounding position identification method provided in the above embodiment. Figure 6 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the obstacle grounding position identification method provided in any of the aforementioned embodiments will be executed.

[0151] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0152] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the obstacle grounding position identification method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0153] It should be noted that:

[0154] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0155] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0156] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0157] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for identifying a grounding position of an obstacle, characterized in that: include: Extracting image features from the panoramic image, wherein the image features include image features of the obstacle contact area; Constructing a coordinate mapping relationship matrix, wherein the coordinate mapping relationship refers to a mapping relationship between a first coordinate system and a second coordinate system, wherein the coordinate mapping relationship matrix includes coordinate mapping relationships of a plurality of position points, wherein the plurality of position points are associated with the first coordinate system, wherein the first coordinate system is used to characterize position features from a first dimension, and wherein the second coordinate system is used to characterize position features from a second dimension; Generate coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix, the coordinate features including pixel features of the multiple location points and coordinate mapping relationship features of the multiple location points; The position information of the obstacle contact area is identified from the coordinate features, where the position information includes the position information of the obstacle contact area in the first dimension and the position information of the second dimension.

2. The method according to claim 1, characterized in that The step of extracting image features of the object to be identified from the panoramic image comprises: Feature extraction is performed on the panoramic image to obtain at least one image sub-feature, wherein the at least one image sub-feature constitutes the image feature; and the at least one image sub-feature represents a feature of the panoramic image at at least one size.

3. The method according to claim 1, characterized in that The step of constructing a coordinate mapping relationship matrix includes: Constructing the first coordinate system according to the size of the image feature, the first coordinate system includes M directions, the multiple position points are distributed in the M directions, wherein M is an integer greater than 3; Corresponding to any one of the multiple position points, calculating the coordinates of the position point in the first coordinate system to obtain a first coordinate of the position point; Calculating the coordinates of the position point in the second coordinate system according to the first coordinates and a mapping relationship between the first coordinate system and the second coordinate system to obtain the second coordinates of the position point; The coordinate mapping relationship matrix is ​​generated by taking the second coordinates of the plurality of position points as eigenvalues.

4. The method according to claim 1 or 3, characterized in that: The step of generating the coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix includes: Obtain pixel features corresponding to the multiple position points in the image features to obtain pixel features of the multiple position points; According to the correspondence between the multiple position points and the eigenvalues ​​in the coordinate mapping relationship matrix, the sorting of each feature point in the pixel feature is deployed so that each feature point is associated with the corresponding eigenvalue in the coordinate mapping relationship matrix, and the sorted pixel feature is the coordinate feature.

5. The method according to claim 1, characterized in that: The step of identifying the location information of the obstacle contact area from the coordinate features includes: Encoding the coordinate features, wherein the encoded features include position features of the obstacle contact area, and the position features include position features of the first dimension and position features of the second dimension; The position information of the obstacle contact area in the first dimension and the position information of the obstacle contact area in the second dimension are identified according to the position feature of the obstacle contact area.

6. The method according to claim 5, characterized in that The encoded features correspond to the multiple position points, the multiple position points are respectively deployed in M ​​directions, the M directions are directions included in the first coordinate system, and after encoding the coordinate features, further comprising: Determine the type of the object to be identified corresponding to each direction of the M directions according to the encoded features, the type of the object to be identified being the ground contact area of ​​the obstacle or the edge of the panoramic image; If the type of the object to be identified corresponding to any direction is the obstacle touchdown area, the position information of the first dimension and the position information of the second dimension of the obstacle touchdown area are identified.

7. The method according to claim 3 or 5, characterized in that: The first coordinate system is a polar coordinate system, and M is 360; The second coordinate system is a Cartesian coordinate system.

8. The method according to claim 5 or 6, characterized in that: The position information of the first dimension is the angle between the ground contact area of ​​the obstacle and the center position of the panoramic image; The position information of the second dimension is the distance between the obstacle contact area and the center of the panoramic image.

9. The method according to claim 1, characterized in that: After identifying the location information of the obstacle contact area from the coordinate features, the method further includes: Determining, in the panoramic image, an outline of the obstacle contact area component; An area enclosed by the outline formed by the obstacle contact area is determined as a drivable area of ​​a vehicle, and the vehicle is a vehicle that collects the panoramic image.

10. The method according to claim 1, characterized in that include: Calling the first convolutional network to extract image features from the panoramic image; Calling a second convolutional network to construct a coordinate mapping relationship matrix, and generating coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix; The location information of the obstacle ground contact area is identified from the coordinate features according to the third convolutional network.

11. An obstacle grounding position identification device, characterized in that: include: A feature extraction module, used to extract image features from the panoramic image, wherein the image features include image features of the obstacle ground contact area; A construction module, used to construct a coordinate mapping relationship matrix, wherein the coordinate mapping relationship refers to a mapping relationship between a first coordinate system and a second coordinate system, wherein the coordinate mapping relationship matrix includes coordinate mapping relationships between a plurality of position points, wherein the plurality of position points are associated with the first coordinate system, wherein the first coordinate system is used to characterize position features from a first dimension, and wherein the second coordinate system is used to characterize position features from a second dimension; A generating module, configured to generate coordinate features of the panoramic image according to the image features and the coordinate mapping relationship matrix, wherein the coordinate features include pixel features of the plurality of position points and coordinate mapping relationship features of the plurality of position points; An identification module is used to identify the position information of the obstacle touchdown area from the coordinate features, wherein the position information includes the position information of the obstacle touchdown area in the first dimension and the position information of the second dimension.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor runs the computer program to implement the method according to any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 10.

Images