Obstacle attribute determination method and device, terminal and storage medium

Abstract

The application discloses a method and device for determining obstacle attributes, a terminal and a storage medium, and belongs to the technical field of image processing. The method is applied to the terminal and comprises the following steps: acquiring an attribute feature image according to a first fisheye image; performing attribute classification on the attribute feature image to acquire an attribute image corresponding to each obstacle attribute; acquiring an attribute vector according to the attribute feature image; and determining the obstacle attribute of an obstacle at a first position in the first fisheye image according to the attribute vector and the attribute image corresponding to each obstacle attribute. After the attribute feature image is acquired from the first fisheye image, the attribute image of each attribute and the attribute vector representing the global classification are acquired based on the attribute feature image. The obstacle attribute of the obstacle at the first position is determined by combining the attribute vector and the attribute image, and the obstacle attribute is obtained based on the two classification results, so that the acquisition of the obstacle attribute is more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a method and device for determining obstacle attribute, a terminal and a storage medium. BACKGROUND

[0002] With the rapid development of science and technology, people have more and more functional requirements for terminals, and the functions that can be realized in the terminals are also more and more, for example, users can use terminals to detect space, play audio, locate position, etc.

[0003] Among them, in the aspect of space detection, a camera is usually used to collect an environment image of the surrounding environment, and the reachable space in the surrounding environment is detected based on the environment image. At present, when the terminal outputs the position of the predicted point in the reachable space, it often needs to output the attribute of the predicted point (for example, the obstacle at the predicted point belongs to static obstacle, dynamic obstacle or suspended obstacle, etc. Several categories of attributes), wherein, for the detection that the predicted point may belong to different attributes, the terminal usually obtains a heat map based on the collected environment image, and the attribute of the obstacle is predicted by the heat map. The way of obtaining the attribute of the obstacle in this process is single, and the accuracy of the obtained attribute of the obstacle is low. SUMMARY

[0004] The embodiments of the present application provide a method and device for determining obstacle attribute, a terminal and a storage medium, which can improve the accuracy of obtaining the attribute of the obstacle corresponding to the obstacle in the image.

[0005] In one aspect, the embodiments of the present application provide a method for determining obstacle attribute, applied to a terminal, the method comprising:

[0006] According to the first fisheye image, an attribute feature image is obtained, the attribute feature image comprising attribute features of each obstacle in the first fisheye image;

[0007] The attribute feature image is classified by attribute to obtain an attribute map corresponding to each obstacle attribute, the attribute map comprising a confidence degree of the obstacle attribute corresponding to each pixel point in the first fisheye image;

[0008] According to the attribute feature image, an attribute vector is obtained, the attribute vector being used to represent the obstacle attribute and the confidence degree corresponding to each pixel point in the first fisheye image under global classification;

[0009] According to the attribute vector and the attribute map corresponding to each obstacle attribute, an obstacle attribute of an obstacle at a first position in the first fisheye image is determined, the first position being a pixel point position contained by any one obstacle in the first fisheye image.

[0010] Optionally, the attribute vector is obtained according to the attribute feature image, and the attribute vector comprises:

[0011] The attribute feature image is one-dimensionally transformed to obtain a one-dimensional feature map.

[0012] The attribute vector is generated according to the one-dimensional feature map and a full connection layer.

[0013] Optionally, the attribute of the obstacle at the first position in the first fisheye image is determined according to the attribute vector and the attribute map corresponding to each obstacle attribute, and the attribute of the obstacle at the first position in the first fisheye image comprises:

[0014] The first obstacle attribute information of the obstacle at the first position in the first fisheye image is determined according to the attribute map corresponding to each attribute, and the first obstacle attribute information comprises a first obstacle attribute of the obstacle at the first position and a confidence degree corresponding to the first obstacle attribute.

[0015] The second obstacle attribute information of the obstacle at the first position in the first fisheye image is determined according to the attribute vector, and the second obstacle attribute information comprises a second obstacle attribute of the obstacle at the first position and a confidence degree corresponding to the second obstacle attribute.

[0016] The attribute of the obstacle at the first position is determined according to the first obstacle attribute information and the second obstacle attribute information.

[0017] Optionally, the first obstacle attribute information of the obstacle at the first position in the first fisheye image is determined according to the attribute map corresponding to each attribute, and the first obstacle attribute information comprises:

[0018] Each obstacle attribute and each confidence degree of the obstacle at the first position in the attribute map corresponding to each attribute are determined.

[0019] The highest confidence degree in the confidence degrees is determined according to the confidence degrees.

[0020] The highest confidence degree and the obstacle attribute corresponding to the highest confidence degree are obtained as the first obstacle attribute information.

[0021] Optionally, the first obstacle attribute information of the obstacle at the first position in the first fisheye image is determined according to the attribute map corresponding to each attribute, and the first obstacle attribute information comprises:

[0022] Each obstacle attribute and each confidence degree of the obstacle at the first position in the attribute map corresponding to each attribute are determined.

[0023] According to each obstacle attribute of the obstacle at the first position in each attribute corresponding attribute graph and each confidence, the attribute graph corresponding to each attribute is fused to obtain a comprehensive attribute graph;

[0024] According to the comprehensive attribute graph, first obstacle attribute information of the obstacle at the first position in the first fisheye image is determined.

[0025] Optionally, the determining of the obstacle attribute of the obstacle at the first position according to the first obstacle attribute information and the second obstacle attribute information comprises:

[0026] When the confidence corresponding to the first obstacle attribute is greater than the confidence corresponding to the second obstacle attribute, the obstacle attribute of the obstacle at the first position is determined as the first obstacle attribute;

[0027] When the confidence corresponding to the first obstacle attribute is less than the confidence corresponding to the second obstacle attribute, the obstacle attribute of the obstacle at the first position is determined as the second obstacle attribute.

[0028] Optionally, the obtaining of the attribute feature image according to the first fisheye image comprises:

[0029] According to the first fisheye image and a target residual network, a plurality of layer feature images are obtained;

[0030] According to the plurality of layer feature images and a pyramid network, a feature pyramid is generated, and the feature pyramid comprises image features of a plurality of layers of the first fisheye image.

[0031] An attribute feature extraction is performed on a target layer in the feature pyramid to obtain the attribute image feature, and the target layer is a layer in the feature pyramid that contains the most features.

[0032] In another aspect, an embodiment of the present application provides a device for determining an obstacle attribute, applied to a terminal, and the device comprises:

[0033] A first obtaining module is configured to obtain an attribute feature image according to a first fisheye image, and the attribute feature image comprises attribute features of each obstacle in the first fisheye image;

[0034] A second obtaining module is configured to perform attribute classification on the attribute feature image to obtain an attribute graph corresponding to each obstacle attribute, and the attribute graph comprises a confidence of an obstacle attribute corresponding to each pixel point in the first fisheye image;

[0035] The third acquisition module is configured to acquire an attribute vector according to the attribute feature image, the attribute vector being used to represent the corresponding obstacle attribute and confidence of each pixel point in the first fisheye image under global classification;

[0036] The attribute determination module is configured to determine the obstacle attribute of the obstacle at the first position in the first fisheye image according to the attribute vector and the attribute map corresponding to each obstacle attribute, the first position being a pixel position included by any obstacle in the first fisheye image.

[0037] In another aspect, the embodiment of the present application provides a terminal, which comprises a memory and a processor, the memory storing a computer program, and the computer program being executed by the processor to enable the processor to implement the method for determining the obstacle attribute according to any one of the above aspects and optional implementation manners.

[0038] In another aspect, the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program being executed by a processor to implement the method for determining the obstacle attribute according to the other aspect and optional manners.

[0039] The technical scheme provided by the embodiment of the present application can at least have the following beneficial effects:

[0040] The terminal according to the present application acquires an attribute feature image according to a first fisheye image, the attribute feature image including attribute features of each obstacle in the first fisheye image; performs attribute classification on the attribute feature image to acquire an attribute map corresponding to each obstacle attribute, the attribute map including the confidence of the obstacle attribute corresponding to each pixel point in the first fisheye image; acquires an attribute vector according to the attribute feature image, the attribute vector being used to represent the corresponding attribute and confidence of each pixel point in the first fisheye image under global classification; and determines the obstacle attribute of the obstacle at a first position in the first fisheye image according to the attribute vector and the attribute map corresponding each obstacle attribute. After acquiring the attribute feature image of the first fisheye image, the present application acquires the attribute map of each attribute and the attribute vector representing the global classification based on the attribute feature image, and determines the obstacle attribute of the obstacle at the first position by combining the attribute vector and the attribute map, so that the obstacle attribute is obtained more accurately based on the two classification results. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0042] Figure 1 is a schematic diagram of an audio playing structure for playing audio data in a terminal according to an example embodiment of the present application;

[0043] Figure 2 is a method flow chart of a method for determining an obstacle attribute according to an example embodiment of the present application;

[0044] Figure 3 is a method flow chart of a method for determining an obstacle attribute according to an example embodiment of the present application;

[0045] Figure 4 is a structural block diagram of a device for determining an obstacle attribute according to an example embodiment of the present application;

[0046] Figure 5 is a structural diagram of a terminal according to an example embodiment of the present application. DETAILED DESCRIPTION

[0047] The example embodiments will be described in detail herein with reference to the drawings. When the description below refers to accompanying drawings, unless otherwise noted, the same numbers in different drawings refer to the same or similar elements. The following example embodiments described in the example embodiments are not representative of all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application, as detailed in the appended claims.

[0048] As used herein, the term "plurality" means two or more. The term "and / or", describing an associated relationship between associated objects, means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0049] It should be noted that the terms "first", "second", "third", and "fourth" and the like in the specification and claims of the present application are used to distinguish different objects, and are not intended to describe a specific order. The terms "include" and "have" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0050] The scheme provided in the application can be used in the scene of detecting reachable space by using a terminal in daily life. In order to facilitate understanding, some special terms and application architectures related to the embodiments of the application are introduced briefly below.

[0051] Neural Networks (NN) is a complex network system formed by a large number of simple processing units (called neurons) widely interconnected, which reflects many basic characteristics of brain function, and is a highly complex nonlinear dynamic learning system. Neural networks have large-scale parallelism, distributed storage and processing, self-organization, self-adaptation and self-learning ability, and are particularly suitable for processing information processing problems that need to consider many factors and conditions, and are imprecise and fuzzy. The development of neural networks is related to neuroscience, mathematical science, cognitive science, computer science, artificial intelligence, information science, control theory, robotics, microelectronics, psychology, optical computing, molecular biology, etc. It is an emerging interdisciplinary subject.

[0052] Global features refer to the overall properties of an image. Common global features include color features, texture features, and shape features, such as intensity histograms. Since they are low-level visual features at the pixel level, global features have good invariance, simple calculation, and intuitive representation, but their high feature dimension and large computational load are their fatal weaknesses. In addition, global feature description is not suitable for image aliasing and occlusion. Local features are features extracted from local regions of an image, including edges, corner points, lines, curves, and regions with special properties. Common local features include two major categories of corner point and region description methods.

[0053] Rectified Linear Unit (ReLU), also known as ReLU, is a commonly used activation function in artificial neural networks. It usually refers to a nonlinear function represented by a ramp function and its variants.

[0054] In daily life, various terminals have been applied in various fields. For example, with the development of the vehicle field, vehicles are gradually developing towards autonomous driving, and the use of vehicle terminals is essential in the field of autonomous driving. Vehicles will encounter various obstacles, such as other vehicles, pedestrians, utility poles, high-rise buildings, etc. It is particularly important for the vehicle to perceive the surrounding environment through the vehicle terminal. In terms of environmental perception, data can be collected and calculated through camera devices, ultrasonic radar, etc. to achieve obstacle detection. For example, common camera devices can include binocular cameras, monocular cameras, fisheye cameras, etc.

[0055] Please refer to Figure 1Fig. 1 shows a schematic diagram of a fisheye image captured by a terminal according to an example embodiment of the present application. As shown in Fig. 1, the fisheye image 100 contains a first obstacle 101 and a second obstacle 102. Figure 1

[0056] In the present application, the terminal can be a terminal device that can install the camera device described above, such as a terminal that can install a fisheye camera. The terminal can be a smartphone, a tablet computer, an e-book reader, smart glasses, a smart watch, an MP3 player, an MP4 player, a notebook computer, a laptop computer, a desktop computer, and the like.

[0057] Optionally, the terminal can be an Internet of Things device that can install the camera device described above, such as a sensor device, a computer of an Internet of Things terminal, for example, a fixed, portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted device. For example, a Station (STA), a subscriber unit, a subscriber station, a mobile station, a mobile, a remote station, an access point, a remote terminal, an access terminal, a user terminal, a user agent, a user device. Alternatively, the terminal can also be a device of an unmanned aerial vehicle. Alternatively, the terminal can also be a vehicle-mounted device, such as a carputer with audio playback function, or a wireless communication device connected to a carputer.

[0058] In the above Figure 1 , the terminal can trigger a detection function of the reachable space of the fisheye image 100, wherein the reachable space generally refers to the space that can be reached in the image. For example, a vehicle-mounted terminal can obtain the range of space that can be driven by itself through the fisheye image. Generally, the terminal can not only detect the positions of the first obstacle 101 and the second obstacle 102 in the fisheye image, but also detect the attributes of the first obstacle 101 and the second obstacle 102.

[0059] ​For example, after obtaining the position coordinates of each obstacle in the reachable space, the terminal can also combine the heat map to determine the attributes of each obstacle in the reachable space (for example, the attribute categories of each obstacle include static obstacles, dynamic obstacles, and suspended obstacles). However, when determining the attributes through the heat map, the data used is relatively single, the accuracy of the obstacle attribute determination is not high enough, and errors in determining the obstacle attributes are prone to occur.

[0060] To improve the accuracy of determining the obstacle attributes, the present application provides a solution. After obtaining the attribute graph combined with the attribute classification result, the corresponding obstacle attributes under the global classification are also combined. According to different attribute results, the most accurate obstacle attributes are determined to improve the accuracy of determining the obstacle attributes.

[0061] Please refer to Figure 2 , which shows a method flowchart of an obstacle attribute determination method provided by an example embodiment of the present application. The obstacle attribute determination method can be applied to the terminal in the scene architecture shown in the above Figure 1 , and the method can be executed by the terminal. As shown in Figure 2 , the obstacle attribute determination method can include the following steps.

[0062] Step 201: Obtain an attribute feature image according to a first fisheye image, the attribute feature image including attribute features of each obstacle in the first fisheye image.

[0063] Optionally, the first fisheye image can be a fisheye image obtained by the terminal through a fisheye camera, or the first fisheye image can be a normal image obtained by the terminal through a normal camera, and the corresponding fisheye image is obtained by distorting the normal image. The present application does not limit the way of obtaining the first fisheye image.

[0064] Optionally, the terminal extracts the attribute features of the first fisheye image obtained by the terminal through a pre-designed neural network to obtain the attribute feature image. For example, the terminal can determine the shape features and position features of each obstacle in the first fisheye image based on the first fisheye image, and obtain the corresponding attribute features according to the shape features and position features of each obstacle, thereby generating the attribute feature image including the attribute features of each obstacle. For example, the image parameters of the first fisheye image are (h1, w1, 3), where h1 represents the height of the first fisheye image, w1 represents the width of the first fisheye image, and 3 represents that each pixel point is 3 channels. After attribute feature extraction is performed on the first fisheye image, the attribute feature image can be as follows: (h1 / 4, w1 / 4, 128).

[0065] In step 202, attribute classification is performed on the attribute feature image to obtain an attribute map corresponding to each obstacle attribute, and the attribute map includes the confidence of the obstacle attribute corresponding to each pixel point in the first fisheye image.

[0066] Optionally, the terminal performs attribute classification on the attribute feature image to obtain an attribute map corresponding to each obstacle attribute, and the attribute map includes the confidence of the obstacle corresponding to each pixel point in the first fisheye image in the attribute map corresponding to the obstacle attribute. That is, the terminal performs attribute classification on the attribute feature image according to the attribute features contained in the attribute feature image, and can obtain the confidence of the corresponding obstacle attribute classification result.

[0067] For example, the obstacle attribute categories of each obstacle are classified into static obstacles, dynamic obstacles, and suspended obstacles, the terminal can obtain an attribute map corresponding to static obstacles, the attribute map corresponding to static obstacles includes that the obstacle attribute of each obstacle in the first fisheye image is a static obstacle, and the confidence that the obstacle attribute of each obstacle is a static obstacle. The terminal can also obtain an attribute map corresponding to dynamic obstacles, the attribute map corresponding to dynamic obstacles includes that the obstacle attribute of each obstacle in the first fisheye image is a dynamic obstacle, and the confidence that the obstacle attribute of each obstacle is a dynamic obstacle. The terminal can also obtain an attribute map corresponding to suspended obstacles, the attribute map corresponding to suspended obstacles includes that the obstacle attribute of each obstacle in the first fisheye image is a suspended obstacle, and the confidence that the obstacle attribute of each obstacle is a dynamic obstacle.

[0068] In step 203, an attribute vector is obtained according to the attribute feature image, and the attribute vector is used to represent the obstacle attribute and the confidence corresponding to each pixel point in the first fisheye image in the global classification.

[0069] Optionally, the terminal can also obtain an attribute vector based on the attribute feature image, and the attribute vector is based on the obstacle attribute and the confidence corresponding to each pixel point in the first fisheye image in the global classification. The global classification can refer to the classification result obtained by the terminal based on the global feature obtained by inputting the attribute feature image into the full connection layer. In the attribute vector, not only the classification result is included, but also the confidence of the classification result. For example, the obstacle attribute of the obstacle at one position is a suspended obstacle, and the confidence that the obstacle at the position is a suspended obstacle is also included in the attribute vector.

[0070] In step 204, the obstacle attribute of the obstacle at a first position in the first fisheye image is determined according to the attribute vector and the attribute map corresponding to each obstacle attribute, and the first position is a pixel point position included by any obstacle in the first fisheye image.

[0071] Optionally, the terminal determines the obstacle attribute corresponding to the pixel position in the first fisheye image according to the attribute vector obtained above and the attribute graph corresponding to each obstacle attribute. That is, in the manner of performing attribute classification on the attribute feature image to obtain the attribute graph corresponding to each obstacle attribute, the terminal obtains the obstacle attribute corresponding to each obstacle and the confidence of the obstacle attribute, and the terminal can also obtain the attribute vector representing the obstacle attribute corresponding to each pixel in the first fisheye image under global classification and the confidence based on the full connection layer according to the attribute feature image, combines the two classification results, and determines the obstacle attribute of the obstacle at the first position in the first fisheye image.

[0072] Optionally, the first position can be a boundary position of a certain obstacle determined by the terminal through marking or calculation. For example, the terminal determines the boundary of any one of the obstacles through the boundary determination model, and takes any one position on the boundary as the first position. The boundary determination model can be trained by the developer in advance using different fisheye images and set in the terminal.

[0073] In summary, the terminal according to the present application obtains an attribute feature image according to the first fisheye image, the attribute feature image including the attribute features of each obstacle in the first fisheye image; performs attribute classification on the attribute feature image to obtain an attribute graph corresponding to each obstacle attribute, the attribute graph including the confidence of the obstacle attribute corresponding to each pixel in the first fisheye image; obtains an attribute vector according to the attribute feature image, the attribute vector being used to represent the obstacle attribute corresponding to each pixel in the first fisheye image under global classification and the confidence; and determines the obstacle attribute of the obstacle at the first position in the first fisheye image according to the attribute vector and the attribute graph corresponding to each obstacle attribute. The present application obtains the attribute graph of each attribute and the attribute vector representing the global classification based on the attribute feature image after obtaining the attribute feature image of the first fisheye image, determines the obstacle attribute of the obstacle at the first position by combining the attribute vector and the attribute graph, and obtains the obstacle attribute based on the two classification results, so that the determination of the obstacle attribute is more accurate.

[0074] In a possible implementation manner, when the attribute feature image is obtained, the attribute feature image can be obtained based on the manner of generating a feature pyramid, attribute feature extraction is performed on a layer with the most features in the feature pyramid, and attribute image features are obtained, so as to improve the accuracy of determining the obstacle attribute.

[0075] Please refer to Figure 3 , which shows a method flowchart of a method for determining an obstacle attribute provided by an example embodiment of the present application. The method for determining an obstacle attribute can be applied to the terminal in the scene architecture shown in Figure 1 , and the method can be executed by the terminal. As shown in Figure 3As shown, the obstacle attribute determination method can include the following steps.

[0076] Step 301, according to the first fisheye image and the target residual network, a multi-layer feature image is obtained.

[0077] Optionally, the first fisheye image can refer to the description in the above step 201, which will not be repeated here. In this application, the terminal obtains the multi-layer feature image by using the target residual network according to the obtained first fisheye image. The target residual network is a multi-layer feature designed by the developer to extract the first fisheye image. For example, the target residual network can be a Resnet34 residual network. Taking the image parameters of the first fisheye image as an example (h1, w1, 3), after the terminal processes the first fisheye image through the Resnet34 residual network, the obtained multi-layer feature is as follows: (h1 / 4, w1 / 4, 64), (h1 / 8, w1 / 8, 128), (h1 / 16, w1 / 16, 256), (h1 / 32, w1 / 32, 512).

[0078] Step 302, according to the multi-layer feature image and the pyramid network, a feature pyramid is generated.

[0079] Among them, the feature pyramid includes the image features of the multi-layer first fisheye image.

[0080] Optionally, the terminal generates a feature pyramid according to the multi-layer feature image obtained after processing by the above target residual network and the pyramid network. The pyramid network is the same as the target residual network, which is also designed by the developer and set in the terminal. For example, the pyramid network can be a feature pyramid network (FPN), wherein the FPN is a top-down feature fusion method, and the FPN is a multi-scale target detection algorithm, that is, there is not only one feature prediction layer. Although some algorithms also use multi-scale feature fusion for target detection, they often only use one scale of feature obtained after fusion, which can combine the semantic information of the top layer feature and the detail information of the bottom layer feature, but in the feature deconvolution process, some deviations will be caused, and only using the feature obtained after fusion for prediction will have adverse effects on detection accuracy. The FPN method starts from the above problems and can predict on multiple fused features of different scales to maximize the detection accuracy.

[0081] Step 303, attribute feature extraction is performed on the target layer in the feature pyramid to obtain attribute image features.

[0082] The target layer is a layer in the feature pyramid that contains the most features. Optionally, after obtaining the feature pyramid, the terminal extracts the layer containing the most features in the feature pyramid (i.e., the target layer), inputs the target layer into the attribute feature extraction model, and obtains attribute image features.

[0083] In a possible implementation, the terminal performs attribute feature extraction on the target layer according to the attribute feature extraction model, obtains attribute features in the target layer, and generates an attribute feature image according to the attribute features. Optionally, the image parameters of the generated attribute feature image are the same as the image parameters of the target layer. For example, the terminal processes the target layer by using a first convolutional layer in the attribute feature extraction model, and extracts attribute features in the target layer.

[0084] Optionally, taking the image parameters of the first fisheye image as (h1, w1, 3) and the image parameters of the target layer as (h1 / 4, w1 / 4, 128) after obtaining the corresponding feature pyramid as examples, the first convolutional layer can be: conv(128*64*3*3), where each number in the conv represents, in sequence, an output channel, an input channel, a channel width, and a channel height. The terminal performs convolution on the target layer and the first convolutional layer, and performs, in sequence, batch normalization (BN) and a first ReLU function on the convolution result to obtain an attribute feature image, where the image parameters of the attribute feature image are the same as the image parameters of the target layer. The image parameters of the attribute feature image are also (h1 / 4, w1 / 4, 128).

[0085] In step 304, the attribute feature image is classified according to attributes, and an attribute map corresponding to each type of obstacle attribute is obtained. The attribute map includes a confidence degree of an obstacle attribute corresponding to each pixel in the first fisheye image.

[0086] Optionally, after obtaining the attribute feature image, the terminal classifies the attribute feature image according to attributes, and obtains an attribute map corresponding to each type of obstacle attribute. Each type of obstacle attribute is related to actual needs. A developer can design N types of obstacle attributes, where N is greater than or equal to 1. In this step, the terminal classifies the attribute feature image according to attributes, and can obtain N attribute maps corresponding to N types of obstacle attributes.

[0087] In a possible implementation, the terminal can input the obtained attribute feature image into an attribute map obtaining model, and the attribute map obtaining model can perform attribute classification on the input attribute feature image to obtain an attribute map corresponding to each obstacle attribute. If the developer designs 6 obstacle attributes, the attribute map obtaining model can perform attribute classification on the input attribute feature image to obtain 6 attribute maps, each attribute map corresponding to one attribute, and each pixel point in each attribute map further including the confidence of the obstacle attribute corresponding to the pixel point. Optionally, the attribute map obtaining model can also be obtained by the developer through training of different sample attribute feature images in advance. After inputting the attribute feature image, the attribute map obtaining model performs a series of processing on the attribute feature image to output attribute maps corresponding to different obstacle attributes.

[0088] For example, still taking the image parameters (h1, w1, 3) of the first fisheye image as an example, after obtaining the corresponding feature pyramid, the image parameters of the target layer are (h1 / 4, w1 / 4, 128), and the image parameters of the obtained attribute feature image are also (h1 / 4, w1 / 4, 128). The attribute map obtaining model can perform attribute classification on the attribute feature image through a second convolution layer to obtain attribute maps corresponding to 6 different obstacle attributes.

[0089] For example, the second convolution layer can be conv(6*128*3*3), where the terminal convolves the attribute feature image with the second convolution layer, and sequentially performs batch normalization (Batch Normalization, BN) and a second ReLU function on the convolution result to obtain attribute maps corresponding to different obstacle attributes, where the image size of the attribute map is the same as the image size of the attribute feature image. That is, the image size of the attribute map is also (h1 / 4, w1 / 4), and the attribute maps corresponding to the 6 different obstacle attributes obtained by combining together are (6, h1 / 4, w1 / 4).

[0090] In step 305, an attribute vector is obtained according to the attribute feature image, and the attribute vector is used to represent the obstacle attribute and the confidence corresponding to each pixel point in the first fisheye image in the global classification.

[0091] Optionally, the terminal can also input the obtained attribute feature image into a vector obtaining model to obtain the attribute vector. Optionally, the vector obtaining model can also be obtained by the developer through training of different sample attribute feature images in advance. After inputting the attribute feature image, the vector obtaining model performs a series of processing on the attribute feature image to output the corresponding attribute vector.

[0092] Optionally, the terminal can perform one-dimensional transformation on the attribute feature image through a vector acquisition model to obtain a one-dimensional feature map; and generate the attribute vector according to the one-dimensional feature map and a full connection layer. For example, the vector acquisition model transforms the attribute feature image into a one-dimensional feature map through a flatten function, and inputs the obtained one-dimensional feature map into the full connection layer to generate the corresponding attribute vector. Optionally, the full connection layer in the vector acquisition model can be as follows: fully connected layer (h*w*128 / 16, x*6), wherein x is the number of output attributes, and *6 means the confidence of 6 obstacle attributes at each position.

[0093] Still taking the image parameters of the first fisheye image above as (h1, w1, 3), the corresponding feature pyramid is obtained, the image parameters of the target layer are (h1 / 4, w1 / 4, 128), and the image parameters of the obtained attribute feature image are (h1 / 4, w1 / 4, 128) as an example. After obtaining the attribute feature image, the terminal inputs the attribute feature image into the vector acquisition model, performs one-dimensional transformation on the attribute feature image through a flatten function, transforms the attribute feature image from (h1 / 4, w1 / 4, 128) to (h*w*128 / 16) in one dimension, and generates the attribute vector (w1 / 4) based on the full connection layer (h*w*128 / 16, x*6).

[0094] In step 306, the attribute vector and the attribute map corresponding to each obstacle attribute are used to determine the obstacle attribute of the obstacle at the first position in the first fisheye image. The first position is a pixel position included in any obstacle in the first fisheye image.

[0095] Optionally, before step 306, the terminal can also determine the first position in the first fisheye image. For example, after generating the feature pyramid, the terminal can also obtain the initial boundary of the reachable space in the first fisheye image based on the target layer and the boundary acquisition model. In one possible implementation manner, the terminal performs boundary extraction on the target layer according to the boundary acquisition model to obtain the boundary of the reachable space in the target layer; and takes the boundary of the reachable space in the target layer as the initial boundary of the reachable space in the first fisheye image. The initial boundary is a single-channel array, and the width of the initial boundary is the same as the width of the target layer. For example, the terminal sequentially processes the target layer through the third convolution layer and the fourth convolution layer in the boundary acquisition model to extract the initial boundary of the reachable space in the target layer.

[0096] Optionally, the image parameter of the first fisheye image is (h1, w1, 3), and the image parameter of the target layer is (h1 / 4, w1 / 4, 128) after the corresponding feature pyramid is obtained. The third convolutional layer can be: conv(128*64*3*3), wherein each number in the conv represents output channel, input channel, kernel width, and kernel height in turn. The terminal convolves the target layer with the third convolutional layer, and performs batch normalization (BN) and the third ReLU function on the convolution result in turn to obtain a regression feature map, wherein the image parameter of the regression feature map is the same as that of the target layer. The image parameter of the regression feature map is also (h1 / 4, w1 / 4, 128).

[0097] Optionally, after obtaining the regression feature map in the boundary acquisition model, the regression feature map is processed by a fourth convolutional layer to extract the initial boundary of the reachable space in the target layer. The fourth convolutional layer can be: conv(320*128*w1 / 4*h1 / 4). The boundary acquisition model convolves the regression feature map obtained above with the second convolutional layer, and performs BN and the fourth ReLU function on the convolution result in turn to obtain the initial boundary of the reachable space in the target layer, and takes the obtained initial boundary of the reachable space in the target layer as the initial boundary of the reachable space in the first fisheye image. Optionally, the initial boundary can be denoted as y_init. Based on the above example, y_init obtained in this step is a single-channel array, and the width of the initial boundary is the same as the width of the target layer, both being w1 / 4.

[0098] It should be noted that the obtained initial boundary can also be corrected to determine a more accurate initial boundary in some schemes, and the present application does not limit comparison. Alternatively, the terminal can also detect obstacles in the first fisheye image before step 306 to detect the positions of the obstacles contained therein or the boundaries of the obstacles, and take the position of any pixel point contained in the first fisheye image as the first position.

[0099] Optionally, the terminal determines first obstacle attribute information of the obstacle at the first position in the first fisheye image according to the attribute graph corresponding to each attribute, the first obstacle attribute information comprising a first obstacle attribute of the obstacle at the first position and a confidence degree corresponding to the first obstacle attribute; determines second obstacle attribute information of the obstacle at the first position in the first fisheye image according to the attribute vector, the second obstacle attribute information comprising a second obstacle attribute of the obstacle at the first position and a confidence degree corresponding to the second obstacle attribute; and determines the obstacle attribute of the obstacle at the first position according to the first obstacle attribute information and the second obstacle attribute information.

[0100] In a possible implementation, the terminal can determine the first obstacle attribute information of the obstacle at the first position in the first fisheye image according to the attribute graph corresponding to each attribute as follows: determine each obstacle attribute and each confidence degree of the obstacle at the first position in the attribute graph corresponding to each attribute; determine the highest confidence degree from the confidence degrees according to the confidence degrees; and obtain the highest confidence degree and the obstacle attribute corresponding to the highest confidence degree as the first obstacle attribute information.

[0101] That is, in different attribute graphs, there are three obstacle attributes (static obstacle, dynamic obstacle or suspended obstacle) at the first position and a confidence degree corresponding to each obstacle attribute, the terminal determines each obstacle attribute and each confidence degree of the obstacle at the first position in the attribute graph corresponding to each attribute, determines the highest confidence degree from the confidence degrees according to the confidence degrees, and obtains the highest confidence degree and the obstacle attribute corresponding to the highest confidence degree as the first obstacle attribute information. For example, in the attribute graph of the obstacle attribute being a static obstacle, the obstacle attribute of the obstacle at the first position is a static obstacle with a confidence degree of 0.1, in the attribute graph of the obstacle attribute being a dynamic obstacle, the obstacle attribute of the obstacle at the first position is a dynamic obstacle with a confidence degree of 0.4, and in the attribute graph of the obstacle attribute being a suspended obstacle, the obstacle attribute of the obstacle at the first position is a suspended obstacle with a confidence degree of 0.8. Therefore, by comparison, the terminal obtains the confidence degree of 0.8 and the corresponding suspended obstacle as the first obstacle attribute information of the obstacle at the first position in the first fisheye image.

[0102] In a possible implementation, the terminal can determine the first obstacle attribute information of the obstacle at the first position in the first fisheye image according to the attribute graph corresponding to each attribute as follows: determine each obstacle attribute and each confidence degree of the obstacle at the first position in the attribute graph corresponding to each attribute; fuse the attribute graphs corresponding to each attribute according to the confidence degrees to obtain a comprehensive attribute graph; and determine the first obstacle attribute information of the obstacle at the first position in the first fisheye image according to the comprehensive attribute graph.

[0103] Optionally, the terminal can also fuse different attribute graphs to obtain a comprehensive attribute graph. For example, in the above different attribute graphs, there are three obstacle attributes (static obstacle, dynamic obstacle or suspended obstacle) corresponding to the first position and the confidence of each obstacle attribute. In the attribute graph where the obstacle attribute is a static obstacle, the confidence of the obstacle attribute of the obstacle at the first position is 0.1. In the attribute graph where the obstacle attribute is a dynamic obstacle, the confidence of the obstacle attribute of the obstacle at the first position is 0.4. In the attribute graph where the obstacle attribute is a suspended obstacle, the confidence of the obstacle attribute of the obstacle at the first position is 0.8. Then, by comparison, the terminal takes the confidence of 0.8 and the corresponding suspended obstacle as the first obstacle attribute information of the obstacle at the first position in the first fisheye image. The terminal can obtain the respective first obstacle attribute information of each position by the above screening and fuse them in one attribute graph. When the first obstacle attribute information of the obstacle at the first position is needed, it can be directly obtained from the fused comprehensive attribute graph.

[0104] Optionally, the terminal can determine the obstacle attribute of the obstacle at the first position according to the first obstacle attribute information and the second obstacle attribute information as follows: when the confidence corresponding to the first obstacle attribute is greater than the confidence corresponding to the second obstacle attribute, the obstacle attribute of the obstacle at the first position is determined as the first obstacle attribute; when the confidence corresponding to the first obstacle attribute is less than the confidence corresponding to the second obstacle attribute, the obstacle attribute of the obstacle at the first position is determined as the second obstacle attribute. That is, the terminal selects the obstacle attribute with higher confidence in the two classification results as the obstacle attribute of the obstacle at the first position. For example, the first obstacle attribute information is represented as prop_id_hm(w / 4) and the confidence is confidence one. The second obstacle attribute information is represented as property_id_vec(w / 4) and the confidence is confidence two. When confidence one is greater than confidence two, the obstacle attribute of the obstacle at the first position is determined as the obstacle attribute corresponding to prop_id_hm(w / 4). When confidence two is greater than confidence one, the obstacle attribute of the obstacle at the first position is determined as the obstacle attribute corresponding to property_id_vec(w / 4).

[0105] To sum up, the terminal according to the first fisheye image, obtains the attribute feature image, the attribute feature image includes the attribute feature of each obstacle in the first fisheye image; attribute classification is carried out on the attribute feature image, and the attribute graph corresponding to each obstacle attribute is obtained, and the attribute graph includes the confidence of the obstacle attribute corresponding to each pixel point in the first fisheye image; according to the attribute feature image, the attribute vector is obtained, and the attribute vector is used to represent the obstacle attribute and the confidence corresponding to each pixel point in the first fisheye image under global classification; according to the attribute vector and the attribute graph corresponding to each obstacle attribute, the obstacle attribute of the obstacle at the first position in the first fisheye image is determined. After the attribute feature image is obtained from the first fisheye image, the attribute graph of each attribute and the attribute vector representing the global classification are obtained based on the attribute feature image, the obstacle attribute of the obstacle at the first position is determined by combining the attribute vector and the attribute graph, and the obstacle attribute is obtained based on two classification results, so that the obstacle attribute is more accurate.

[0106] The following is an embodiment of the device of the application, which can be used to execute the method embodiment of the application. For details not disclosed in the device embodiment of the application, please refer to the method embodiment of the application.

[0107] Please refer to Figure 4 which shows the structure block diagram of a kind of obstacle attribute determination device provided by an exemplary embodiment of the application, the obstacle attribute determination device 400 can be applied to terminal, the obstacle attribute determination device includes:

[0108] The first acquisition module 401 is used to obtain the attribute feature image according to the first fisheye image, and the attribute feature image includes the attribute feature of each obstacle in the first fisheye image;

[0109] The second acquisition module 402 is used to carry out attribute classification on the attribute feature image, and the attribute graph corresponding to each obstacle attribute is obtained, and the attribute graph includes the confidence of the obstacle attribute corresponding to each pixel point in the first fisheye image;

[0110] The third acquisition module 403 is used to obtain the attribute vector according to the attribute feature image, and the attribute vector is used to represent the obstacle attribute and the confidence corresponding to each pixel point in the first fisheye image under global classification;

[0111] The attribute determination module 404 is used to determine the obstacle attribute of the obstacle at the first position in the first fisheye image according to the attribute vector and the attribute graph corresponding to each obstacle attribute, and the first position is the pixel point position contained by any one obstacle in the first fisheye image.

[0112] In summary, the terminal according to the first fisheye image, obtains the attribute feature image, the attribute feature image includes the attribute feature of each obstacle in the first fisheye image; the attribute classification is carried out to the attribute feature image, and the attribute graph corresponding to each obstacle attribute is obtained, and the attribute graph includes the confidence of the obstacle attribute corresponding to each pixel point in the first fisheye image; according to the attribute feature image, the attribute vector is obtained, and the attribute vector is used to represent the obstacle attribute and the confidence corresponding to each pixel point in the first fisheye image under global classification; according to the attribute vector and the attribute graph corresponding to each obstacle attribute, the obstacle attribute of the obstacle at the first position in the first fisheye image is determined. After the attribute feature image is obtained from the first fisheye image, the attribute graph of each attribute and the attribute vector representing the global classification are obtained based on the attribute feature image, the obstacle attribute of the obstacle at the first position is determined by combining the attribute vector and the attribute graph, and the obstacle attribute is obtained based on two classification results, so that the acquisition of the obstacle attribute is more accurate.

[0113] Optionally, the third obtaining module 403 comprises a first obtaining unit and a first generating unit.

[0114] The first obtaining unit is configured to perform one-dimensional transformation on the attribute feature image to obtain a one-dimensional feature map.

[0115] The first generating unit is configured to generate the attribute vector according to the one-dimensional feature map and a full connection layer.

[0116] Optionally, the attribute determining module 404 comprises a first determining unit, a second determining unit and a third determining unit.

[0117] The first determining unit is configured to determine first obstacle attribute information of the obstacle at the first position in the first fisheye image according to the attribute graph corresponding to each attribute, wherein the first obstacle attribute information comprises a first obstacle attribute of the obstacle at the first position and a confidence corresponding to the first obstacle attribute.

[0118] The second determining unit is configured to determine second obstacle attribute information of the obstacle at the first position in the first fisheye image according to the attribute vector, wherein the second obstacle attribute information comprises a second obstacle attribute of the obstacle at the first position and a confidence corresponding to the second obstacle attribute.

[0119] The third determining unit is configured to determine the obstacle attribute of the obstacle at the first position according to the first obstacle attribute information and the second obstacle attribute information.

[0120] Optionally, the first determining unit is further configured to,

[0121] determining each obstacle attribute of the obstacle at the first position in the attribute graph corresponding to each attribute and each confidence;

[0122] determining the highest confidence among the confidences according to the confidences;

[0123] obtaining the highest confidence and the obstacle attribute corresponding to the highest confidence as the first obstacle attribute information.

[0124] Optionally, the first determining unit is further configured to,

[0125] determining each obstacle attribute of the obstacle at the first position in the attribute graph corresponding to each attribute and each confidence;

[0126] fusing the attribute graph corresponding to each attribute according to the confidences to obtain a comprehensive attribute graph;

[0127] determining the first obstacle attribute information of the obstacle at the first position in the first fisheye image according to the comprehensive attribute graph.

[0128] Optionally, the third determining unit is further configured to,

[0129] when the confidence corresponding to the first obstacle attribute is greater than the confidence corresponding to the second obstacle attribute, determining the obstacle attribute of the obstacle at the first position as the first obstacle attribute;

[0130] when the confidence corresponding to the first obstacle attribute is less than the confidence corresponding to the second obstacle attribute, determining the obstacle attribute of the obstacle at the first position as the second obstacle attribute.

[0131] Optionally, the first obtaining module 401 includes a second obtaining unit, a second generating unit and a third obtaining unit.

[0132] The second obtaining unit is configured to obtain a plurality of layer feature images according to the first fisheye image and a target residual network.

[0133] The second generating unit is configured to generate a feature pyramid according to the plurality of layer feature images and a pyramid network, the feature pyramid including image features of a plurality of layers of the first fisheye image.

[0134] The third obtaining unit is configured to perform attribute feature extraction on a target layer in the feature pyramid to obtain the attribute image features, the target layer being a layer in the feature pyramid that contains the most features.

[0135] Figure 5is a structural schematic diagram of a terminal provided by an exemplary embodiment of the present application. As shown in Figure 5 The terminal 500 includes a central processing unit (CPU) 501, a system memory 504 including a random access memory (RAM) 502 and a read-only memory (ROM) 503, and a system bus 505 that couples the system memory 504 to the central processing unit 501. The terminal 500 also includes a basic input / output system (I / O system) 506 that helps to transfer information between various devices within the computer, and a mass storage device 507 for storing an operating system 512, application programs 513, and other program modules 514.

[0136] The basic input / output system 506 includes a display 508 for displaying information and input devices 509 such as a mouse, keyboard, and the like for inputting information by a user. The display 508 and the input devices 509 are both connected to the central processing unit 501 through an input / output controller 510 connected to the system bus 505. The basic input / output system 506 can also include the input / output controller 510 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 510 also provides output to a display screen, printer, or other type of output device.

[0137] The mass storage device 507 is connected to the central processing unit 501 through a mass storage controller (not shown) connected to the system bus 505. The mass storage device 507 and its associated computer-readable media provide nonvolatile storage for the terminal 500. That is, the mass storage device 507 can include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0138] The computer readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. It should be understood by those skilled in the art that computer storage media does not limit to the above-mentioned several kinds. The system memory 504 and the mass storage device 507 mentioned above can be collectively referred to as memory.

[0139] The terminal 500 can be connected to the Internet or other network devices through a network interface unit 511 connected to the system bus 505.

[0140] The memory further includes one or more programs stored in the memory, and the central processing unit 501 implements all or part of the steps performed by the terminal in the method provided by each embodiment of the present application by executing the one or more programs. Optionally, the terminal described above can be integrated into a vehicle as a vehicle terminal.

[0141] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part.

[0142] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a high-density digital video disc (Digital Video Disc, DVD)), or a semiconductor medium (for example, a solid state disk (Solid State Disk, SSD)) and the like.

[0143] The embodiment of the present application also discloses a vehicle, which comprises a vehicle terminal, and the vehicle terminal comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to implement the method for determining the attribute of the obstacle in the above method embodiment. Optionally, the terminal can be the vehicle terminal in the embodiment.

[0144] The embodiment of the present application also discloses a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method in the above method embodiment.

[0145] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0146] In various embodiments of the present application, it should be understood that the size of the sequence number of the above processes does not mean the inevitable sequence of execution order, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0147] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e. they can be located in one place, or they can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0148] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0149] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer accessible memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product stored in a storage medium includes a plurality of steps for causing a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in the computer device) to execute the above-mentioned methods of various embodiments of the present application.

[0150] Those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to carry or store data in a computer readable manner.

[0151] The above describes the method and device for determining the attribute of an obstacle, the terminal and the storage medium, and the principles and implementation manners of the present application are described by using examples. The above description of the embodiments is only used to help understand the method and the core idea of the present application. Meanwhile, for those skilled in the art, the implementation manner and the application range can be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for determining obstacle properties, characterized in that, The method is executed by a terminal, and the method includes: Based on the first fisheye image, an attribute feature image is obtained, wherein the attribute feature image includes the attribute features of each obstacle in the first fisheye image; The attribute feature image is classified to obtain an attribute map corresponding to each obstacle attribute. The attribute map includes the confidence level of the obstacle attribute corresponding to each pixel in the first fisheye image. The obstacle attributes include static obstacles, dynamic obstacles, and suspended obstacles. The step of classifying the attribute features of the image to obtain an attribute map corresponding to each obstacle attribute includes: classifying the attributes based on the attribute features using an attribute map acquisition model to obtain N attribute maps, each attribute map corresponding to one attribute, and each pixel in each attribute map including the confidence score of the obstacle attribute corresponding to the pixel, where N is the total number of obstacle attribute types; the attribute map acquisition model is obtained by training on different sample attribute feature images. Based on the attribute feature image, an attribute vector is obtained. The attribute vector is used to characterize the obstacle attribute and confidence level of each pixel in the first fisheye image under global classification. The global classification refers to the classification result obtained by the terminal after inputting the attribute feature image into the fully connected layer to obtain global features. Based on the attribute vector and the attribute map corresponding to each obstacle attribute, the obstacle attribute of the obstacle at the first position in the first fisheye image is determined, where the first position is the pixel position contained in any obstacle in the first fisheye image.

2. The method according to claim 1, characterized in that, The step of obtaining the attribute vector based on the attribute feature image includes: Perform a one-dimensional transformation on the attribute feature image to obtain a one-dimensional feature map; The attribute vector is generated based on the one-dimensional feature map and the fully connected layer.

3. The method according to claim 1, characterized in that, Determining the obstacle attribute of the obstacle at the first position in the first fisheye image based on the attribute vector and the attribute map corresponding to each obstacle attribute includes: Based on the attribute map corresponding to each attribute, the first obstacle attribute information of the obstacle at the first position in the first fisheye image is determined. The first obstacle attribute information includes the first obstacle attribute of the obstacle at the first position and the confidence level corresponding to the first obstacle attribute. Based on the attribute vector, the second obstacle attribute information of the obstacle at the first position in the first fisheye image is determined. The second obstacle attribute information includes the second obstacle attribute of the obstacle at the first position and the confidence level corresponding to the second obstacle attribute. Based on the first obstacle attribute information and the second obstacle attribute information, the obstacle attributes of the obstacle at the first location are determined.

4. The method according to claim 3, characterized in that, The step of determining the first obstacle attribute information of the obstacle at the first position in the first fisheye image based on the attribute map corresponding to each attribute includes: Determine the obstacle attributes and confidence levels of the obstacle at the first location in the attribute graph corresponding to each attribute; Based on the given confidence levels, determine the highest confidence level among the given confidence levels; The highest confidence level and the obstacle attribute corresponding to the highest confidence level are obtained as the first obstacle attribute information.

5. The method according to claim 3, characterized in that, The step of determining the first obstacle attribute information of the obstacle at the first position in the first fisheye image based on the attribute map corresponding to each attribute includes: Determine the obstacle attributes and confidence levels of the obstacle at the first location in the attribute graph corresponding to each attribute; Based on the confidence levels, the attribute graphs corresponding to each attribute are fused to obtain a comprehensive attribute graph; Based on the comprehensive attribute map, the first obstacle attribute information of the obstacle at the first position in the first fisheye image is determined.

6. The method according to claim 3, characterized in that, Determining the obstacle attributes of the obstacle at the first location based on the first obstacle attribute information and the second obstacle attribute information includes: When the confidence level corresponding to the first obstacle attribute is greater than the confidence level corresponding to the second obstacle attribute, the obstacle attribute of the obstacle at the first location is determined to be the first obstacle attribute; When the confidence level corresponding to the first obstacle attribute is less than the confidence level corresponding to the second obstacle attribute, the obstacle attribute of the obstacle at the first location is determined to be the second obstacle attribute.

7. The method according to any one of claims 1 to 6, characterized in that, The step of obtaining the attribute feature image based on the first fisheye image includes: Based on the first fisheye image and the target residual network, obtain multi-layer feature images; Based on the multi-layer feature image and the pyramid network, a feature pyramid is generated, wherein the feature pyramid includes image features of multiple layers of the first fisheye image; Attribute features are extracted from the target layer in the feature pyramid to obtain the attribute feature image. The target layer is the layer in the feature pyramid that contains the most features.

8. A device for determining the properties of an obstacle, characterized in that, The device is executed by a terminal, and the device includes: The first acquisition module is used to acquire an attribute feature image based on the first fisheye image, wherein the attribute feature image includes the attribute features of each obstacle in the first fisheye image; The second acquisition module is used to classify the attribute feature images to obtain an attribute map corresponding to each obstacle attribute. The attribute map includes the confidence score of the obstacle attribute corresponding to each pixel in the first fisheye image. The obstacle attributes include static obstacles, dynamic obstacles, and suspended obstacles. It is also used to perform attribute classification based on the attribute feature images using an attribute map acquisition model to obtain N attribute maps. Each attribute map corresponds to one attribute, and each pixel in each attribute map includes the confidence score of the obstacle attribute corresponding to that pixel. N is the total number of obstacle attribute types. The attribute map acquisition model is trained using different sample attribute feature images. The third acquisition module is used to acquire an attribute vector based on the attribute feature image. The attribute vector is used to characterize the obstacle attribute and confidence level of each pixel in the first fisheye image under global classification. The global classification refers to the classification result obtained by the terminal after inputting the attribute feature image into the fully connected layer to obtain global features. The attribute determination module is used to determine the obstacle attribute of an obstacle at a first position in the first fisheye image based on the attribute vector and the attribute map corresponding to each obstacle attribute. The first position is the pixel position contained in any obstacle in the first fisheye image.

9. A terminal, characterized in that, The terminal includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to implement the method for determining obstacle attributes as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for determining obstacle attributes as described in any one of claims 1 to 7.

Images