Method for determining three-dimensional reachable space, vehicle and computer-readable storage medium

Abstract

The embodiment of the present invention discloses a method for determining a three-dimensional reachable space, a vehicle, and a computer-readable storage medium, which are used to output a depth estimation map with high accuracy when the current position image to be processed is input into a trained image recognition model, so that the accuracy of the acquired three-dimensional reachable space is also high. The embodiment of the present invention may include: acquiring the current position image to be processed to be identified; performing depth estimation on the current position image to be processed by a trained image recognition model to obtain a depth estimation map corresponding to the current position image to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, and the reachable space mask is used to indicate the image position of the vehicle's reachable space in the sample image; determining the depth value corresponding to the two-dimensional reachable space included in the current position image to be processed according to the depth estimation map, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for determining a three-dimensional reachable space, a vehicle, and a computer-readable storage medium. Background Art

[0002] During the process of driving or parking, the vehicle needs to obtain the three-dimensional reachable space of the current position of the vehicle and then perform corresponding operation. Generally, the vehicle obtains the three-dimensional reachable space of the current position through three-dimensional perception by laser radar or monocular camera. However, the cost of laser radar to obtain three-dimensional reachable space is high, and the monocular camera directly converts the two-dimensional reachable space into the three-dimensional reachable space through the depth map. The accuracy of the three-dimensional reachable space obtained by this method is low. Summary of the invention

[0003] The embodiments of the present invention provide a method for determining a three-dimensional reachable space, a vehicle and a computer-readable storage medium. Since the accuracy of the depth estimation map in the trained image recognition model is relatively high, when the image of the current processing position to be identified is input into the trained image recognition model, a depth estimation map with relatively high accuracy will be output. In this way, based on the depth estimation map with relatively high accuracy, a three-dimensional reachable space with relatively high accuracy corresponding to the vehicle can be obtained.

[0004] A first aspect of an embodiment of the present invention provides a method for determining a three-dimensional reachable space, which may include:

[0005] Obtain the current position image to be processed to be identified;

[0006] Performing depth estimation on the current image at the position to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the current image at the position to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate an image position of a vehicle's reachable space in the sample image;

[0007] A depth value corresponding to the two-dimensional reachable space included in the current position image to be processed is determined according to the depth estimation map, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0008] Optionally, before obtaining the image of the current processing position to be identified, the method also includes: performing depth estimation on the sample image through the image recognition model to be trained to obtain a predicted depth map corresponding to the sample image; calculating a total loss value based on the predicted depth map, the true depth reference map and the reachable space mask; and updating the parameters in the image recognition model based on the total loss value.

[0009] Optionally, before calculating the total loss value based on the predicted depth map, the true depth reference map and the reachable space mask, the method also includes: obtaining a predicted height map corresponding to the sample image, and a height truth reference map corresponding to the sample image; and, calculating the total loss value based on the predicted depth map, the true depth reference map and the reachable space mask, includes: calculating the total loss value based on the predicted depth map, the true depth reference map, the predicted height map, the height truth reference map and the reachable space mask.

[0010] Optionally, obtaining the predicted height map corresponding to the sample image includes: performing geometric transformation on the predicted depth map to obtain the predicted height map corresponding to the sample image.

[0011] Optionally, the total loss value is calculated based on the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask, including: calculating the depth loss based on the predicted depth map, the true depth reference map and the reachable space mask; calculating the height loss based on the predicted height map, the true height reference map and the reachable space mask; calculating the total loss value based on the depth loss and the height loss.

[0012] Optionally, the method also includes: obtaining an image of a current position to be processed to be identified; performing height estimation on the image of the current position to be processed by a trained image recognition model to obtain a height estimation map corresponding to the image of the current position to be processed; the image recognition model is trained using a sample image, a height truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the vehicle's reachable space in the sample image; determining a height value corresponding to the two-dimensional reachable space included in the image of the current position to be processed according to the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0013] Optionally, before obtaining the image of the current position to be processed to be identified, the method also includes: performing height estimation on the sample image through the image recognition model to be trained to obtain a predicted height map corresponding to the sample image; obtaining a height truth reference map and a reachable space mask corresponding to the sample image; the reachable space mask is used to indicate the image position of the vehicle's reachable space in the height truth reference map; calculating a total loss value based on the predicted height map, the height truth reference map and the reachable space mask; and updating the parameters in the image recognition model based on the total loss value.

[0014] A second aspect of an embodiment of the present invention provides a vehicle, which may include:

[0015] An acquisition module is used to acquire the current position image to be processed to be identified;

[0016] A processing module is used to perform depth estimation on the current image of the position to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the current image of the position to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the vehicle's reachable space in the sample image; and the depth value corresponding to the two-dimensional reachable space included in the current image of the position to be processed is determined according to the depth estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0017] Optionally, the processing module is also used to perform depth estimation on the sample image through the image recognition model to be trained to obtain a predicted depth map corresponding to the sample image; calculate the total loss value based on the predicted depth map, the true depth reference map and the reachable space mask; and update the parameters in the image recognition model based on the total loss value.

[0018] Optionally, the acquisition module is further used to acquire a predicted height map corresponding to the sample image and a height true value reference map corresponding to the sample image;

[0019] The processing module is also used to calculate the total loss value based on the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask.

[0020] Optionally, the acquisition module is specifically used to perform geometric transformation on the predicted depth map to obtain a predicted height map corresponding to the sample image.

[0021] Optionally, the processing module is specifically used to calculate the depth loss based on the predicted depth map, the true depth reference map and the reachable space mask; calculate the height loss based on the predicted height map, the true height reference map and the reachable space mask; and calculate the total loss value based on the depth loss and the height loss.

[0022] Optionally, the acquisition module is further used to acquire the current position image to be processed to be identified;

[0023] The processing module is also used to perform height estimation on the current image of the position to be processed by using a trained image recognition model to obtain a height estimation map corresponding to the current image of the position to be processed; the image recognition model is trained using a sample image, a height truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the vehicle's reachable space in the sample image; the height value corresponding to the two-dimensional reachable space included in the current image of the position to be processed is determined based on the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0024] Optionally, the processing module is further used to estimate the height of the sample image through the image recognition model to be trained to obtain a predicted height map corresponding to the sample image;

[0025] The acquisition module is further used to acquire a height truth reference map and a reachable space mask corresponding to the sample image; the reachable space mask is used to indicate an image position of a vehicle's reachable space in the height truth reference map;

[0026] The processing module is also used to calculate the total loss value based on the predicted height map, the true height reference map and the reachable space mask; and update the parameters in the image recognition model according to the total loss value.

[0027] A third aspect of an embodiment of the present invention provides a vehicle, which may include:

[0028] A memory storing executable program code;

[0029] and a processor coupled to the memory;

[0030] The processor calls the executable program code stored in the memory, and when the executable program code is executed by the processor, the processor implements the method as described in the first aspect of the embodiment of the present invention.

[0031] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having executable program code stored thereon. When the executable program code is executed by a processor, the method described in the first aspect of the embodiment of the present invention is implemented.

[0032] A fifth aspect of an embodiment of the present invention discloses a computer program product. When the computer program product runs on a computer, the computer is enabled to execute any one of the methods disclosed in the first aspect of the embodiment of the present invention.

[0033] A sixth aspect of an embodiment of the present invention discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes any one of the methods disclosed in the first aspect of the embodiment of the present invention.

[0034] It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

[0035] In an embodiment of the present invention, an image of a current position to be processed to be identified is obtained; depth estimation is performed on the image of the current position to be processed by a trained image recognition model to obtain a depth estimation map corresponding to the image of the current position to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the reachable space of the vehicle in the sample image; the depth value corresponding to the two-dimensional reachable space included in the image of the current position to be processed is determined according to the depth estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space. Since the accuracy of the depth estimation map in the trained image recognition model is relatively high, when the image of the current position to be processed to be identified is input into the trained image recognition model, a depth estimation map with relatively high accuracy will be output, so that the three-dimensional reachable space with relatively high accuracy corresponding to the vehicle can be obtained according to the depth estimation map with relatively high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments and the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and other drawings can be obtained based on these drawings.

[0037] Figure 1 A schematic diagram of an embodiment of a method for determining a three-dimensional reachable space in an embodiment of the present invention;

[0038] Figure 2 It is a schematic diagram of another embodiment of the method for determining a three-dimensional reachable space in an embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram of another embodiment of the method for determining a three-dimensional reachable space in an embodiment of the present invention;

[0040] Figure 4 It is a schematic diagram of another embodiment of the method for determining a three-dimensional reachable space in an embodiment of the present invention;

[0041] Figure 5 A schematic diagram of a vehicle in an embodiment of the present invention;

[0042] Figure 6 FIG. 2 is a schematic diagram of another embodiment of a vehicle in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] An embodiment of the present invention provides a method for determining a three-dimensional reachable space, a vehicle, and a computer-readable storage medium. Since the accuracy of the depth estimation map in the trained image recognition model is relatively high, when the image of the current position to be processed to be identified is input into the trained image recognition model, a depth estimation map with relatively high accuracy will be output. In this way, based on the depth estimation map with relatively high accuracy, a three-dimensional reachable space with relatively high accuracy corresponding to the vehicle can be obtained.

[0044] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments in the present invention, they should all fall within the scope of protection of the present invention.

[0045] The technical solution of the present invention is further described below by way of embodiments. Figure 1 FIG. 1 is a schematic diagram of an embodiment of a method for determining a three-dimensional reachable space in an embodiment of the present invention, which may include:

[0046] 101. Obtain a current position image to be processed to be identified.

[0047] It can be understood that the current position image to be processed (Image) can be obtained by taking a picture with a monocular camera arranged on the vehicle, and the number of the current position image to be processed is not limited.

[0048] Optionally, the vehicle acquiring the image of the current position to be processed may include: the vehicle detecting the current brightness of the current position to be detected; and when the current brightness is within a preset brightness range, the vehicle acquiring the image of the current position to be processed.

[0049] It should be noted that the preset brightness range is an interval constructed by the first brightness threshold and the second brightness threshold. The current brightness is within the preset brightness range, that is, the current brightness is greater than the first brightness threshold and less than or equal to the second brightness threshold. The preset brightness threshold can be set before the vehicle leaves the factory or can be set by the user, which is not specifically limited here.

[0050] For example, it is assumed that the first brightness threshold is 120 candela / square meter (abbreviated as cd / m 2 ), the second brightness threshold is 150cd / m 2 , the preset brightness range is (120cd / m 2 , 150cd / m 2 ). The current brightness of the vehicle detection position is 136cd / m 2 , the 136cd / m 2Within the preset brightness setting range (120cd / m 2 , 150cd / m 2 ), at this time, the vehicle obtains the current position image to be processed.

[0051] In this way, when the current brightness of the current position to be measured is within the preset brightness range, the acquired image of the current position to be processed is clearer, which is convenient for the vehicle to perform subsequent processing on the image of the current position to be processed.

[0052] 102. Perform depth estimation on the image at the current position to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the image at the current position to be processed.

[0053] Among them, the image recognition model is trained using a sample image, a depth truth reference image corresponding to the sample image, and a reachable space mask, and the reachable space mask is used to indicate the image position of the vehicle's reachable space in the sample image.

[0054] It should be noted that the sample images may be obtained by taking pictures with a monocular camera installed on the vehicle, and the number of the sample images is not limited.

[0055] The depth truth values ​​in the depth truth reference images corresponding to the sample images can be obtained based on the light beams collected by the laser radar installed on the vehicle. The depth truth value is the relatively accurate depth annotation data measured by the laser radar of the vehicle.

[0056] The accessible space mask is a preset two-dimensional accessible space mask, which may be set in the vehicle before leaving the factory.

[0057] Optionally, the trained image recognition model includes a trained deep residual network (Deep ResidualNetwork, ResNet) and a trained feature pyramid network (Feature Pyramid Networks, FPN); the vehicle performs depth estimation on the current position image to be processed through the trained image recognition model to obtain a depth estimation map corresponding to the current position image to be processed, which may include: the vehicle inputs the current position image to the trained ResNet to obtain N layers of features (multi-level features), where N is an integer greater than or equal to 1; the vehicle selects n layers of features from the N layers of features and inputs them into the trained FPN to obtain a first fused feature (FPN feature), where n is an integer greater than or equal to 1; the vehicle passes the first fused feature through a deep convolution layer (Depth Head) to obtain a depth estimation map corresponding to the current position image to be processed. Exemplarily, assume that N is 5 and n is 4.

[0058] It is understandable that the vehicle selects n layers of features from N layers of features to input into the trained FPN, that is, N ≥ n. After obtaining the N layers of features, the vehicle does not need to input all the N layers of features into the trained FPN. It only needs to select some layers of features (such as n layers of features) from the N layers of features and input them into the trained FPN to obtain the first fused feature. Optionally, the selection of n layers of features can be random by the vehicle or user-defined, which is not specifically limited here.

[0059] It should be noted that the trained ResNet refers to a network that can extract features of different layers in the current image to be processed. The more layers the trained ResNet has, the richer the features extracted by the trained ResNet in the current image to be processed; the deeper the network depth of the trained ResNet, the more abstract the features extracted by the trained ResNet in the current image to be processed.

[0060] Optionally, the trained ResNet may be a trained ResNet34. The trained ResNet34 has a better network convergence effect.

[0061] It should be noted that a trained FPN is one that can perform traditional top-down feature convolution and fuse multiple layers of features.

[0062] Optionally, the depth convolution layer can be a two-dimensional convolution (2Dimensional convolution, 2Dconvolution).

[0063] The Depth Head can be conv(output_chn, input_chn, width_kernel_size, height_kernel_size)+BatchNorm+Relu+conv(output_chn, input_chn, width_kernel_size, height_kernel_size)+BatchNorm+Relu.

[0064] Exemplarily, the Depth Head can be conv(64, 128, 3, 3)+BatchNorm+Relu+conv(1, 64, 3, 3)+BatchNorm+Relu.

[0065] Optionally, the vehicle inputs the current image of the position to be processed into the trained ResNet to obtain N layers of features, which may include: the vehicle inputs the height (height, h), width (width, w) and number of color channels of the current image of the position to be processed into the trained ResNet to obtain N layers of features.

[0066] Among them, the height of the current position image to be processed corresponding to each layer of features in the N layers of features is different, the width of the current position image to be processed corresponding to each layer of features is different, and the number of color channels of the current position image to be processed corresponding to each layer of features is different.

[0067] Exemplarily, the vehicle inputs the height h, width w and 3 color channels of the image at the current position to be processed into the trained ResNet34 to obtain 5 layers of features; wherein the height h, width w and 3 color channels can be represented by (h, w, 3); then, the vehicle inputs 4 layers of features out of the 5 layers of features into the trained FPN to obtain the first fused feature; wherein the 4 layers of features are (h / 4, w / 4, 64), (h / 8, w / 8, 128), (h / 16, w / 16, 256) and (h / 32, w / 32, 512), respectively, and the first fused feature is (h / 4, w / 4, 128); finally, the vehicle passes the first fused feature through the deep convolution layer to obtain a depth estimation map corresponding to the image at the current position to be processed; wherein the depth estimation map can be represented by (h / 4, w / 4).

[0068] It can be understood that the height of the current position image to be processed corresponding to each of the four layers of features is different, the width of the current position image to be processed corresponding to each of the four layers of features is different, and the number of color channels of the current position image to be processed corresponding to each of the four layers of features is different.

[0069] It should be noted that since the trained image recognition model has a high accuracy, the vehicle uses the trained image recognition model to perform depth estimation on the current image to be processed, and the accuracy of the depth estimation map corresponding to the current image to be processed is also high.

[0070] 103. Determine, according to the depth estimation map, a depth value corresponding to a two-dimensional reachable space included in the current position image to be processed, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0071] It should be noted that the depth value corresponding to the two-dimensional reachable space included in the current position to be processed image is the distance between the monocular camera of the vehicle and the current position to be processed.

[0072] Optionally, the vehicle determines the depth value corresponding to the two-dimensional reachable space included in the current position image to be processed based on the depth estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space, which may include: the vehicle converts the two-dimensional reachable space into a three-dimensional reachable space according to a first formula.

[0073] The first formula is x1=[(uc x )z1] / f x , y1=[(vc y )z1] / f y ;f x Indicates the focal length of the monocular camera on the x-axis in the world coordinate system, f y represents the focal length of the monocular camera on the y-axis in the world coordinate system, (c x , c y ) represents the aperture center of the monocular camera, and z1 represents the depth value corresponding to the two-dimensional reachable space included in the current position image to be processed.

[0074] In this way, the vehicle can obtain the size of the x1 value and the size of the y1 value according to the first formula. That is, the vehicle can convert the pixel coordinates (u, v) in the image of the current position to be processed into the corresponding world coordinates (x1, y1, z1) in the three-dimensional reachable space, thereby converting the two-dimensional reachable space corresponding to the current position to be processed into a three-dimensional reachable space.

[0075] Among them, x1 represents the coordinate of the pixel point in the current position image to be processed in the world coordinate system on the x-axis, y1 represents the coordinate of the pixel point in the current position image to be processed in the world coordinate system on the y-axis, and z1 can also represent the coordinate of the pixel point in the current position image to be processed in the world coordinate system on the z-axis.

[0076] It can be understood that the above method is a process in which the vehicle predicts and outputs the depth estimation map corresponding to the image of the current position to be processed through a neural network model (i.e., a trained image recognition model), and then determines the three-dimensional reachable space corresponding to the current position to be processed based on the depth estimation map. The vehicle uses the depth estimation image as redundant supervisory information (i.e., as a training basis for the training of the image recognition model) to strengthen the learning of the neural network model, so that the depth estimation map obtained by the vehicle is more accurate, thereby obtaining a three-dimensional reachable space with higher accuracy. In this way, the vehicle does not need to rely on a laser radar, but only needs to rely on a monocular camera. After obtaining an image of the current position to be processed through the monocular camera, it can directly use the neural network model to obtain a three-dimensional reachable space with higher accuracy in the vehicle.

[0077] In an embodiment of the present invention, an image of a current position to be processed to be identified is obtained; depth estimation is performed on the image of the current position to be processed by a trained image recognition model to obtain a depth estimation map corresponding to the image of the current position to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the reachable space of the vehicle in the sample image; the depth value corresponding to the two-dimensional reachable space included in the image of the current position to be processed is determined according to the depth estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space. Since the accuracy of the depth estimation map in the trained image recognition model is relatively high, when the image of the current position to be processed to be identified is input into the trained image recognition model, a depth estimation map with relatively high accuracy will be output, so that the three-dimensional reachable space with relatively high accuracy corresponding to the vehicle can be obtained according to the depth estimation map with relatively high accuracy.

[0078] like Figure 2 FIG. 2 is a schematic diagram of another embodiment of a method for determining a three-dimensional reachable space in an embodiment of the present invention, which may include:

[0079] 201. Perform depth estimation on the sample image by using the image recognition model to be trained to obtain a predicted depth map corresponding to the sample image.

[0080] Optionally, the image recognition model to be trained includes a ResNet to be trained and an FPN to be trained; the vehicle performs depth estimation on the sample image through the image recognition model to be trained to obtain a depth estimation map corresponding to the sample image, which may include: the vehicle inputs the sample image to the ResNet to be trained to obtain M layers of features, where M is an integer greater than or equal to 1; the vehicle selects m layers of features from the M layers of features and inputs them to the FPN to be trained to obtain a second fused feature, where m is an integer greater than or equal to 1; the vehicle passes the second fused feature through a deep convolution layer to obtain a depth estimation map corresponding to the sample image. Exemplarily, assume that M is 5 and m is 4.

[0081] It is understandable that the vehicle selects m-layer features from the M-layer features to input to the FPN to be trained, that is, M≥m. After obtaining the M-layer features, the vehicle does not need to input all the M-layer features to the FPN to be trained. It only needs to select some layers of features (such as m-layer features) from the M-layer features and input them to the FPN to be trained to obtain the second fused feature. Optionally, the selection of m-layer features can be random by the vehicle or user-defined, which is not specifically limited here.

[0082] It should be noted that the ResNet to be trained refers to a network that can extract features of different layers in the sample image. The more layers the ResNet to be trained has, the richer the features extracted by the ResNet to be trained in the sample image; the deeper the network depth of the ResNet to be trained, the more abstract the features extracted by the ResNet to be trained in the sample image.

[0083] Optionally, the ResNet to be trained may be a ResNet34 to be trained. The ResNet34 to be trained has a better network convergence effect.

[0084] It should be noted that the FPN to be trained refers to the one that can perform traditional top-down feature convolution to fuse multiple layers of features.

[0085] Optionally, the vehicle inputs the sample image into the ResNet to be trained to obtain M layers of features, which may include: the vehicle inputs the height, width, and number of color channels of the sample image into the ResNet to be trained to obtain M layers of features.

[0086] Among them, the height of the sample image corresponding to each layer of features in the M layers of features is different, the width of the sample image corresponding to each layer of features is different, and the number of color channels of the sample image corresponding to each layer of features is different.

[0087] It should be noted that, since the accuracy of the image recognition model to be trained is relatively poor, the accuracy of the depth estimation map corresponding to the sample image obtained by the vehicle through the image recognition model to be trained for depth estimation of the sample image is also relatively poor.

[0088] 202. Calculate a total loss value according to the predicted depth map, the true depth reference map, and the reachable space mask.

[0089] Optionally, before step 202, the method may further include: the vehicle obtains a predicted height map corresponding to the sample image, and a height truth reference map corresponding to the sample image.

[0090] Optionally, the vehicle obtaining a predicted height map corresponding to the sample image may include: the vehicle geometrically transforming the predicted depth map to obtain the predicted height map corresponding to the sample image.

[0091] Optionally, the vehicle performs a geometric transformation on the predicted depth map to obtain a predicted height map corresponding to the sample image, which may include: the vehicle obtains the predicted height map corresponding to the sample image according to the second formula.

[0092] The second formula is x2=[(uc x )z2] / f x, y2=[(vc y )z2] / f y ;f x Indicates the focal length of the monocular camera on the x-axis in the world coordinate system, f y represents the focal length of the monocular camera on the y-axis in the world coordinate system, (c x , c y ) represents the aperture center of the monocular camera, and z2 represents the depth value corresponding to the two-dimensional reachable space included in the sample image.

[0093] In this way, the vehicle can obtain the size of the y2 value according to the second formula, where y2 represents the height value corresponding to the predicted height map corresponding to the sample image.

[0094] Among them, x2 represents the coordinate of the pixel point in the sample image on the x-axis in the world coordinate system, z2 represents the coordinate of the pixel point in the sample image on the z-axis in the world coordinate system, and y2 can also represent the coordinate of the pixel point in the sample image on the y-axis in the world coordinate system.

[0095] It can be understood that because the height value is independent of the pixel position of the sample image, the total height loss value obtained by the vehicle based on the height prediction value corresponding to the predicted height map is more conducive to updating the parameters in the image recognition model.

[0096] Optionally, the vehicle calculates the total loss value based on the predicted depth map, the true depth reference map and the reachable space mask, which may include: the vehicle calculates the total loss value based on the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask.

[0097] Optionally, the vehicle calculates the total loss value based on the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask, which may include: the vehicle calculates the depth loss based on the predicted depth map, the true depth reference map and the reachable space mask; the vehicle calculates the height loss based on the predicted height map, the true height reference map and the reachable space mask; the vehicle calculates the total loss value based on the depth loss and the height loss.

[0098] Optionally, the vehicle calculates the depth loss based on the predicted depth map, the true depth reference map and the reachable space mask, which may include: the vehicle calculates the depth loss according to a third formula.

[0099] The third formula is depth_loss_total=sum(depth_loss);

[0100] depth_loss=L1_loss(depth_map_gt, depth_map_pred)*Drivable Space Mask;

[0101] depth_map_gt represents the true depth value corresponding to the true depth reference map, depth_map_pred represents the predicted depth value corresponding to the predicted depth map, and Drivable Space Mask represents the drivable space mask.

[0102] It can be understood that the third formula is a depth loss function, which is used to estimate the inconsistency between the depth value corresponding to the predicted reachable space in the position image to be processed and the depth value corresponding to the actual output three-dimensional reachable space. The depth loss function is a non-negative real-valued function.

[0103] Optionally, the vehicle calculates the height loss based on the predicted height map, the true height reference map and the reachable space mask, which may include: the vehicle calculates the height loss according to a fourth formula.

[0104] Wherein, the fourth formula is height_loss_total=sum(height_loss);

[0105] height_loss=L1_loss(height_map_gt, height_map_pred)*Drivable SpaceMask; height_map_gt represents the true height value corresponding to the true height reference map, and height_map_pred represents the predicted height value corresponding to the predicted height map.

[0106] It can be understood that the fourth formula is a height loss function, which is used to estimate the inconsistency between the height value corresponding to the predicted reachable space in the position image to be processed and the height value corresponding to the actual output three-dimensional reachable space. The height loss function is a non-negative real-valued function.

[0107] It should be noted that the vehicle may use the depth loss as the total loss value, or may use both the depth loss and the height loss as the total loss value, which is not specifically limited here.

[0108] 203. Update parameters in the image recognition model according to the total loss value.

[0109] It should be noted that, since the accuracy of the image recognition model to be trained is relatively poor, the vehicle updates the parameters in the image recognition model to be trained according to the total loss value, and can obtain an image recognition model with higher recognition accuracy. That is, the vehicle updates the parameters of the image recognition model to be trained, and can obtain a trained image recognition model. The vehicle uses the trained image recognition model to perform depth estimation on the image acquired by the monocular camera, and can obtain a depth estimation map corresponding to the image with higher accuracy.

[0110] 204. Obtain the current position image to be processed to be identified.

[0111] 205. Perform depth estimation on the image at the current position to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the image at the current position to be processed.

[0112] Among them, the image recognition model is trained using a sample image, a depth truth reference image corresponding to the sample image, and a reachable space mask, and the reachable space mask is used to indicate the image position of the vehicle's reachable space in the sample image.

[0113] 206. Determine, according to the depth estimation map, a depth value corresponding to the two-dimensional reachable space included in the current position image to be processed, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0114] It should be noted that steps 204-206 are Figure 1 The steps 101-103 shown are similar and will not be described in detail here.

[0115] In an embodiment of the present invention, the vehicle does not need to rely on a laser radar, but only needs to rely on a monocular camera. After obtaining an image of the current position to be processed through the monocular camera, the image of the current position to be processed can be directly input into the trained image recognition model to output a depth estimation map with higher accuracy. In this way, based on the depth estimation map with higher accuracy, a three-dimensional reachable space with higher accuracy corresponding to the vehicle can be obtained.

[0116] like Figure 3 FIG. 2 is a schematic diagram of another embodiment of a method for determining a three-dimensional reachable space in an embodiment of the present invention, which may include:

[0117] 301. Obtaining a current position image to be processed to be identified;

[0118] It should be noted that step 301 and Figure 1 The step 101 shown is similar and will not be described in detail here.

[0119] 302. Perform height estimation on the current image at the to-be-processed position using a trained image recognition model to obtain a height estimation map corresponding to the current image at the to-be-processed position.

[0120] Among them, the image recognition model is trained using a sample image, a height truth reference image corresponding to the sample image, and a reachable space mask, and the reachable space mask is used to indicate the image position of the vehicle's reachable space in the sample image.

[0121] It should be noted that the true height values ​​in the true height reference images corresponding to the sample images can be obtained based on the light beams collected by the laser radar installed on the vehicle. The true height value is the relatively accurate height annotation data measured by the laser radar of the vehicle.

[0122] Optionally, the trained image recognition model includes a trained ResNet and a trained FPN; the vehicle performs height estimation on the current image to be processed through the trained image recognition model to obtain a height estimation map corresponding to the current image to be processed, which may include: the vehicle inputs the current image to be processed into the trained ResNet to obtain A-layer features, where A is an integer greater than or equal to 1; the vehicle selects a-layer features from the A-layer features and inputs them into the trained FPN to obtain a third fused feature, where a is an integer greater than or equal to 1; the vehicle passes the third fused feature through a height convolution layer (Height Head) to obtain a height estimation map corresponding to the current image to be processed.

[0123] It is understandable that the vehicle selects the a-layer feature from the A-layer feature to input into the trained FPN, that is, A≥a. After obtaining the A-layer feature, the vehicle does not need to input all the A-layer features into the trained FPN. It only needs to select some layer features (such as the a-layer feature) from the A-layer feature and input them into the trained FPN to obtain the third fused feature. Optionally, the selection of the a-layer feature can be random by the vehicle or user-defined, which is not specifically limited here.

[0124] Optionally, the highly convolutional layer can be a 2D convolution.

[0125] The Height Head can be conv(output_chn, input_chn, width_kernel_size, height_kernel_size)+BatchNorm+Relu+conv(output_chn, input_chn, width_kernel_size, height_kernel_size)+BatchNorm+Relu.

[0126] Exemplarily, the Depth Head can be conv(64, 128, 3, 3)+BatchNorm+Relu+conv(1, 64, 3, 3)+BatchNorm+Relu.

[0127] 303. Determine a height value corresponding to a two-dimensional reachable space included in the current position image to be processed according to the height estimation map, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0128] It should be noted that the depth value corresponding to the two-dimensional reachable space included in the current position to be processed image is the distance between the monocular camera of the vehicle and the current position to be processed.

[0129] Optionally, the vehicle determines the height value corresponding to the two-dimensional reachable space included in the current position image to be processed based on the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space, which may include: the vehicle converts the two-dimensional reachable space into a three-dimensional reachable space based on the first formula.

[0130] The first formula is x1=[(uc x )z1] / f x , y1=[(vc y )z1] / f y ;f x Indicates the focal length of the monocular camera on the x-axis in the world coordinate system, f y represents the focal length of the monocular camera on the y-axis in the world coordinate system, (c x , c y ) represents the aperture center of the monocular camera, and y1 represents the height value corresponding to the two-dimensional reachable space included in the current position image to be processed.

[0131] It is understandable that y1 may also be the y coordinate of the pixel point in the current position image to be processed in the world coordinate system.

[0132] In this way, the vehicle can obtain the size of the x1 value and the size of the z1 value according to the first formula. That is, the vehicle can convert the pixel coordinates (u, v) in the image of the current position to be processed into the corresponding world coordinates (x1, y1, z1) in the three-dimensional reachable space, thereby converting the two-dimensional reachable space corresponding to the current position to be processed into a three-dimensional reachable space.

[0133] Among them, x1 represents the coordinate of the pixel point in the current position image to be processed in the world coordinate system on the x-axis, and z1 represents the coordinate of the pixel point in the current position image to be processed in the world coordinate system on the z-axis.

[0134] It can be understood that the above method is a process in which the vehicle predicts and outputs the height estimation map corresponding to the image of the current position to be processed through a trained image recognition model, and then determines the three-dimensional reachable space corresponding to the current position to be processed based on the height estimation map. The vehicle uses the height estimation image as the training basis for the image recognition model training, strengthens the learning of the neural network model, and makes the height estimation map obtained by the vehicle more accurate, thereby obtaining a three-dimensional reachable space with higher accuracy. In this way, the vehicle does not need to rely on a laser radar, but only needs to rely on a monocular camera. After obtaining an image of the current position to be processed through the monocular camera, the three-dimensional reachable space with higher accuracy can be obtained directly in the vehicle through the neural network model.

[0135] In an embodiment of the present invention, an image of a current position to be processed to be identified is obtained; the image of the current position to be processed is height estimated by a trained image recognition model to obtain a height estimation map corresponding to the image of the current position to be processed; the image recognition model is trained using a sample image, a height truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the reachable space of the vehicle in the sample image; the height value corresponding to the two-dimensional reachable space included in the image of the current position to be processed is determined according to the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space. Since the accuracy of the height estimation map is high in the trained image recognition model, when the image of the current position to be processed to be identified is input into the trained image recognition model, a height estimation map with high accuracy will be output, so that the three-dimensional reachable space with high accuracy corresponding to the vehicle can be obtained according to the height estimation map with high accuracy.

[0136] It should be noted that the vehicle can first execute Figure 1 Steps 101-103 shown in the figure can also be performed first Figure 3 Steps 301-303 shown can also be performed simultaneously Figure 1 Steps 101-103 shown are similar to Figure 3 The steps 301-303 shown are not specifically limited here.

[0137] like Figure 4 FIG. 2 is a schematic diagram of another embodiment of a method for determining a three-dimensional reachable space in an embodiment of the present invention, which may include:

[0138] 401. Perform height estimation on the sample image using the image recognition model to be trained to obtain a predicted height map corresponding to the sample image.

[0139] 402. Calculate a total loss value according to the predicted height map, the true height reference map, and the reachable space mask.

[0140] Optionally, before step 402, the method may further include: the vehicle obtains a predicted depth map corresponding to the sample image, and a true depth reference map corresponding to the sample image.

[0141] Optionally, the vehicle obtaining the predicted depth map corresponding to the sample image may include: the vehicle geometrically transforming the predicted height map to obtain the predicted depth map corresponding to the sample image.

[0142] Optionally, before step 402, the method may further include: the vehicle obtains a reachable space mask.

[0143] The accessible space mask is used to indicate the image position of the vehicle's accessible space in the height truth reference image.

[0144] Optionally, the vehicle calculates the total loss value based on the predicted height map, the true height reference map and the reachable space mask, which may include: the vehicle calculates the total loss value based on the predicted height map, the true height reference map, the predicted depth map, the true depth reference map and the reachable space mask.

[0145] Optionally, the vehicle calculates a total loss value based on the predicted height map, the true height reference map, the predicted depth map, the true depth reference map and the reachable space mask, which may include: the vehicle calculates the height loss based on the predicted height map, the true height reference map and the reachable space mask; the vehicle calculates the depth loss based on the predicted depth map, the true depth reference map and the reachable space mask; the vehicle calculates the total loss value based on the height loss and the depth loss.

[0146] It should be noted that the vehicle may use the height loss as the total loss value, or may use both the height loss and the depth loss as the total loss value, which is not specifically limited here.

[0147] 403. Update parameters in the image recognition model according to the total loss value.

[0148] 404. Obtain the current position image to be processed to be identified.

[0149] 405. Perform height estimation on the current image at the to-be-processed position using a trained image recognition model to obtain a height estimation map corresponding to the current image at the to-be-processed position.

[0150] Among them, the image recognition model is trained using a sample image, a height truth reference image corresponding to the sample image, and a reachable space mask, and the reachable space mask is used to indicate the image position of the vehicle's reachable space in the sample image.

[0151] 406. Determine a height value corresponding to a two-dimensional reachable space included in the current position image to be processed according to the height estimation map, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0152] It should be noted that steps 401-406 are Figure 2 The steps 201-206 shown are similar and will not be described in detail here.

[0153] In an embodiment of the present invention, the vehicle does not need to rely on a laser radar, but only needs to rely on a monocular camera. After obtaining an image of the current position to be processed through the monocular camera, the image of the current position to be processed can be directly input into the trained image recognition model to output a height estimation map with higher accuracy. In this way, based on the height estimation map with higher accuracy, a three-dimensional reachable space with higher accuracy corresponding to the vehicle can be obtained.

[0154] like Figure 5 As shown, it is a schematic diagram of an embodiment of a vehicle in an embodiment of the present invention, which may include: an acquisition module 501 and a processing module 502;

[0155] An acquisition module 501 is used to acquire an image of a current position to be processed to be identified;

[0156] Processing module 502 is used to perform depth estimation on the current image to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the current image to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the vehicle's reachable space in the sample image; the depth value corresponding to the two-dimensional reachable space included in the current image to be processed is determined according to the depth estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0157] Optionally, in some embodiments of the present invention,

[0158] The processing module 502 is also used to perform depth estimation on the sample image through the image recognition model to be trained to obtain a predicted depth map corresponding to the sample image; calculate the total loss value based on the predicted depth map, the true depth reference map and the reachable space mask; and update the parameters in the image recognition model based on the total loss value.

[0159] Optionally, in some embodiments of the present invention,

[0160] The acquisition module 501 is further used to acquire a predicted height map corresponding to the sample image and a height true value reference map corresponding to the sample image;

[0161] The processing module 502 is further used to calculate the total loss value according to the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask.

[0162] Optionally, in some embodiments of the present invention,

[0163] The acquisition module 501 is specifically used to perform geometric transformation on the predicted depth map to obtain a predicted height map corresponding to the sample image.

[0164] Optionally, in some embodiments of the present invention,

[0165] Processing module 502 is specifically used to calculate the depth loss according to the predicted depth map, the depth truth reference map and the reachable space mask; calculate the height loss according to the predicted height map, the height truth reference map and the reachable space mask; calculate the total loss value according to the depth loss and the height loss.

[0166] Optionally, in some embodiments of the present invention,

[0167] The acquisition module 501 is also used to acquire the current position image to be processed to be identified;

[0168] The processing module 502 is also used to perform height estimation on the current image of the position to be processed by using a trained image recognition model to obtain a height estimation map corresponding to the current image of the position to be processed; the image recognition model is trained using a sample image, a height truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the vehicle's reachable space in the sample image; the height value corresponding to the two-dimensional reachable space included in the current image of the position to be processed is determined according to the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0169] Optionally, in some embodiments of the present invention,

[0170] The processing module 502 is further used to estimate the height of the sample image by using the image recognition model to be trained to obtain a predicted height map corresponding to the sample image;

[0171] The acquisition module 501 is further used to acquire a height truth reference map and a reachable space mask corresponding to the sample image; the reachable space mask is used to indicate the image position of the vehicle's reachable space in the height truth reference map;

[0172] The processing module 502 is further used to calculate the total loss value based on the predicted height map, the true height reference map and the reachable space mask; and update the parameters in the image recognition model according to the total loss value.

[0173] like Figure 6 , which is a schematic diagram of another embodiment of a vehicle in an embodiment of the present invention, may include: a memory 601 and a processor 602; the memory 601 and the processor 602 are coupled, and the processor 602 calls the executable program code stored in the memory 601;

[0174] The processor 602 has the following functions:

[0175] Obtain the current position image to be processed to be identified;

[0176] Performing depth estimation on the current image at the position to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the current image at the position to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate an image position of a vehicle's reachable space in the sample image;

[0177] A depth value corresponding to the two-dimensional reachable space included in the current position image to be processed is determined according to the depth estimation map, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0178] Optionally, the processor 602 further has the following functions:

[0179] The depth of the sample image is estimated by the image recognition model to be trained to obtain a predicted depth map corresponding to the sample image; a total loss value is calculated based on the predicted depth map, the true depth reference map and the reachable space mask; and the parameters in the image recognition model are updated based on the total loss value.

[0180] Optionally, the processor 602 further has the following functions:

[0181] Obtain a predicted height map corresponding to the sample image and a true height reference map corresponding to the sample image; and, calculate a total loss value based on the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask.

[0182] Optionally, the processor 602 further has the following functions:

[0183] The predicted depth map is geometrically transformed to obtain a predicted height map corresponding to the sample image.

[0184] Optionally, the processor 602 further has the following functions:

[0185] The depth loss is calculated according to the predicted depth map, the true depth reference map and the reachable space mask; the height loss is calculated according to the predicted height map, the true height reference map and the reachable space mask; and the total loss value is calculated according to the depth loss and the height loss.

[0186] Optionally, the processor 602 further has the following functions:

[0187] Acquire a current image of a processing position to be identified; perform height estimation on the current image of a processing position using a trained image recognition model to obtain a height estimation map corresponding to the current image of a processing position; the image recognition model is trained using a sample image, a height truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the vehicle's reachable space in the sample image; determine a height value corresponding to a two-dimensional reachable space included in the current image of a processing position according to the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

[0188] Optionally, the processor 602 further has the following functions:

[0189] The height of the sample image is estimated by the image recognition model to be trained to obtain a predicted height map corresponding to the sample image; a true height reference map and a reachable space mask corresponding to the sample image are obtained; the reachable space mask is used to indicate the image position of the vehicle's reachable space in the true height reference map; a total loss value is calculated based on the predicted height map, the true height reference map and the reachable space mask; and the parameters in the image recognition model are updated based on the total loss value.

[0190] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0191] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center to another website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0193] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0196] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0197] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining a three-dimensional reachable space, characterized in that: The method comprises: Obtain the current position image to be processed to be identified; The depth of the image at the current position to be processed is estimated by a trained image recognition model to obtain a depth estimation map corresponding to the image at the current position to be processed; the image recognition model is trained using a sample image, a depth truth reference map corresponding to the sample image, and a reachable space mask, the reachable space mask being used to indicate the image position of the reachable space of the vehicle in the sample image, and the parameters in the image recognition model are updated according to a total loss value, which is calculated based on a height loss obtained from a height prediction value corresponding to a predicted height map and a depth loss obtained from a depth prediction value corresponding to a predicted depth map; A depth value corresponding to a two-dimensional reachable space included in the current position image to be processed is determined according to the depth estimation map, so as to convert the two-dimensional reachable space into a three-dimensional reachable space.

2. The method according to claim 1, characterized in that Before acquiring the current position image to be processed to be identified, the method further includes: Performing depth estimation on the sample image by using the image recognition model to be trained to obtain a predicted depth map corresponding to the sample image; Acquire a predicted height map corresponding to the sample image and a true height reference map corresponding to the sample image; Calculate a total loss value according to the predicted depth map, the true depth reference map, the predicted height map, the true height reference map, and the reachable space mask; Update parameters in the image recognition model according to the total loss value.

3. The method according to claim 2, characterized in that The obtaining of a predicted height map corresponding to the sample image comprises: The predicted depth map is geometrically transformed to obtain a predicted height map corresponding to the sample image.

4. The method according to claim 2, characterized in that: The calculating a total loss value according to the predicted depth map, the true depth reference map, the predicted height map, the true height reference map and the reachable space mask includes: Calculating a depth loss based on the predicted depth map, the ground truth reference map, and the reachable space mask; Calculating the height loss according to the predicted height map, the true height reference map and the reachable space mask; A total loss value is calculated according to the depth loss and the height loss.

5. A method for determining a three-dimensional reachable space, characterized in that: The method further comprises: The image of the current position to be processed to be identified is obtained by shooting with a monocular camera arranged on the vehicle; The height of the current position image to be processed is estimated by using a trained image recognition model to obtain a height estimation map corresponding to the current position image to be processed; the image recognition model is trained using a sample image, a height truth reference map corresponding to each of the sample images, and a reachable space mask, wherein the reachable space mask is used to indicate an image position of a vehicle's reachable space in the sample image; The height value corresponding to the two-dimensional reachable space included in the current position image to be processed is determined according to the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space. The parameters in the image recognition model are updated according to the total loss value, and the total loss value is calculated based on the height loss obtained by the height prediction value corresponding to the predicted height map and the depth loss obtained by the depth prediction value corresponding to the predicted depth map.

6. The method according to claim 5, characterized in that Before acquiring the current position image to be processed to be identified, the method further includes: Performing height estimation on the sample image by using the image recognition model to be trained to obtain a predicted height map corresponding to the sample image; Acquire a height truth reference map and a reachable space mask corresponding to the sample image; the reachable space mask is used to indicate the image position of the reachable space of the vehicle in the height truth reference map; Calculating a total loss value according to the predicted height map, the true height reference map and the reachable space mask; Update parameters in the image recognition model according to the total loss value.

7. A vehicle, characterized in that: include: An acquisition module is used to acquire the current position image to be processed to be identified; A processing module, used to perform depth estimation on the image at the current position to be processed by using a trained image recognition model to obtain a depth estimation map corresponding to the image at the current position to be processed; The image recognition model is trained using sample images, depth truth reference maps corresponding to the sample images respectively, and a reachable space mask, wherein the reachable space mask is used to indicate the image position of the vehicle's reachable space in the sample image, and the parameters in the image recognition model are updated according to a total loss value, wherein the total loss value is calculated based on a height loss obtained by a height prediction value corresponding to a predicted height map and a depth loss obtained by a depth prediction value corresponding to a predicted depth map; a depth value corresponding to the two-dimensional reachable space included in the current position image to be processed is determined based on the depth estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space.

8. A vehicle, characterized in that: include: The acquisition module is also used to acquire the image of the current position to be processed to be identified by photographing through a monocular camera arranged on the vehicle; The processing module is further used to perform height estimation on the current position image to be processed by using a trained image recognition model to obtain a height estimation map corresponding to the current position image to be processed; the image recognition model is trained using a sample image, a height truth reference map corresponding to each of the sample images, and a reachable space mask, wherein the reachable space mask is used to indicate an image position of a vehicle's reachable space in the sample image; The height value corresponding to the two-dimensional reachable space included in the current position image to be processed is determined according to the height estimation map to convert the two-dimensional reachable space into a three-dimensional reachable space. The parameters in the image recognition model are updated according to the total loss value, and the total loss value is calculated based on the height loss obtained by the height prediction value corresponding to the predicted height map and the depth loss obtained by the depth prediction value corresponding to the predicted depth map.

9. A vehicle, characterized in that: include: A memory storing executable program code; and a processor coupled to the memory; The processor calls the executable program code stored in the memory, and when the executable program code is executed by the processor, the processor implements the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having executable program code stored thereon, characterized in that: When the executable program code is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images