Horizon detection method based on Hourglass network from the perspective of a monocular roadside camera

Abstract

This application relates to a horizon detection method based on Hourglass network from the perspective of a monocular roadside camera. This method can acquire real-time horizon information more quickly and effectively avoid the interference of noise on the horizon detection algorithm. In addition, using Hourglass can effectively and quickly extract and fuse multi-scale features of the input image, thereby realizing the extraction of horizon information in complex scenes such as traffic.

Description

Technical Field

[0001] This application relates to the field of horizon detection technology, specifically to a horizon detection method based on an Hourglass network from the perspective of a monocular roadside camera. Background Technology

[0002] In Intelligent Transportation Systems (ITS), the horizon is a crucial geometric reference for autonomous and assisted driving. The horizon, in perspective projection, is the horizontal line formed by the intersection of a horizontal line within the field of view and the ground. The horizon provides vital information about a scene, such as its horizontal orientation, camera height, and distance. In computer vision, this information is obtained by detecting the horizon in an image and used in applications such as image correction, object detection, and scene reconstruction. Furthermore, the horizon can be used in tasks like image classification and image retrieval to help computers better understand and process images.

[0003] Traditional horizon detection algorithms primarily rely on edge detection and geometric feature analysis. These methods are sensitive to image noise and lighting variations, require image preprocessing, and are computationally intensive. With the development of deep learning, deep learning-based object detection has been increasingly applied to horizon detection, improving its accuracy. However, current methods still cannot achieve real-time speed, hindering timely information provision for vehicle-assisted driving. In recent years, keypoint detection networks have demonstrated strong performance in human keypoint detection. As an important branch of object detection, it extracts image features through convolution and pooling operations, then uses these features to extract keypoint locations or confidence levels. Keypoint detection networks address the over-reliance on bounding boxes in traditional object detection. Furthermore, by detecting keypoints, target segmentation and recognition can be achieved, effectively resolving occlusion issues. However, all of the above methods suffer from low detection accuracy and slow detection speed. Currently, horizon detection mainly suffers from the following problems: the horizon detection speed is not high, and it cannot provide real-time basis for the autonomous driving of vehicles; the horizon detection is relatively sensitive to noise and other factors, and its accuracy is not high. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a horizon detection method based on Hourglass network from the perspective of a monocular roadside camera.

[0005] Firstly, a method for constructing a horizon detection model based on the Hourglass network is provided, including:

[0006] Obtain the original vehicle image dataset; the original vehicle image dataset includes multiple original vehicle images;

[0007] Multiple original vehicle images in the original vehicle image dataset are converted from the original physical space to diamond space to obtain the spatially transformed vehicle image dataset. The original vehicle images include two label information under the original physical space, and the images in the spatially transformed vehicle image dataset include two label information under diamond space.

[0008] A training dataset is constructed based on the spatially transformed vehicle image dataset; the training dataset includes a training set and a test set.

[0009] The horizon detection model based on the Hourglass network is trained using the training dataset to obtain the trained horizon detection model based on the Hourglass network. The horizon detection model based on the Hourglass network includes the Hourglass network and a multi-scale transformation module. The Hourglass network is used to output two heatmaps based on the samples in the training dataset. The multi-scale transformation module is used to perform multi-scale transformations on the two heatmaps output by the Hourglass network to obtain a first heatmap and a second heatmap at four different scales.

[0010] In one embodiment, training a horizon detection model based on an Hourglass network using a training dataset includes:

[0011] The test set was input into the horizon detection model based on the Hourglass network, resulting in four first heatmaps at different scales and four second heatmaps at different scales. Each of the four first heatmaps and four second heatmaps at different scales contains vanishing point location information in diamond space.

[0012] Among the vanishing point location information in the diamond space corresponding to the four first heatmaps, the vanishing point location information that is closest to the first label information in the diamond space corresponding to the sample in the test set is selected as the first selected vanishing point location information; the first label information is the first vanishing point location information.

[0013] Among the vanishing point location information in the diamond space corresponding to the four second heatmaps, the one that is closest to the second label information in the diamond space corresponding to the sample in the test set is selected as the second selected vanishing point location information; the second label information is the second vanishing point location information.

[0014] Decode the vanishing point location information of the first and second selected options to obtain the two vanishing point location information in the original physical space;

[0015] The model output accuracy is determined based on the location information of the two vanishing points in the original physical space and the first and second label information of the samples in the original physical space.

[0016] The parameters of the horizon detection model based on the model output accuracy are adjusted.

[0017] In one embodiment, the loss function used during training is the Focal Loss loss function.

[0018] Secondly, a horizon detection method based on an Hourglass network is provided from the perspective of a monocular roadside camera, including:

[0019] Obtain the image to be detected; the image contains n vehicles.

[0020] The image to be detected is input into the horizon detection model based on the Hourglass network to obtain n diamond spatial heatmap groups. Each diamond spatial heatmap group includes four first heatmaps at different scales and four second heatmaps at different scales. The horizon detection model based on the Hourglass network is obtained according to the above-described method for constructing the horizon detection model based on the Hourglass network.

[0021] Decode each of the n diamond space heatmap groups to obtain n horizons;

[0022] Determine the median of the slopes of all n horizons and the median of the intercepts of all horizons;

[0023] The final horizon is determined based on the median of all horizon slopes and the median of all horizon intercepts.

[0024] In one embodiment, decoding each of the n diamond space heatmap groups yields n horizons, including:

[0025] For each diamond space heatmap group, the vanishing point location information in the first and second heatmaps of the same scale is decoded to obtain 4 sets of vanishing point location information in the original physical space. Each set of vanishing point location information in the original physical space includes vanishing point location information in 2 original physical spaces. A horizon is obtained based on the vanishing point location information in 2 original physical spaces. Finally, 4 horizons are obtained for each diamond space heatmap group.

[0026] Find the median of the slopes of all four horizons and the median of the intercepts of all four horizons.

[0027] The horizon corresponding to the diamond space heatmap group is determined based on the median of all horizon slopes and the median of all horizon intercepts.

[0028] Thirdly, a horizon detection model construction device based on the Hourglass network is provided, comprising:

[0029] The dataset acquisition module is used to acquire the original vehicle image dataset; the original vehicle image dataset includes multiple original vehicle images.

[0030] The spatial transformation module is used to convert multiple original vehicle images in the original vehicle image dataset from the original physical space to diamond space, resulting in a spatially transformed vehicle image dataset. The original vehicle images include two label information in the original physical space, and the images in the spatially transformed vehicle image dataset include two label information in diamond space.

[0031] The training data construction module is used to build a training dataset based on the spatially transformed vehicle image dataset; the training dataset includes a training set and a test set.

[0032] The training module is used to train the horizon detection model based on the Hourglass network using the training dataset, resulting in the trained horizon detection model based on the Hourglass network. The horizon detection model based on the Hourglass network includes the Hourglass network and a multi-scale transformation module. The Hourglass network outputs two heatmaps based on samples in the training dataset, and the multi-scale transformation module performs multi-scale transformations on the two heatmaps output by the Hourglass network to obtain a first heatmap and a second heatmap at four different scales.

[0033] In one embodiment, the training module is also used for:

[0034] The test set was input into the horizon detection model based on the Hourglass network, resulting in four first heatmaps at different scales and four second heatmaps at different scales. Each of the four first heatmaps and four second heatmaps at different scales contains vanishing point location information in diamond space.

[0035] Among the vanishing point location information in the diamond space corresponding to the four first heatmaps, the vanishing point location information that is closest to the first label information in the diamond space corresponding to the sample in the test set is selected as the first selected vanishing point location information; the first label information is the first vanishing point location information.

[0036] Among the vanishing point location information in the diamond space corresponding to the four second heatmaps, the one that is closest to the second label information in the diamond space corresponding to the sample in the test set is selected as the second selected vanishing point location information; the second label information is the second vanishing point location information.

[0037] Decode the vanishing point location information of the first and second selected options to obtain the two vanishing point location information in the original physical space;

[0038] The model output accuracy is determined based on the location information of the two vanishing points in the original physical space and the first and second label information of the samples in the original physical space.

[0039] The parameters of the horizon detection model based on the model output accuracy are adjusted.

[0040] In one embodiment, the loss function used during training is the Focal Loss loss function.

[0041] Fourthly, a horizon detection device based on an Hourglass network is provided from the perspective of a monocular roadside camera, comprising:

[0042] The image acquisition module is used to acquire the image to be detected; the image to be detected contains n vehicles.

[0043] The diamond spatial heatmap acquisition module is used to input the image to be detected into the horizon detection model based on the Hourglass network to obtain n diamond spatial heatmap groups. Each diamond spatial heatmap group includes four first heatmaps at different scales and four second heatmaps at different scales. The horizon detection model based on the Hourglass network is obtained according to the above-described method for constructing the horizon detection model based on the Hourglass network.

[0044] The decoding module is used to decode each of the n diamond space heatmap groups to obtain n horizons;

[0045] The slope and intercept determination module is used to determine the median of the slopes of all n horizons and the median of the intercepts of all horizons.

[0046] The horizon determination module is used to determine the final horizon based on the median of all horizon slopes and the median of all horizon intercepts.

[0047] In one embodiment, the decoding module is further configured to:

[0048] For each diamond space heatmap group, the vanishing point location information in the first and second heatmaps of the same scale is decoded to obtain 4 sets of vanishing point location information in the original physical space. Each set of vanishing point location information in the original physical space includes vanishing point location information in 2 original physical spaces. A horizon is obtained based on the vanishing point location information in 2 original physical spaces. Finally, 4 horizons are obtained for each diamond space heatmap group.

[0049] Find the median of the slopes of all four horizons and the median of the intercepts of all four horizons.

[0050] The horizon corresponding to the diamond space heatmap group is determined based on the median of all horizon slopes and the median of all horizon intercepts.

[0051] Compared with the prior art, this application has the following advantages: This application can acquire real-time horizon information more quickly, and can effectively avoid the interference of noise on the horizon detection algorithm; in addition, using Hourglass can effectively and quickly extract and fuse multi-scale features of the input image, thereby realizing the extraction of horizon information in complex scenes such as traffic. Attached Figure Description

[0052] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0053] Figure 1 A flowchart illustrating a method for constructing a horizon detection model based on an Hourglass network according to an embodiment of this application is shown.

[0054] Figure 2 This shows a portion of the data from the original vehicle image dataset;

[0055] Figure 3 A schematic diagram of a diamond space heat map is shown;

[0056] Figure 4 A flowchart illustrating a horizon detection method based on an Hourglass network from the perspective of a monocular roadside camera according to an embodiment of this application is shown.

[0057] Figure 5 A schematic diagram is shown illustrating the determination of the horizon based on the vanishing point location information;

[0058] Figure 6 A structural block diagram of a horizon detection model construction device based on an Hourglass network according to an embodiment of this application is shown.

[0059] Figure 7A structural block diagram of a horizon detection device based on an Hourglass network from the perspective of a monocular roadside camera according to an embodiment of this application is shown. Detailed Implementation

[0060] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0061] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0062] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0063] This application provides a method for constructing a horizon detection model based on the Hourglass network. Figure 1 A flowchart illustrating a horizon detection model construction method based on an Hourglass network according to an embodiment of this application is shown. See also... Figure 1 The methods include:

[0064] Step S11: Obtain the original vehicle image dataset; the original vehicle image dataset includes multiple original vehicle images.

[0065] Here, the original vehicle image dataset used is BoxCars116k, which contains 116,000 vehicle samples from different angles as seen by roadside cameras. Figure 2 This shows a portion of the data from the original vehicle image dataset.

[0066] Step S12: Convert multiple original vehicle images in the original vehicle image dataset from the original physical space to diamond space to obtain the spatially converted vehicle image dataset. The original vehicle images include two label information in the original physical space, namely the two vanishing point position information in the original physical space. The images in the spatially converted vehicle image dataset include two label information in diamond space, namely the two vanishing point position information in diamond space.

[0067] Here, before spatial transformation of the original vehicle images, multiple original vehicle images in the original vehicle image dataset are preprocessed to obtain a preprocessed vehicle image dataset, which includes multiple preprocessed vehicle images. The preprocessing includes random perspective transformation, cropping the long side with the fixed short side, and resizing to 128*128. The purpose is to adjust the original vehicle images to a suitable size before inputting them into the network.

[0068] Since the original vehicle image dataset consists of vehicle images detected by object detection algorithms, their scale cannot be kept uniform. Therefore, the detected vehicle images need to be preprocessed before being fed into a horizon detection model based on the Hourglass network. Image preprocessing includes random perspective transformation and random cropping. Here, the images in the original vehicle image dataset have label information, namely, two vanishing point positions. During random perspective transformation, the label information in the images also changes accordingly; that is, the vanishing point positions in the original physical space are converted into vanishing point positions in diamond space.

[0069] (x,y,w) o →(-w,-x,sgn(xy)x+y+sgn(y)w) d

[0070] Where (x,y,w) o Let (-w, -x, sgn(xy)x+y+sgn(y)w) be the coordinates of the vanishing point in the original physical space o. d sgn represents the vanishing point location information in diamond space d, and sgn is the sign function.

[0071] Step S13: Construct a training dataset based on the spatially transformed vehicle image dataset; the training dataset includes a training set and a test set. Here, the training dataset is divided into a training set and a test set in a 9:1 ratio to train the network.

[0072] Step S14: Train the horizon detection model based on the Hourglass network based on the training dataset to obtain the trained horizon detection model based on the Hourglass network. The horizon detection model based on the Hourglass network includes the Hourglass network and a multi-scale transformation module. The Hourglass network is used to output two heatmaps based on the samples in the training dataset. The multi-scale transformation module is used to perform multi-scale transformations on the two heatmaps output by the Hourglass network to obtain a first heatmap and a second heatmap at four different scales.

[0073] Here, the Hourglass network outputs two heatmaps. The multi-scale transformation module is set with four scales: 0.03, 0.1, 0.3, and 1. This ensures that the vanishing point information is not lost due to the different distances between the vanishing point and the origin after transformation to diamond space. This results in four first heatmaps and four second heatmaps at different scales. Each pair of first and second heatmaps at the same scale can be used to extract a set of vanishing point information at the same scale. Figure 3 A schematic diagram of a diamond space heat map is shown.

[0074] In one embodiment, training a horizon detection model based on an Hourglass network using a training dataset includes:

[0075] First, the test set is input into the horizon detection model based on the Hourglass network to obtain four first heatmaps at different scales and four second heatmaps at different scales. Each of the four first heatmaps at different scales and the four second heatmaps at different scales contains vanishing point location information in diamond space.

[0076] Then, among the vanishing point location information in the diamond space corresponding to the four first heatmaps, the vanishing point location information that is closest to the first label information in the diamond space corresponding to the sample in the test set is selected as the first selected vanishing point location information; the first label information is the first vanishing point location information; here, the samples in the test set include the first label information in the original physical space, and after preprocessing the samples, the first label information in the diamond space corresponding to the samples is obtained.

[0077] Then, among the vanishing point location information in the diamond space corresponding to the four second heatmaps, the vanishing point location information that is closest to the second label information in the diamond space corresponding to the sample in the test set is selected as the second selected vanishing point location information; the second label information is the second vanishing point location information.

[0078] Then, the vanishing point location information of the first and second selections is decoded to obtain the two vanishing point location information in the original physical space; here, the decoding formula is as follows:

[0079] (x,y,w) d →(y,sgn(x)x+sgn(y)yw,x) o

[0080] Where (x,y,w) d For the vanishing point location information in diamond space d, (y, sgn(x)x+sgn(y)yw,x,) o This refers to the vanishing point location information in the original physical space.

[0081] Then, based on the two vanishing point location information in the original physical space and the first label information and second label information in the original physical space corresponding to the samples in the test set, the model output accuracy is determined.

[0082] Then, the parameters of the horizon detection model based on the model output accuracy are adjusted.

[0083] In one embodiment, the Focal Loss loss function is used during training. Considering that during network training, positive samples occupy only two pixels in the heatmap (two vanishing points), while the rest of the heatmap consists of negative samples, resulting in a severe imbalance between positive and negative samples, the Focal Loss loss function is used instead of the traditional loss function. This adds weights to control the difficulty of the samples compared to the original cross-entropy loss function. The loss function FL( t The formula is:

[0084] FL(p t )=-α t (1-p t ) γ log(p t )

[0085] Where α t γ is the shared weight, p t This represents the probability of correctly classifying the information.

[0086] This application also provides a horizon detection method based on an Hourglass network from the perspective of a monocular roadside camera. Figure 4 A flowchart illustrating a horizon detection method based on an Hourglass network from the perspective of a monocular roadside camera according to an embodiment of this application is shown. See also... Figure 4 The methods include:

[0087] Step S41: Obtain the image to be detected; the image contains n vehicles. Here, the image to be detected is obtained from the perspective of a monocular roadside camera, and vehicles on the road in the scene are detected based on the YOLOv7 object detection model. Each detected vehicle is then numbered.

[0088] Step S42: Input the image to be detected into the horizon detection model based on the Hourglass network to obtain n diamond spatial heatmap groups. Each diamond spatial heatmap group includes four first heatmaps at different scales and four second heatmaps at different scales. The horizon detection model based on the Hourglass network is obtained according to the horizon detection model construction method based on the Hourglass network described above.

[0089] Step S43: Decode each of the n diamond space heatmap groups to obtain n horizons. Figure 5 A schematic diagram illustrating the determination of the horizon based on vanishing point location information is shown. Specifically, for each diamond space heatmap group, a horizon can be obtained using the following method:

[0090] First, for each diamond space heatmap group, the vanishing point location information in the first and second heatmaps of the same scale is decoded to obtain 4 sets of vanishing point location information in the original physical space. Each set of vanishing point location information in the original physical space includes 2 sets of vanishing point location information in the original physical space. A horizon is obtained based on the 2 sets of vanishing point location information in the original physical space. Finally, 4 horizons are obtained for each diamond space heatmap group.

[0091] Then, find the median of the slopes of all four horizons and the median of the intercepts of all four horizons.

[0092] Finally, the horizon corresponding to the diamond space heatmap group is determined based on the median of all horizon slopes and the median of all horizon intercepts.

[0093] Step S44: Determine the median of the slopes of all n horizons and the median of the intercepts of all horizons.

[0094] Step S45: Determine the final horizon based on the median of all horizon slopes and the median of all horizon intercepts.

[0095] Employing the same inventive concept as the horizon detection model construction method based on Hourglass networks, this embodiment also provides a corresponding horizon detection model construction device based on Hourglass networks. Figure 6 A structural block diagram of a horizon detection model construction device based on an Hourglass network according to an embodiment of this application is shown. See also Figure 6 The device includes:

[0096] The dataset acquisition module 61 is used to acquire the original vehicle image dataset; the original vehicle image dataset includes multiple original vehicle images.

[0097] Spatial Transformation 62 is used to convert multiple original vehicle images in the original vehicle image dataset from the original physical space to diamond space, resulting in a spatially transformed vehicle image dataset. The original vehicle images include two label information under the original physical space, and the images in the spatially transformed vehicle image dataset include two label information under the diamond space.

[0098] Training data construction module 63 is used to construct a training dataset based on the spatially transformed vehicle image dataset; the training dataset includes a training set and a test set.

[0099] Training module 64 is used to train the horizon detection model based on the Hourglass network based on the training dataset to obtain the trained horizon detection model based on the Hourglass network. The horizon detection model based on the Hourglass network includes the Hourglass network and a multi-scale transformation module. The Hourglass network is used to output two heatmaps based on the samples in the training dataset. The multi-scale transformation module is used to perform multi-scale transformations on the two heatmaps output by the Hourglass network to obtain a first heatmap and a second heatmap at four different scales.

[0100] The horizon detection model construction device based on Hourglass network in this embodiment has the same inventive concept as the horizon detection model construction method based on Hourglass network described above. Therefore, the specific implementation of this device can be found in the embodiment section of the horizon detection model construction method based on Hourglass network described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0101] Employing the same inventive concept as the horizon detection method based on Hourglass networks from the perspective of a monocular roadside camera, this embodiment also provides a corresponding horizon detection device based on Hourglass networks from the perspective of a monocular roadside camera. Figure 7 A structural block diagram of a horizon detection device based on an Hourglass network from the perspective of a monocular roadside camera according to an embodiment of this application is shown. The device includes:

[0102] Image acquisition module 71 is used to acquire the image to be detected; the image to be detected contains n vehicles.

[0103] The diamond spatial heatmap acquisition module 72 is used to input the image to be detected into the horizon detection model based on the Hourglass network to obtain n diamond spatial heatmap groups. Each diamond spatial heatmap group includes four first heatmaps at different scales and four second heatmaps at different scales. The horizon detection model based on the Hourglass network is obtained by the horizon detection model construction method based on the Hourglass network described above.

[0104] Decoding module 73 is used to decode each of the n diamond space heatmap groups to obtain n horizons;

[0105] Slope and intercept determination module 74 is used to determine the median of the slopes of all n horizons and the median of the intercepts of all horizons.

[0106] The horizon determination module 75 is used to determine the final horizon based on the median of all horizon slopes and the median of all horizon intercepts.

[0107] The horizon detection device based on Hourglass network under the monocular roadside camera view in this embodiment has the same inventive concept as the horizon detection method based on Hourglass network under the monocular roadside camera view described above. Therefore, the specific implementation of this device can be found in the embodiment section of the horizon detection method based on Hourglass network under the monocular roadside camera view described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0108] In summary, this application can acquire real-time horizon information more quickly, while effectively avoiding the interference of noise on the horizon detection algorithm. In addition, using hourglass can effectively and quickly extract and fuse multi-scale features from the input image, thereby realizing the extraction of horizon information in complex scenes such as traffic.

[0109] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing a horizon detection model based on an Hourglass network, characterized in that, The method comprises the following steps: obtaining an original vehicle picture dataset; the original vehicle picture dataset comprises a plurality of original vehicle pictures; converting the plurality of original vehicle pictures in the original vehicle picture dataset from an original physical space to a diamond space to obtain a spatially converted vehicle picture dataset; the original vehicle pictures comprise two label information in the original physical space, and the pictures in the spatially converted vehicle picture dataset comprise two label information in the diamond space; constructing a training dataset based on the spatially converted vehicle picture dataset; the training dataset comprises a training set and a test set; training a horizon detection model based on an Hourglass network based on the training dataset to obtain a trained horizon detection model based on the Hourglass network; the horizon detection model based on the Hourglass network comprises an Hourglass network and a multi-scale transformation module, the Hourglass network is used to output two heat maps based on samples in the training dataset, and the multi-scale transformation module is used to perform multi-scale transformation on the two heat maps output by the Hourglass network respectively to obtain four first heat maps of different scales and four second heat maps of different scales respectively; wherein, training the horizon detection model based on the Hourglass network based on the training dataset comprises: inputting the test set into the horizon detection model based on the Hourglass network to obtain the four first heat maps of different scales and the four second heat maps of different scales; each of the four first heat maps of different scales and the four second heat maps of different scales contains a vanishing point position information in the diamond space; selecting one of the vanishing point position information in the diamond space corresponding to the four first heat maps closest to the first label information in the diamond space corresponding to the sample in the test set as the first selected vanishing point position information; the first label information is the first vanishing point position information; selecting one of the vanishing point position information in the diamond space corresponding to the four second heat maps closest to the second label information in the diamond space corresponding to the sample in the test set as the second selected vanishing point position information; the second label information is the second vanishing point position information; decoding the first selected vanishing point position information and the second selected vanishing point position information to obtain two vanishing point position information in the original physical space; determining the model output accuracy according to the two vanishing point position information in the original physical space and the first label information and the second label information in the original physical space corresponding to the sample in the test set; adjusting the parameters of the horizon detection model based on the Hourglass network according to the model output accuracy.

2. The method of claim 1, wherein, The loss function used in the training process is a Focal Loss loss function.

3. A horizon detection method based on an Hourglass network under a monocular roadside camera view, characterized in that, The method comprises the following steps: obtaining a to-be-detected picture; the to-be-detected picture contains n vehicles; inputting the picture to be detected into a horizon detection model based on an Hourglass network to obtain n diamond space heat maps, each of the diamond space heat maps including four first heat maps of different scales and four second heat maps of different scales; the horizon detection model based on the Hourglass network is obtained by the method for constructing the horizon detection model based on the Hourglass network according to any one of claims 1-2; decoding each of the n diamond space heat maps to obtain n horizons; determining a median value of all horizon slopes and a median value of all horizon intercepts in the n horizons; determining a final horizon according to the median value of all horizon slopes and the median value of all horizon intercepts.

4. The method of claim 3, wherein, wherein, decoding each of the n diamond space heat maps to obtain n horizons, including: for each diamond space heat map, decoding the vanishing point position information in the first heat map and the second heat map of the same scale to obtain four groups of vanishing point position information in the original physical space, each group of vanishing point position information in the original physical space including two vanishing point position information in the original physical space; obtaining a horizon according to the two vanishing point position information in the original physical space; finally, each diamond space heat map obtains four horizons; determining a median value of all horizon slopes and a median value of all horizon intercepts in the four horizons; determining a horizon corresponding to the diamond space heat map according to the median value of all horizon slopes and the median value of all horizon intercepts. 5.A horizon detection model construction device based on an Hourglass network, characterized in that, including: a data set acquisition module configured to acquire an original vehicle picture data set; the original vehicle picture data set includes a plurality of original vehicle pictures; a space conversion module configured to convert the plurality of original vehicle pictures in the original vehicle picture data set from an original physical space to a diamond space to obtain a vehicle picture data set after space conversion; the original vehicle picture includes two label information in the original physical space, and the picture in the vehicle picture data set after space conversion includes two label information in the diamond space; a training data construction module configured to construct a training data set based on the vehicle picture data set after space conversion; the training data set includes a training set and a test set; a training module configured to train a horizon detection model based on an Hourglass network based on the training data set to obtain a trained horizon detection model based on the Hourglass network; the horizon detection model based on the Hourglass network includes an Hourglass network and a multi-scale transformation module, the Hourglass network is configured to output two heat maps based on samples in the training data set, and the multi-scale transformation module is configured to perform multi-scale transformation on the two heat maps output by the Hourglass network to obtain four first heat maps of different scales and four second heat maps of different scales, respectively; the training module is further configured to: inputting the test set into the horizon detection model based on the Hourglass network to obtain the first heat map of 4 different scales and the second heat map of 4 different scales; each of the first heat map of 4 different scales and the second heat map of 4 different scales contains the vanishing point position information under a diamond space; selecting one of the vanishing point position information under the diamond space corresponding to the four first heat maps closest to the first label information under the diamond space corresponding to the sample in the test set as the first selected vanishing point position information; the first label information is the first vanishing point position information; selecting one of the vanishing point position information under the diamond space corresponding to the four second heat maps closest to the second label information under the diamond space corresponding to the sample in the test set as the second selected vanishing point position information; the second label information is the second vanishing point position information; decoding the first selected vanishing point position information and the second selected vanishing point position information to obtain two vanishing point position information under the original physical space; determining the model output accuracy according to the two vanishing point position information under the original physical space and the first label information and the second label information under the original physical space corresponding to the sample in the test set; adjusting the parameters of the horizon detection model based on the Hourglass network according to the model output accuracy.

6. The apparatus of claim 5, wherein, The loss function used in the training process is a Focal Loss loss function.

7. A horizon detection device based on an Hourglass network under monocular roadside camera view, characterized in that, Comprise: a picture acquisition module for acquiring a to-be-detected picture; the to-be-detected picture contains n vehicles; a diamond space heat map group acquisition module for inputting the to-be-detected picture into a horizon detection model based on the Hourglass network to obtain n diamond space heat map groups, each of which includes four first heat maps of different scales and four second heat maps of different scales; the horizon detection model based on the Hourglass network is obtained according to the horizon detection model construction method based on the Hourglass network in any one of claims 1-2; a decoding module for decoding each of the n diamond space heat map groups to obtain n horizons; a slope and intercept determination module for determining the median of all horizon slopes and the median of all horizon intercepts in the n horizons; a horizon determination module for determining the final horizon according to the median of all horizon slopes and the median of all horizon intercepts.

8. The apparatus of claim 7, wherein, The decoding module is also used for: For each diamond space thermal map group, the vanishing point position information in the first thermal map and the second thermal map of the same scale is decoded to obtain 4 groups of vanishing point position information in original physical space, each group of vanishing point position information in original physical space includes 2 vanishing point position information in original physical space; a horizon is obtained according to the 2 vanishing point position information in original physical space; finally, each diamond space thermal map group obtains 4 horizons; The median of all horizon slopes and the median of all horizon intercepts in the 4 horizons are obtained; The horizon corresponding to the diamond space thermal map group is determined according to the median of all horizon slopes and the median of all horizon intercepts.

Images