Training method of missing detection model, missing detection method and device of diversion area

Abstract

The present disclosure provides a method for training a missing detection model, a method and device for detecting missing diversion areas, and relates to the field of artificial intelligence technology, particularly to the fields of high-precision maps, computer vision, and image processing, and can be applied to high-precision map production scenarios. The specific implementation scheme is as follows: based on the road section data of the high-precision map, a target road section combination is determined; wherein the target road section combination includes two road sections, the two road sections are connected to the same road section at a first location, and the high-precision map does not include the diversion area at the first location; based on the image data of the high-precision map, a first road surface image corresponding to the target road section combination is obtained; the missing diversion area is marked in the first road surface image, and the first road surface image and the missing diversion area are used as sample data; based on the sample data, a missing detection model is trained. The present disclosure can reduce the cost of troubleshooting missing diversion areas in high-precision maps, while significantly improving efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the fields of high-precision maps, computer vision, and image processing, and can be applied to high-precision map production scenarios. Background Art

[0002] High-precision maps are commonly used in the field of autonomous driving. Compared to standard navigation maps, HD maps provide higher-precision and richer map information. For example, HD maps can include a wide range of features such as road curvature, road slope, lane position, lane type, lane width, traffic lights, traffic signs, and roadside landmarks.

[0003] Diversion zones, as road surface elements in HD maps, serve as a reminder for vehicles to stay on designated routes and avoid crossing or overtaking lanes. They are a crucial component of HD maps. However, diversion zones are primarily located at intersections that are excessively wide, irregular, or have complex driving conditions, such as ramps at interchanges or other special locations. Currently, HD map generation algorithms may not cover all diversion zone scenarios, resulting in missing diversion zones in HD maps, creating gaps in the road surface, and significantly impacting HD map quality.

[0004] In order to avoid the impact of missing diversion areas on autonomous driving, after generating a high-precision map based on the algorithm, testers need to check the designated route and edit the missing diversion areas in the high-precision map if they find any missing areas. Summary of the Invention

[0005] The present disclosure provides a training method for a missing detection model, and a method and device for detecting missing elements in a diversion area.

[0006] According to one aspect of the present disclosure, a method for training a missing detection model is provided, comprising:

[0007] Determining a target road segment combination based on road segment data in the high-precision map; wherein the target road segment combination includes two road segments, the two road segments are connected to the same road segment at a first location, and the high-precision map does not include a diversion area at the first location;

[0008] Based on the image data of the high-precision map, a first road surface image corresponding to the target road section combination is obtained;

[0009] Marking the missing area of the diversion area in the first road surface image, and using the first road surface image and the missing area of the diversion area as sample data;

[0010] Based on the sample data, a missing detection model is trained.

[0011] According to another aspect of the present disclosure, a method for detecting the absence of a diversion area is provided, comprising:

[0012] Obtaining a second road surface image to be detected based on the image data of the high-precision map;

[0013] The missing area of the diversion area is detected in the second road surface image using the missing area detection model.

[0014] According to another aspect of the present disclosure, a training apparatus for a missing detection model is provided, comprising:

[0015] a road segment determination module, configured to determine a target road segment combination based on road segment data in a high-precision map; wherein the target road segment combination includes two road segments, the two road segments are connected to the same road segment at a first location, and the high-precision map does not include a diversion area at the first location;

[0016] An image acquisition module, configured to obtain a first road surface image corresponding to a target road section combination based on image data of a high-precision map;

[0017] a marking module, configured to mark the missing area of the diversion area in the first road surface image, and use the first road surface image and the missing area of the diversion area as sample data;

[0018] The training module is used to train a missing detection model based on sample data.

[0019] According to another aspect of the present disclosure, a device for detecting the absence of a diversion area is provided, comprising:

[0020] An image determination module, configured to obtain a second road surface image to be detected based on image data of the high-precision map;

[0021] The detection module is used to detect the missing area of the diversion area in the second road surface image by using the missing detection model.

[0022] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0027] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements any one of the methods according to the embodiments of the present disclosure.

[0028] By adopting the technical solution of the embodiment of the present disclosure, the trained missing detection model can be used to automatically detect the missing areas of the diversion area in the road image in the high-precision map, reducing the cost of troubleshooting the missing diversion areas in the high-precision map and greatly improving efficiency.

[0029] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0031] Figure 1 1 is a flow chart of a method for training a missing detection model provided by an embodiment of the present disclosure;

[0032] Figure 2 is a schematic diagram of a diversion area in a diversion scenario in an embodiment of the present disclosure;

[0033] Figure 3 is a schematic diagram of a diversion area in a merging scenario in an embodiment of the present disclosure;

[0034] Figure 4A This is a schematic diagram of an application example of an embodiment of the present disclosure before the display style of a high-precision map is adjusted;

[0035] Figure 4B is a schematic diagram of the display style of the high-precision map after adjustment in the application example of the embodiment of the present disclosure;

[0036] Figure 5 1 is a flow chart of a method for detecting the absence of a diversion area provided by an embodiment of the present disclosure;

[0037] Figure 6 Schematic diagram of the architecture of the preset convolution module in the embodiment of the present disclosure;

[0038] Figure 7 is a schematic diagram of the architecture of the attention module in an embodiment of the present disclosure;

[0039] Figure 8 1 is a schematic diagram of the architecture of the feature fusion module in an embodiment of the present disclosure;

[0040] Figure 9 is a flowchart of a specific application example of an embodiment of the present disclosure;

[0041] Figure 10 This is a schematic diagram of a flow chart for roughly judging the missing diversion area in an application example of an embodiment of the present disclosure;

[0042] Figure 11 is a schematic diagram of the model architecture of the missing detection model in the application example of the embodiment of the present disclosure;

[0043] Figure 12 1 is a schematic diagram of the structure of a training device for a missing detection model provided in one embodiment of the present disclosure;

[0044] Figure 13 is a structural diagram of a training device for a deletion detection model according to another embodiment of the present disclosure;

[0045] Figure 14 1 is a schematic structural diagram of a device for detecting the absence of a diversion area provided by an embodiment of the present disclosure;

[0046] Figure 15 is a schematic structural diagram of a device for detecting the absence of a diversion area according to another embodiment of the present disclosure;

[0047] Figure 16 is a schematic block diagram of an electronic device that can implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0049] Figure 1 A flow chart of a method for training a missing detection model provided by an embodiment of the present disclosure is shown. The method can be applied to a training device for a missing detection model, which can be deployed in an electronic device. The electronic device is, for example, a single or multi-machine terminal, server or other processing device. The terminal may be a user equipment (UE) such as a mobile device, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the method may include:

[0050] S110. Determine a target road section combination based on road section data of the high-precision map; wherein the target road section combination includes two road sections, the two road sections are connected to the same road section at a first location, and the high-precision map does not include a diversion area at the first location.

[0051] S120. Obtain a first road surface image corresponding to the target road section combination based on the image data of the high-precision map.

[0052] S130: Mark the diversion area missing region in the first road surface image, and use the first road surface image and the diversion area missing region as sample data.

[0053] S140. Based on the sample data, a missing detection model is trained.

[0054] In the disclosed embodiments, the road segment data of an HD map may include one or more types of information, such as the location of each road segment in the HD map, its relationship to other road segments, curvature, slope, the location of each lane line included, and the type of each lane line. A road segment may refer to a road connection element in the map, also known as a LINK, which is an abstract representation of each part of a road in the map. For example, a road segment may be the portion of a road between two intersections.

[0055] It should be noted that, in the disclosed embodiments, the connection between road sections may refer to a predecessor or successor relationship between the road sections. For example, when road section A connects to road section B, it can mean that road section A is the predecessor of road section B (i.e., road section B is the successor of road section A), or it can mean that road section B is the predecessor of road section A (i.e., road section A is the successor of road section B). The location where the road sections are connected, such as the first location mentioned above, can be the starting point or end point of the road section.

[0056] For example, in the above step S110, according to the definition of the target road section combination, the target road section combination can be determined by using one or more information including the position of each road section in the high-precision map, the relationship with other road sections, and the type of each lane line included.

[0057] In the disclosed embodiment, the image data of the HD map may include a rendering of the HD map. The first road image may include images of two road segments in the target road segment combination, for example, the first road image may be a rendering of the intersection of the two road segments.

[0058] For example, in the above step S120, the location information of the target road segment combination or the location information of the first location can be read from the road segment data, and the first road surface image can be extracted from the image data of the high-precision map based on the location information.

[0059] Since the two road sections in the target road section combination in the embodiment of the present disclosure are connected to the same road section at the first location, but the high-precision map does not include the diversion area of the first location, the first location may be a location where a diversion area is set in the real-world road environment but the diversion area is missing in the high-precision map.

[0060] For example, in practical applications, diversion areas are often set up at locations where road sections merge or diverge. For example, a diversion area can be set up at a location where two road sections merge into the next road section, or at a location where one road section diverges into two road sections. Figure 2 Schematic diagram of the diversion area of the diversion scenario in the embodiment of the present disclosure is shown. Figure 2 As shown, section 22 is connected to section 21, and section 23 is also connected to section 21. Specifically, section 21 is divided into sections 22 and 23, and a diversion area 24 can be set at the diversion location. Figure 3 A schematic diagram showing the diversion area of the merging scene in an embodiment of the present disclosure is shown. Figure 3 As shown, road section 31 is connected to road section 33, and road section 32 is also connected to road section 33. Specifically, road sections 31 and 32 merge into road section 33, and a diversion area 34 can be set at the merging location. If the HD map does not include a diversion area at a diverging location or merging location, then the diversion area is missing at that diverging location or merging location.

[0061] Based on this, in the method provided in the embodiment of the present disclosure, the diversion area missing area is marked in the first road surface image. Exemplarily, the pixel area containing the first location in the first road surface image can be marked as the diversion area missing area. Optionally, a manual identification step can be further provided, that is, in the above-mentioned step S130, when it is manually confirmed that the diversion area is missing in the first road surface image, the pixel area corresponding to the first location can be marked as the diversion area missing area, and sample data can be obtained based on the first road surface image and the diversion area missing area, that is, the diversion area missing area is used as the label of the first road surface image to obtain labeled sample data. In this way, step S140 can be executed to train a missing detection model based on the sample data, and the missing detection model can be used to detect the diversion area missing area in the input road surface image.

[0062] In the disclosed embodiment, the output information or prediction result of the missing area detection model can be used to characterize whether the road surface image contains a missing diversion area and the location information of the missing diversion area. For example, the prediction result can include the coordinates of the detection frame of the missing diversion area. When the prediction result is empty, it means that the image does not contain the missing diversion area. For another example, the prediction result can include the coordinates of the diversion area detection frame and the detection frame type, where the type includes normal diversion area or missing diversion area. When the type is missing diversion area, the coordinates of the diversion area detection frame are the location information of the missing diversion area.

[0063] As can be seen, the training method for the missing area detection model provided in the embodiments of this disclosure can be used to automatically detect missing diversion areas in road images in high-precision maps using the trained missing area detection model. Compared to manually inspecting designated routes to determine if diversion areas are missing from high-precision maps, this method can reduce manpower and material costs while significantly improving efficiency. Furthermore, in the above method, by utilizing road segment data from high-precision maps and searching for combinations of target road segments that meet the requirements, the missing diversion areas can be found, which can improve accuracy.

[0064] In an exemplary embodiment, the above-mentioned step S110, determining the target road section combination based on the road section data of the high-precision map, may include: determining a first road section based on the road section information table of the high-precision map; wherein the first road section connects multiple other road sections at the same location; when the first road section is not connected to the diversion area, determining two road sections that meet the conditions for establishing the diversion area from multiple other road sections; and obtaining the target road section combination based on the two road sections that meet the conditions for establishing the diversion area.

[0065] Specifically, the HD map's road segment information table can record the connection relationship between each road segment and other road segments, such as the predecessor and successor road segments of each road segment. Based on this, if the road segment information table records that a certain road segment has multiple predecessor or successor road segments, the road segment can be determined to be connected to multiple other road segments at the same location, that is, the road segment can be determined to be the first road segment.

[0066] The road segment information table of the HD map may also record the type of lane markings within the road segment. For example, the type may indicate whether the lane marking is a dashed line, a solid line, or a lane edge connecting to a diversion area. If the lane edge is not a lane edge connecting to a diversion area, it can be determined that the first road segment is not connected to a diversion area.

[0067] In a real-world road environment, a diversion area may be established between any two adjacent road segments connected to the first road segment at the same location. Whether each pair of adjacent road segments meets the diversion area establishment criteria can be determined separately to identify two road segments from the multiple other road segments that meet the diversion area establishment criteria. If two road segments meet the diversion area establishment criteria, these two road segments are identified as the target road segment combination.

[0068] This implementation method can use the road section information table to traverse each road section, find the first road section that connects multiple other road sections, and then judge the conditions for establishing diversion areas for other road sections connected to the first road section, so that the areas where diversion areas are missing can be automatically identified with high accuracy.

[0069] In an exemplary embodiment, when a first road section is not connected to a diversion area, determining two road sections that meet the conditions for establishing a diversion area among multiple other road sections may include: when the first road section is not connected to the diversion area, determining the distance between the first lane line of a second road section among the multiple other road sections and the last lane line of a third road section among the multiple other road sections; when the distance between the first lane line of the second road section and the last lane line of the third road section is not greater than a first value, determining the distance between the first lane line of the second road section and the guardrail and the distance between the last lane line of the third road section and the guardrail; when the distance between the first lane line and the guardrail or the distance between the last lane line and the guardrail is not less than a second value, determining that the second road section and the third road section meet the conditions for establishing a diversion area.

[0070] The second road section and the third road section may be two adjacent road sections among a plurality of other road sections. It should be noted that, in the embodiment disclosed herein, the order of the lane lines in the second road section and the third road section is not limited, and may be ordered from right to left or from left to right. Here, left and right are relative to the direction of the road. For example, Figure 3 In the example, section 31 is the second section, section 32 is the third section, and the lane lines are sorted from right to left. Figure 3 As shown, the first lane line of section 31 is the rightmost lane line 311, and the last lane line of section 32 is the leftmost lane line 321. Based on this example, it can be understood that in the above embodiment, the distance between the first lane line of the second section and the last lane line of the third section is the distance between the second section and the third section. In actual applications, if the two lane lines are not parallel, the distance between the two lane lines at the vertex where they merge or diverge can be taken as the distance between the two lane lines. For example, Figure 3 The distance between lane line 311 and lane line 321 in is the distance at location 35 .

[0071] The guardrail in the above embodiment includes a guardrail between two lane lines. In practical applications, the guardrail closest to the lane line can be used to calculate the distance between the lane line and the guardrail. For example, Figure 3 In this example, guardrail 36 is used to calculate the distance between lane marking 311 and the guardrail, and guardrail 37 is used to calculate the distance between lane marking 321 and the guardrail. In other words, the distance between lane marking 311 and the guardrail is the distance between lane marking 311 and guardrail 36, and the distance between lane marking 321 and the guardrail is the distance between lane marking 321 and guardrail 37.

[0072] In the above embodiment, the first value and the second value may be preset thresholds, and the first value and the second value may not be equal.

[0073] According to the above implementation, the conditions for establishing a diversion zone are that the distance between two road sections is too small and the distance between the lane edge and the guardrail is too large. When the distance between the two road sections is too small, vehicles must strictly follow the prescribed route; when the distance between the lane edge and the guardrail is too large, a diversion zone is required to regulate the road's driving area. Based on this, the diversion zone establishment conditions of the above implementation meet the requirements of actual road environments, can improve the accuracy of determining diversion zone absence, facilitate the determination of accurate sample data, and thus enhance the accuracy of the absence detection model.

[0074] In an exemplary embodiment, step S120, obtaining a first road surface image corresponding to the target road section combination based on the image data of the high-precision map, may include: obtaining rendering data of the high-precision map based on multiple map elements in the high-precision map, wherein the multiple map elements include a road surface, a diversion area, and a guardrail; and obtaining a first road surface image corresponding to the target road section combination based on the rendering data.

[0075] In the above embodiment, other map elements except the road surface, diversion area and guardrail, such as background, buildings, green space and water system, can be turned off in the high-precision map, and then the rendering data of the high-precision map can be exported.

[0076] For example, after disabling other map elements, all rendered image data of the high-precision map can be exported, and then the first road surface image can be determined from the rendered image data based on the location information of the target road segment combination or the location information of the first location. Alternatively, after disabling other map elements, the rendered image data corresponding to the target road segment combination can be exported based on the location information of the target road segment combination or the location information of the first location, and this rendered image data can be used as the first road surface image.

[0077] Using the above-mentioned implementation method, the first road surface image corresponding to the target road section combination is intercepted after closing irrelevant map elements. Therefore, other irrelevant elements are not displayed in the first road surface image, avoiding interference of other irrelevant elements on the prediction of the model, and facilitating the identification of whether there is a missing diversion area and the location of the missing diversion area.

[0078] Exemplarily, in some embodiments, the training method of the missing detection model may also include: setting the color of the road surface in the rendered image data to a first color; setting the color of the diversion area in the rendered image data to a second color; setting the color of the guardrail in the rendered image data to a third color; wherein the color difference between the first color, the second color, and the third color is greater than a third value.

[0079] For example, the colors of the road surface, diversion area, and guardrail can be modified in the HD map, so that the colors of the road surface, diversion area, and guardrail in the rendered image data are set to three different colors with large color differences. For example, the color difference can be measured using ΔE (Delta-E), and the third value can be 5, that is, the color of each of the above map elements is a color with ΔE greater than 5. For example, the color of the road surface is set to black, the color of the diversion area is set to blue, and the color of the guardrail is set to red.

[0080] To facilitate understanding of the above embodiments, Figure 4A A schematic diagram showing an application example of an embodiment of the present disclosure, showing a high-precision map before the display style is adjusted; Figure 4B The following figure shows the schematic diagram of the high-precision map after the display style is adjusted in this application example. Figure 4A As shown in the figure, before the style adjustment, the high-precision map rendering may include map elements such as road surface 41, diversion area 42, guardrail 43, road sign 44, green space and water system 45. By turning off some map elements and setting the colors of road surface 41, diversion area 42 and guardrail 43, the following can be obtained: Figure 4B The first road surface image is shown.

[0081] By adopting the above implementation, the color difference between the road surface, diversion area and guardrail can be increased, thereby improving the quality of sample data and correspondingly improving the accuracy of model prediction.

[0082] Optionally, in some embodiments of the present disclosure, after obtaining the first road surface image, data enhancement can be performed based on the first road surface image to obtain more images and obtain richer sample data. For example, data enhancement can be achieved by performing horizontal flipping, vertical flipping, random rotation, and multi-scale scaling on the first road surface image. Multi-scale scaling can generate diversion area images of different sizes, thereby enabling the model to better detect diversion areas of different sizes.

[0083] Optionally, some embodiments of the present disclosure may also include improvements to the model architecture of the missing detection model.

[0084] Exemplarily, the defect detection model may include a backbone network, a feature fusion module, and a prediction module. The backbone network is used to extract features at multiple scales from the road surface image input to the defect detection model. The feature fusion module is used to fuse the features at multiple scales to generate fused features. The prediction module is used to generate prediction results corresponding to the fused features. The prediction results are used to characterize the location of missing areas in the diversion zone.

[0085] Exemplarily, the backbone network may include a basic convolution module, a preset convolution module, and an attention module. The basic convolution module is, for example, a CBS (Conv-BatchNorm-SiLU, convolution-batch normalization-activation) module, which is used to extract basic features of road images. The preset convolution module can be used to: sequentially perform depthwise separable convolution and a first conventional convolution based on the basic features to obtain a first deep feature; sequentially perform a second conventional convolution, a depthwise separable convolution, and a third conventional convolution based on the basic features to obtain a second deep feature; and fuse the first deep feature with the second deep feature to obtain the output feature of the preset convolution module. The attention module can be used to weight the features on the channel based on the output features of the preset convolution module using a channel attention mechanism to obtain features of a specific scale.

[0086] In some embodiments of the present disclosure, the feature fusion module may use multiple fusion methods to fuse the multi-scale features output by the backbone network. Exemplarily, the multi-scale features output by the backbone network include first-scale features, second-scale features, and third-scale features. The feature fusion module may include a first fusion submodule, a second fusion submodule, and a third fusion submodule. The first fusion submodule is used to obtain intermediate features based on the third-scale features and the second-scale features, and to fuse the intermediate features and the first-scale features to obtain the first output features in the fused features. The second fusion submodule is used to fuse the first output features in the fused features, the second-scale features, and the intermediate features to obtain the second output features in the fused features. The third fusion submodule is used to fuse the second output features in the fused features and the third-scale features to obtain the third output features in the fused features.

[0087] The following describes the application process of the defect detection model in the embodiment of the present disclosure. Specifically, the present disclosure also provides a method for detecting the absence of a diversion area. Figure 5 The flowchart of the method for detecting the absence of a diversion area provided by an embodiment of the present disclosure is shown. The method can be applied to a device for detecting the absence of a diversion area, which can be deployed in an electronic device. The electronic device is, for example, a single or multi-machine terminal, server or other processing device. Among them, the terminal can be a user equipment (UE) such as a mobile device, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 5 As shown, the method may include:

[0088] S510: Obtain a second road surface image to be detected based on the image data of the high-precision map.

[0089] S520: Detect missing areas of the diversion area in the second road surface image using a missing area detection model.

[0090] Exemplarily, the missing detection model is trained based on the training method in the aforementioned embodiment.

[0091] For example, in step S510, the second road surface image to be detected can be obtained by traversing the image data of the high-precision map. For example, a certain area can be designated as the detection range, and each road surface image within the detection range can be traversed, with the traversed road surface image being used as the second road surface image to be detected.

[0092] For example, in step S520, the second road surface image can be input into the missing area detection model, and the second road surface image can be processed in the missing area detection model to obtain a prediction result output by the missing area detection model. The prediction result can be used to characterize the location information of the missing area of the diversion area in the second road surface image. For example, the prediction result can include the coordinates of the detection frame of the missing area of the diversion area. When the prediction result is empty, it means that the image does not contain the missing area of the diversion area. For another example, the prediction result can include the coordinates of the detection frame of the diversion area and the detection frame type, where the type includes normal diversion area or missing diversion area. When the type is missing diversion area, the coordinates of the detection frame of the diversion area are the location information of the missing diversion area.

[0093] By adopting the above-mentioned diversion area missing detection method, the diversion area missing area can be automatically detected in the road surface image in the high-precision map, which can reduce the cost of troubleshooting the diversion area missing in the high-precision map and greatly improve efficiency.

[0094] Optionally, after the diversion area is detected to be missing, the diversion area can be supplemented in the high-precision map based on the position information of the diversion area missing.

[0095] In an exemplary embodiment, step S520, using the missing detection model to detect the missing area of the diversion area in the second road surface image, includes: using the backbone network in the missing detection model to extract multiple features of different scales in the second road surface image; using the feature fusion module in the missing detection model to fuse the multiple features of different scales to obtain a fused feature; using the prediction module in the missing detection model to obtain a prediction result corresponding to the fused feature; wherein the prediction result is used to characterize the location information of the missing area of the diversion area.

[0096] Specifically, the defect detection model consists of a backbone network, a feature fusion module, and a prediction module. The backbone network extracts features at multiple scales from the road surface image input to the defect detection model. The feature fusion module fuses these features at multiple scales to generate fused features. The prediction module generates prediction results corresponding to the fused features. These prediction results represent the location of missing areas in the diversion zone.

[0097] By adopting the above implementation method, refined local features and global features can be extracted and fused, and the prediction accuracy of the model can be improved by improving the feature expression ability, thereby improving the detection accuracy of the missing area of the diversion zone.

[0098] In an exemplary embodiment, a backbone network in a missing detection model is used to extract multiple features of different scales in a second road surface image, including: inputting basic features in the second road surface image into a preset convolution module in the backbone network; sequentially performing depthwise separable convolution and a first conventional convolution based on the basic features to obtain a first deep feature; wherein the convolution kernel size of the depthwise separable convolution is greater than a predetermined size; sequentially performing a second conventional convolution, a depthwise separable convolution, and a third conventional convolution based on the basic features to obtain a second deep feature; and fusing the first deep feature with the second deep feature to obtain an output feature of the preset convolution module.

[0099] Specifically, the backbone network may include a basic feature extraction module and a preset convolution module. It may further include an attention module, a pooling module, and the like. The basic feature extraction module is used to extract basic features from road images; the preset convolution module is used to obtain output features of the module based on the basic features. These output features, after processing by the attention module and / or the pooling module, can serve as features at one scale output by the backbone network. Subsequently, the preset convolution module can be used to obtain new features, which, after processing by the attention module and / or the pooling module, can serve as features at other scales output by the backbone network.

[0100] Exemplarily, the basic feature extraction module can be a CBS module, which is the basic convolution layer of the backbone network, which can include a convolution layer, a normalization layer, and an activation layer. For example, assuming that the input feature of the CBS module is X and the output feature is Y, the processing process of the CBS module is:

[0101] T = BN(Conv(X));

[0102]

[0103] Where T is the output feature of the normalization layer; Conv(X) means convolution of X, for example, 3x3 convolution; BN means normalization operation.

[0104] Exemplarily, the preset convolution module may include 2 depth-wise separable convolution layers and 3 regular convolution layers. Figure 6 FIG. 1 shows a schematic diagram of the architecture of the preset convolution module in the disclosed embodiment. Figure 6 As shown, for the input features of the preset convolution module (such as the basic features output by the CBS module), on the one hand, depthwise separable convolution 601 and regular convolution 602 are sequentially performed on the features to obtain the first deep features; on the other hand, regular convolution 603, depthwise separable convolution 604 and regular convolution 605 are sequentially performed on the features to obtain the second deep features. Afterwards, the output features of the preset convolution module are obtained by performing concatenation 606 on the first deep features and the second deep features. Among them, normalization (BN) can be performed between the depthwise separable convolution 601 and the regular convolution 602, and normalization and activation (ReLu) can be performed on the features output by the regular convolution 602. Normalization and activation can be performed between the regular convolution 603 and the depthwise separable convolution 604, and normalization can be performed between the depthwise separable convolution 604 and the regular convolution 605; normalization and activation can be performed on the features output by the regular convolution 605.

[0105] According to the above embodiment, Figure 6 In the architecture shown, the kernel size of the depthwise separable convolution can be larger. For example, assuming that both the input and output features are feature maps of size 128x256x256, that is, the number of channels is 128 and the feature size is 256x256, then the depthwise separable convolution layer can use a 5x5 convolution kernel.

[0106] In real-world scenarios, the flow area can be quite large. Convolutions with small receptive fields cannot adequately capture the features of this large flow area, resulting in reduced performance. Therefore, large-scale depthwise separable convolutions are used to capture this large flow area and increase the model's expressiveness. Furthermore, conventional convolutions can use small convolution kernels, such as 1×1 kernels, to increase convolution speed with little performance loss.

[0107] The processing of the preset convolution module in the above example can be expressed as:

[0108] Y1=ReLu(BN(Conv 1×1 (BN(DWConv 5×5 (X))))));

[0109] Y2=ReLu(BN(Conv 1×1 (BN(DWConv 5×5 (ReLu(BN(Conv 1×1 (X))))))));

[0110] Y=Concat[Y1,Y2]

[0111] Among them, X is the input feature of the preset convolution module; Y is the output feature of the preset convolution module; Y1 is the first deep feature; Y2 is the first deep feature; DWConv 5×5 represents 5×5 depth separable convolution, BN represents normalization, Conv 1×1 Represents a 1×1 regular convolution, ReLu represents an activation function, and Concat represents concatenation.

[0112] For example, Figure 7 FIG. 1 shows a schematic diagram of the architecture of the attention module in the embodiment of the present disclosure. Figure 7 As shown, the attention module may include an average pooling layer 701, a convolutional layer 702, a convolutional layer 703, an activation layer 704, a dot product calculation unit 705, and an addition unit 706. Specifically, in the attention module, the average pooling layer 701, the convolutional layer 702, the convolutional layer 703, and the activation layer 704 are sequentially executed based on the input features, and the obtained features are dot-producted and summed with the input features to obtain the output features. The specific processing process can be expressed as:

[0113] Y att =Conv(Conv(AVGPool(X)));

[0114]

[0115] Among them, X is the input feature of the attention module; Y is the output feature of the attention module; AVGPool represents average pooling, and Conv represents convolution.

[0116] This attention module uses the channel attention mechanism to weight the input features on the channel, highlighting important features and reducing minor features, thereby improving the model's feature expression ability and correspondingly improving the model's prediction accuracy.

[0117] In an exemplary embodiment, a feature fusion module in a missing detection model is used to fuse multiple features of different scales to obtain a fused feature, which may include: inputting the first scale feature, the second scale feature, and the third scale feature output by the backbone network into the feature fusion module; in the feature fusion module, obtaining an intermediate feature based on the third scale feature and the second scale feature, and fusing the intermediate feature with the first scale feature to obtain a first output feature in the fused feature.

[0118] The first-scale features, second-scale features, and third-scale features output by the backbone network can be features of different sizes. For example, the size of the first-scale features is larger than the size of the second-scale features, and the size of the second-scale features is larger than the size of the third-scale features. For example, the first-scale features are 80×80 feature maps, the second-scale features are 40×40 feature maps, and the third-scale features are 20×20 feature maps.

[0119] The above-mentioned implementation provides a feature fusion method. Unlike existing technologies that directly fuse input features or fuse the current-level input features with the previous-level fused output features to obtain the current-level output features, this implementation fuses at least two input features to obtain intermediate features, which are then fused with other input features. Thus, by providing intermediate features, additional feature fusion methods are added, which can enhance the model's feature expression capabilities and thus improve the accuracy of diversion area loss detection.

[0120] In an exemplary embodiment, a feature fusion module in a missing detection model is used to fuse multiple scale features to obtain a fused feature, and the method also includes: in the feature fusion module, fusing the first output feature, the second scale feature and the intermediate feature in the fused feature to obtain the second output feature in the fused feature.

[0121] In this implementation, after obtaining the first output feature, the first output feature, the second scale feature, and the intermediate feature are fused to obtain the second output feature. This means that the current level's output feature is obtained by combining the current level's input features, the intermediate features, and the fused output features of the previous level. This provides a different feature fusion method from the above implementation, adding a new feature fusion method that can enhance the model's feature expression capabilities and thus improve the accuracy of diversion area loss detection.

[0122] In an exemplary embodiment, a feature fusion module in a missing detection model is used to fuse multiple features of different scales to obtain a fused feature, and the method also includes: in the feature fusion module, fusing the second output feature and the third scale feature in the fused feature to obtain a third output feature in the fused feature.

[0123] In this implementation, after obtaining the second output feature, the second output feature and the third scale feature are fused to obtain the third output feature. This means that the current level's output feature is obtained by combining the input feature of the current level with the fused output feature of the previous level. This provides a different feature fusion method from the above implementation, adding a new feature fusion method that can enhance the model's feature expression capabilities and thus improve the accuracy of diversion area loss detection.

[0124] In order to more clearly understand the above embodiment, illustratively, Figure 8 A schematic diagram of the architecture of the feature fusion module in the embodiment of the present disclosure is provided. Figure 8 As shown, the features input to the feature fusion module include the first scale feature C1, the second scale feature C2 and the third scale feature C3, and the fusion features output by the feature fusion module include the first output feature P1, the second output feature P2 and the third output feature P3. Figure 8 In the network architecture shown, an intermediate feature M1 obtained by fusing the second scale feature C2 and the third scale feature C3 is provided. By adding network nodes of the intermediate feature, multiple fusion methods are provided.

[0125] Specifically, based on the first scale feature C1 and the intermediate feature M2, the first output feature P1 is fused; based on the second scale feature C2, the intermediate feature M1 and the first output feature P1, the second output feature P2 is fused; based on the third scale feature C3 and the second output feature P2, the third output feature P3 is fused.

[0126] It can be understood that the above implementation has made multiple optimizations based on the bidirectional feature pyramid. First, the fusion node with only input features is removed. This node contributes little to the feature network, and its removal improves efficiency. Secondly, an intermediate node is added between the input and output nodes of the same level to fuse more features. In addition, a bidirectional path of the intermediate node is added, and the intermediate node is reused multiple times to achieve higher-level feature fusion. Through this feature fusion module, the output information can be further refined and fused to enhance the performance of the entire model, serving as an effective connection between the backbone network and prediction.

[0127] A specific application example according to an embodiment of the present disclosure is provided below. Figure 9 The flowchart of this application example is shown in FIG. Figure 9 As shown in the figure, in this application example, the method for detecting the absence of the guide area includes:

[0128] S901. Rough judgment of missing diversion area.

[0129] Specifically, high-precision map data is used to roughly identify missing diversion areas, avoiding manual inspections to obtain samples.

[0130] S902: The diversion area marking is missing.

[0131] Specifically, the missing diversion areas are marked to provide training samples for the model.

[0132] S903: Missing detection model training.

[0133] Specifically, the visual model is trained using labeled missing diversion area images until the model converges.

[0134] S904: Determine whether the missing detection model has converged.

[0135] S905: If the missing detection model has not converged, continue to tune the missing detection model parameters.

[0136] S906: If the missing detection model converges, the missing detection model is deployed.

[0137] Specifically, the converged model is deployed to the production application environment to predict the missing diversion areas in the high-precision map, and the longitude and latitude of the missing diversion areas are marked for manual editing and repair of the missing diversion areas.

[0138] In the above step S901, the LINK information of the high-precision map data can be used to preliminarily screen the missing diversion areas. Figure 10 The flowchart of the rough judgment of the missing diversion area is shown in FIG.

[0139] like Figure 10 As shown in the figure, the process of rough judgment of missing diversion area includes:

[0140] S1001. Search for LINKs with multiple predecessor and successor relationships. Specifically, this is achieved by querying a LINK information table. If there are more than two predecessor LINKs or successor LINKs, these are LINKs with multiple predecessor and successor relationships.

[0141] S1002. Record the links for which no diversion area is established. Specifically, for the links corresponding to the diversion lane edges marked, it can be determined that a diversion area has been established, and record the remaining links for which no diversion area is established.

[0142] S1003. Query the two predecessor LINKs or successor LINKs of the recorded LINK.

[0143] S1004: Query the lane edges corresponding to the two links. Specifically, the lane edge corresponding to the first lane of the left link and the lane edge corresponding to the last lane of the right link can be queried.

[0144] S1005: Determine whether the distance between the two lane edges is greater than a threshold. If so, proceed to step S1003 for the next LINK for which no diversion area has been established.

[0145] S1006: If the distance between the two lane edges is not greater than the threshold, determine whether the distance between the guardrail and the lane edge is less than the threshold. If so, proceed to step S1003 for the next LINK for which no diversion area has been established.

[0146] S1007: If the distance between the guardrail and the lane edge is not less than the threshold, the two lane edges are generated as a common diversion zone. The process proceeds to step S1003 for the next link for which no diversion zone has been established.

[0147] In the above step S902, the missing diversion area annotation stage provides training samples for the missing detection model. In order to better improve the performance of the model, the style of the high-precision map is first adjusted. For example, the color of the road surface is set to black, the diversion area color is set to blue, and the guardrail color is set to red to increase the color difference between the elements. At the same time, some elements are closed and not allowed to be displayed, such as background, buildings, green space and water systems, because these elements will interfere with the prediction of the model. Non-essential elements are closed and only necessary elements are displayed. Then, screenshots are taken of the abnormal points identified in the rough judgment stage of the missing diversion area, and the screenshots are annotated using annotation tools.

[0148] To better train the model, we use data augmentation methods to process the original images before training. These include commonly used horizontal and vertical flipping and random rotation. Furthermore, we use multi-scale scaling to resize the original images, enabling the model to better detect smaller diversion areas.

[0149] Figure 11 The model architecture of the missing detection model in this application example is shown in Figure 2. Figure 11 As shown, the missing detection model includes a backbone network 1101, a feature fusion module 1102 and a prediction module 1103.

[0150] Among them, the backbone network 1101 is used to extract features, which includes 4 modules: CBS module, preset convolution module (ShuffeLayer), attention module (CoorAtt) and SPPF module. Among them, the CBS module consists of a convolution layer, a normalization layer and a SILU activation function layer. The features are first extracted by the convolution layer, and then the features are standardized by the normalization layer, and finally the nonlinear expression ability of the network is enhanced by the activation function layer. ShuffeLayer uses ShuffleNet as the basic backbone and has made multiple modifications to improve its performance. The CoordAtt module is a novel, lightweight and efficient attention mechanism attention module used to enhance model performance and can be easily integrated into mobile networks to improve accuracy with little additional computational overhead.

[0151] The feature fusion module 1102 is improved based on the bidirectional feature pyramid network to achieve more efficient feature fusion. The bidirectional feature pyramid network is designed for target detection algorithms and can extract features of different scales. It uses feature maps at different stages to further form a feature pyramid network to detect objects of different scales. In this application example, three optimizations are performed on the basis of the original feature pyramid network: first, nodes with only one input edge are removed because they contribute little to the feature network; second, an additional edge is added between the original input and output nodes at the same level to fuse more features; third, each bidirectional path is regarded as a feature network layer, which is repeated multiple times to achieve higher-level feature fusion. The bidirectional feature pyramid network architecture can enhance the performance of the entire model by further refining and fusing output information, serving as an effective connection between the backbone network and prediction.

[0152] According to an embodiment of the present disclosure, the present disclosure also provides a training device for a missing detection model. Figure 12 FIG. 1 shows a schematic diagram of a structure of a training device for a missing detection model provided by an embodiment of the present disclosure. Figure 12 As shown, the device includes:

[0153] a road segment determination module 1210 for determining a target road segment combination based on road segment data in a high-precision map; wherein the target road segment combination includes two road segments, the two road segments are connected to the same road segment at a first location, and the high-precision map does not include a diversion area at the first location;

[0154] An image acquisition module 1220 is configured to obtain a first road surface image corresponding to a target road segment combination based on image data of a high-precision map;

[0155] A marking module 1230 is configured to mark the area where the diversion area is missing in the first road surface image, and use the first road surface image and the area where the diversion area is missing as sample data;

[0156] The training module 1240 is used to train a missing detection model based on sample data.

[0157] Figure 13 FIG2 is a schematic structural diagram of a training device for a missing detection model according to another embodiment of the present disclosure. The device may include one or more features of the training device for a missing detection model in the above embodiment. In one possible implementation, the road segment determination module 1210 includes:

[0158] The determining unit 1310 is configured to determine a first road segment based on the road segment information table of the high-precision map; wherein the first road segment connects to multiple other road segments at the same location;

[0159] A screening unit 1320 is configured to determine, when the first road segment is not connected to the diversion area, two road segments that meet the conditions for establishing the diversion area from a plurality of other road segments;

[0160] The combining unit 1330 is configured to obtain a target road segment combination based on two road segments that meet the diversion area establishment conditions.

[0161] Illustratively, in the embodiment of the present disclosure, the screening unit 1220 is configured to:

[0162] In a case where the first road segment is not connected to the diversion area, determining a distance between a first lane marking of a second road segment among the plurality of other road segments and a last lane marking of a third road segment among the plurality of other road segments;

[0163] When the distance between the first lane marking of the second road section and the last lane marking of the third road section is not greater than the first value, determining the distance between the first lane marking and the guardrail and the distance between the last lane marking and the guardrail;

[0164] When the distance between the first lane line and the guardrail or the distance between the last lane line and the guardrail is not less than the second value, it is determined that the second road section and the third road section meet the conditions for establishing the diversion area.

[0165] For example, Figure 13 As shown, the image acquisition module 1220 includes:

[0166] A rendering unit 1340 is configured to obtain rendering data of a high-precision map based on a plurality of map elements in the high-precision map, wherein the plurality of map elements include a road surface, a diversion area, and a guardrail;

[0167] The interception unit 1350 is configured to obtain a first road surface image corresponding to the target road segment combination based on the rendering image data.

[0168] For example, Figure 13 As shown, the training device for the defect detection model further includes a color setting module 1360, which is used to:

[0169] Set the color of the road surface in the rendering data to the first color;

[0170] Set the color of the diversion area in the rendering data to the second color;

[0171] Set the color of the guardrail in the rendering data to the third color;

[0172] The color difference between any two of the first color, the second color, and the third color is greater than a third value.

[0173] According to an embodiment of the present disclosure, the present disclosure also provides a device for detecting the absence of a diversion area. Figure 14FIG. 1 shows a schematic structural diagram of a device for detecting the absence of a diversion area according to an embodiment of the present disclosure. Figure 14 As shown, the device includes:

[0174] An image determination module 1410 is configured to obtain a second road surface image to be detected based on the image data of the high-precision map;

[0175] The detection module 1420 is configured to detect a missing area of the diversion area in the second road surface image using a missing area detection model.

[0176] Figure 15 14 is a schematic structural diagram of a device for detecting the absence of a flow guide area according to another embodiment of the present disclosure. The device may include one or more features of the device for detecting the absence of a flow guide area of the above embodiment. In one possible implementation, the detection module 1420 includes:

[0177] A feature extraction unit 1510 is configured to extract a plurality of features of different scales from the second road surface image using a backbone network in the missing detection model;

[0178] A fusion unit 1520 is used to fuse multiple features of different scales using a feature fusion module in the missing detection model to obtain a fused feature;

[0179] The prediction unit 1530 is used to obtain a prediction result corresponding to the fusion feature using the prediction module in the missing detection model; wherein the prediction result is used to represent the position information of the missing area of the diversion area.

[0180] Illustratively, in the embodiment of the present disclosure, the feature extraction unit 1510 is configured to:

[0181] Inputting the basic features in the second road surface image into a preset convolution module in the backbone network;

[0182] Performing depth-wise separable convolution and first conventional convolution in sequence based on the basic features to obtain a first deep feature; wherein the convolution kernel size of the depth-wise separable convolution is larger than a predetermined size;

[0183] Based on the basic features, the second regular convolution, the depth-separable convolution and the third regular convolution are performed in sequence to obtain the second deep features;

[0184] The first deep feature and the second deep feature are fused to obtain the output feature of the preset convolution module.

[0185] Illustratively, in the embodiment of the present disclosure, the fusion unit 1520 is configured to:

[0186] Input the first-scale features, second-scale features, and third-scale features output by the backbone network into the feature fusion module;

[0187] In the feature fusion module, an intermediate feature is obtained based on the third-scale feature and the second-scale feature, and the intermediate feature and the first-scale feature are fused to obtain a first output feature in the fused feature.

[0188] Illustratively, in the embodiment of the present disclosure, the fusion unit 1520 is further configured to:

[0189] In the feature fusion module, the first output feature, the second scale feature and the intermediate feature in the fused feature are fused to obtain the second output feature in the fused feature.

[0190] Illustratively, in the embodiment of the present disclosure, the fusion unit 1520 is further configured to:

[0191] In the feature fusion module, the second output feature and the third scale feature in the fused feature are fused to obtain the third output feature in the fused feature.

[0192] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0193] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0194] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0195] Figure 16 A schematic block diagram of an example electronic device 1600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0196] like Figure 16As shown, device 1600 includes a computing unit 1601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1602 or a computer program loaded from a storage unit 1608 into a random access memory (RAM) 1603. Various programs and data required for the operation of device 1600 can also be stored in RAM 1603. Computing unit 1601, ROM 1602, and RAM 1603 are connected to each other via a bus 1604. An input / output (I / O) interface 1605 is also connected to bus 1604.

[0197] Various components in device 1600 are connected to I / O interface 1605, including an input unit 1606, such as a keyboard and mouse; an output unit 1607, such as various types of displays and speakers; a storage unit 1608, such as a magnetic disk and optical disk; and a communication unit 1609, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1609 allows device 1600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0198] The computing unit 1601 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 1601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1601 performs the various methods and processes described above, such as the method for training the absence detection model and the method for detecting absence of the guide area. For example, in some embodiments, the method for training the absence detection model and the method for detecting absence of the guide area can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1600 via the ROM 1602 and / or the communication unit 1609. When the computer program is loaded into the RAM 1603 and executed by the computing unit 1601, one or more steps of the method for training the absence detection model and the method for detecting absence of the guide area described above can be performed. Alternatively, in other embodiments, the computing unit 1601 may be configured to execute the missing detection model training method and the guide area missing detection method in any other appropriate manner (for example, by means of firmware).

[0199] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0200] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0201] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0202] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0203] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0204] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0205] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0206] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for training a missing detection model, comprising: Determining a target road segment combination based on road segment data of a high-precision map; wherein the target road segment combination includes two road segments, the two road segments are connected to the same road segment at a first location, and the high-precision map does not include a diversion area at the first location; Based on the image data of the high-precision map, a first road surface image corresponding to the target road section combination is obtained; Marking a diversion area missing region in the first road surface image, and using the first road surface image and the diversion area missing region as sample data; Based on the sample data, a missing detection model is trained; The determination of the target road section combination based on the road section data of the high-precision map includes: Determining a first road segment based on a road segment information table of a high-precision map; wherein the first road segment connects multiple other road segments at the same location; In a case where the first road section is not connected to a diversion area, determining two road sections that meet the conditions for establishing a diversion area from the multiple other road sections; Obtaining a target road section combination based on the two road sections that meet the diversion area establishment conditions; Wherein, when the first road section is not connected to the diversion area, determining two road sections that meet the conditions for establishing the diversion area from the multiple other road sections includes: In a case where the first road segment is not connected to a diversion area, determining a distance between a first lane marking of a second road segment among the multiple other road segments and a last lane marking of a third road segment among the multiple other road segments; When the distance between the first lane marking of the second road section and the last lane marking of the third road section is not greater than a first value, determining a distance between the first lane marking and the guardrail and a distance between the last lane marking and the guardrail; When the distance between the first lane line and the guardrail or the distance between the last lane line and the guardrail is not less than a second value, it is determined that the second road section and the third road section meet the conditions for establishing a diversion area.

2. The method according to claim 1, wherein The obtaining, based on the image data of the high-precision map, a first road surface image corresponding to the target road section combination, includes: Obtaining rendering data of the high-precision map based on a plurality of map elements in the high-precision map, wherein the plurality of map elements include a road surface, a diversion area, and a guardrail; Based on the rendering image data, a first road surface image corresponding to the target road section combination is obtained.

3. The method according to claim 2, wherein: The method further comprises: Setting the color of the road surface in the rendering data to a first color; Setting the color of the diversion area in the rendering data to a second color; Setting the color of the guardrail in the rendering data to a third color; The color difference between any two of the first color, the second color, and the third color is greater than a third value.

4. A method for detecting the absence of a diversion area, comprising: Obtaining a second road surface image to be detected based on the image data of the high-precision map; Utilize a missing detection model to detect missing areas of the diversion area in the second road surface image; wherein the missing detection model is trained based on the method according to any one of claims 1 to 3.

5. The method according to claim 4, wherein The detecting the missing area of the diversion area in the second road surface image by using the missing area detection model includes: Extracting features of multiple scales from the second road surface image using a backbone network in the missing detection model; Using the feature fusion module in the missing detection model, the features of the multiple different scales are fused to obtain a fused feature; The prediction module in the missing detection model is used to obtain a prediction result corresponding to the fusion feature; wherein the prediction result is used to characterize the position information of the missing area of the diversion area.

6. The method according to claim 5, wherein: The extracting multiple features of different scales from the second road surface image using the backbone network in the missing detection model includes: Inputting the basic features in the second road surface image into a preset convolution module in the backbone network; Based on the basic features, a depthwise separable convolution and a first conventional convolution are sequentially performed to obtain a first deep feature; wherein a convolution kernel size of the depthwise separable convolution is larger than a predetermined size; Based on the basic features, sequentially performing a second conventional convolution, the depthwise separable convolution, and a third conventional convolution to obtain a second deep feature; The first deep features and the second deep features are fused to obtain the output features of the preset convolution module.

7. The method according to claim 5 or 6, wherein: The method of using the feature fusion module in the missing detection model to fuse the features of the multiple scales to obtain a fused feature includes: Inputting the first scale feature, the second scale feature, and the third scale feature output by the backbone network into the feature fusion module; In the feature fusion module, an intermediate feature is obtained based on the third scale feature and the second scale feature, and the intermediate feature and the first scale feature are fused to obtain a first output feature in the fused feature.

8. The method according to claim 7, wherein: The step of fusing the features of the plurality of different scales using a feature fusion module in the missing detection model to obtain a fused feature further includes: In the feature fusion module, the first output feature, the second scale feature, and the intermediate feature in the fused feature are fused to obtain the second output feature in the fused feature.

9. The method according to claim 8, wherein The step of fusing the features of the plurality of different scales using a feature fusion module in the missing detection model to obtain a fused feature further includes: In the feature fusion module, the second output feature in the fused feature and the third scale feature are fused to obtain the third output feature in the fused feature.

10. A training device for a missing detection model, comprising: a road segment determination module, configured to determine a target road segment combination based on road segment data in a high-precision map; wherein the target road segment combination includes two road segments, the two road segments are connected to the same road segment at a first location, and the high-precision map does not include a diversion area at the first location; An image acquisition module, configured to obtain a first road surface image corresponding to the target road section combination based on the image data of the high-precision map; a marking module, configured to mark the diversion area missing region in the first road surface image, and use the first road surface image and the diversion area missing region as sample data; A training module, configured to train a missing detection model based on the sample data; Wherein, the road section determination module includes: a determining unit, configured to determine a first road segment based on a road segment information table of a high-precision map; wherein the first road segment connects a plurality of other road segments at the same location; a screening unit, configured to determine, if the first road section is not connected to the diversion area, two road sections that meet the conditions for establishing the diversion area from the plurality of other road sections; A combining unit, configured to obtain a target road section combination based on the two road sections meeting the diversion area establishment conditions; Wherein, the screening unit is used for: In a case where the first road segment is not connected to a diversion area, determining a distance between a first lane marking of a second road segment among the multiple other road segments and a last lane marking of a third road segment among the multiple other road segments; When the distance between the first lane marking of the second road section and the last lane marking of the third road section is not greater than a first value, determining a distance between the first lane marking and the guardrail and a distance between the last lane marking and the guardrail; When the distance between the first lane line and the guardrail or the distance between the last lane line and the guardrail is not less than a second value, it is determined that the second road section and the third road section meet the conditions for establishing a diversion area.

11. The device according to claim 10, wherein The image acquisition module includes: a rendering unit, configured to obtain rendering data of the high-precision map based on a plurality of map elements in the high-precision map, wherein the plurality of map elements include a road surface, a diversion area, and a guardrail; The interception unit is configured to obtain a first road surface image corresponding to the target road segment combination based on the rendering image data.

12. The device according to claim 11, wherein The device further includes a color setting module, wherein the color setting module is configured to: Setting the color of the road surface in the rendering data to a first color; Setting the color of the diversion area in the rendering data to a second color; Setting the color of the guardrail in the rendering data to a third color; The color difference between any two of the first color, the second color, and the third color is greater than a third value.

13. A device for detecting the absence of a diversion area, comprising: An image determination module, configured to obtain a second road surface image to be detected based on image data of the high-precision map; A detection module is used to detect the missing area of the diversion area in the second road surface image using a missing detection model; wherein the missing detection model is trained based on the method according to any one of claims 1 to 3.

14. The device according to claim 13, wherein The detection module includes: a feature extraction unit, configured to extract features of multiple scales in the second road surface image using a backbone network in the missing detection model; a fusion unit, configured to fuse the features of the plurality of different scales using a feature fusion module in the missing detection model to obtain a fused feature; A prediction unit is used to obtain a prediction result corresponding to the fusion feature by using a prediction module in the missing detection model; wherein the prediction result is used to characterize the position information of the missing area of the diversion area.

15. The device according to claim 14, wherein The feature extraction unit is used for: Inputting the basic features in the second road surface image into a preset convolution module in the backbone network; Based on the basic features, a depthwise separable convolution and a first conventional convolution are sequentially performed to obtain a first deep feature; wherein a convolution kernel size of the depthwise separable convolution is larger than a predetermined size; Based on the basic features, sequentially performing a second conventional convolution, the depthwise separable convolution, and a third conventional convolution to obtain a second deep feature; The first deep features and the second deep features are fused to obtain the output features of the preset convolution module.

16. The device according to claim 14 or 15, wherein The fusion unit is used for: Inputting the first scale feature, the second scale feature, and the third scale feature output by the backbone network into the feature fusion module; In the feature fusion module, an intermediate feature is obtained based on the third scale feature and the second scale feature, and the intermediate feature and the first scale feature are fused to obtain a first output feature in the fused feature.

17. The device according to claim 16, wherein The fusion unit is further configured to: In the feature fusion module, the first output feature, the second scale feature, and the intermediate feature in the fused feature are fused to obtain the second output feature in the fused feature.

18. The device according to claim 17, wherein The fusion unit is further configured to: In the feature fusion module, the second output feature in the fused feature and the third scale feature are fused to obtain the third output feature in the fused feature.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images