Road edge detection method and device, electronic equipment and readable medium

Abstract

The invention relates to a road edge detection method and device, electronic equipment and a readable medium. The method comprises the following steps: acquiring front image data acquired by front imaging equipment in a vehicle driving process; judging whether the vehicle is within the range of the intersection or not according to the recognition of the front image data; and if the vehicle is within the range of the intersection, performing road edge detection by using right image data acquired by right imaging equipment of the vehicle. According to the scheme provided by the invention, the road edge precision in the intersection range can be improved, manual dotting is not needed, and the labor cost is reduced.

Description

Technical Field

[0001] This application relates to the technical field of high-precision maps, and particularly to a road edge detection method, apparatus, electronic device, and readable medium. Background Art

[0002] Since the traffic conditions within the intersection range of a road are very complex and there are no clear lane lines to assist traffic driving, the road edge within the intersection range becomes the only factor restricting driving. As the only element in the high-precision map that reflects the real situation within the intersection range, the accuracy of the road edge is crucial.

[0003] In the related art, the road edge within the intersection range is determined by identifying the drivable area through a front imaging device. However, the road edge within the intersection range often exceeds the field of view of the front imaging device, resulting in the road edge not being detected. Often, it is necessary to manually use a handheld rtk (real-time kinematic) to mark points to supplement the accuracy of the road edge, which undoubtedly incurs a huge amount of labor costs. Summary of the Invention

[0004] To solve or partially solve the problems existing in the related art, this application provides a road edge detection method, apparatus, electronic device, and readable medium, which can improve the accuracy of the road edge within the intersection range and do not require manual marking, thereby reducing labor costs.

[0005] In a first aspect of this application, a road edge detection method is provided, and the method includes: Obtain the front image data collected by a front imaging device during the driving of the vehicle; Judge whether the vehicle is within the intersection range according to the recognition of the front image data; If the vehicle is within the intersection range, use the right image data collected by the right imaging device of the vehicle for road edge detection.

[0006] In an embodiment, the method further includes: If the vehicle is not within the intersection range, use the front image data collected by the front imaging device of the vehicle and / or the left image data collected by the left imaging device of the vehicle for road edge detection.

[0007] In an embodiment, the judging whether the vehicle is within the intersection range according to the recognition of the front image data includes: Recognize the road elements in the front image data; When the road elements include a stop line and a zebra crossing, extract a first lane line group in a first time period and a second lane line group in a second time period from the road elements; the first time period and the second time period are adjacent time periods before and after. Determine whether the first lane line group is in a disappearing state and determine whether the second lane line group is in a reappearing state. If the first lane line group is in a disappearing state and the second lane line group is in a reappearing state, it is determined that the vehicle is within the intersection range.

[0008] In an embodiment, if the vehicle is within the intersection range, road edge detection is performed using the right image data collected by the right imaging device of the vehicle, including: If the vehicle is within the intersection range, obtain a first position where the vehicle enters the intersection range and obtain a second position where the vehicle exits the intersection range. Determine the area between the first position and the second position as the first driving area of the vehicle within the intersection range. Obtain the right image data collected by the right imaging device of the vehicle within the first driving area. Input the right image data into a pre-trained semantic segmentation model to identify the position information of the road edge in the right image data through the semantic segmentation model; and, Input the right image data into a pre-trained object detection model to identify the type information of the road edge in the right image data through the object detection model.

[0009] In an embodiment, the first lane line group includes at least one lane line, the front image data includes N image frames in the first time period, and N is an integer greater than 2; the determination of whether the first lane line group is in a disappearing state includes: In the N image frames, when it is recognized that the first i image frames contain the lane line and the subsequent j image frames do not contain the lane line, it is determined that the first lane line group is in a disappearing state; where i and j are both positive integers, and i < j. The obtaining of the first position where the vehicle enters the intersection range includes: If the first lane line group is in a disappearing state, it is determined that the vehicle enters the intersection range. Obtain the first timestamp corresponding to the first image frame among the N image frames. Take the vehicle position corresponding to the first timestamp as the first position where the vehicle enters the intersection range.

[0010] In one embodiment, the second lane line group includes at least one lane line, and the front image data further includes M image frames in the second time period, where M is an integer greater than 2; determining whether the second lane line group is in a reproduced state includes: In the M image frames, when it is recognized that the first x image frames do not include the lane line and the subsequent y image frames include the lane line, it is determined that the second lane line group is in a reproduced state; where x and y are both positive integers, and x > y; Obtaining the second position where the vehicle exits the intersection range includes: If the second lane line group is in a reproduced state, it is determined that the vehicle has exited the intersection range; Obtaining the second timestamp corresponding to the image frame at the end in the M image frames; Taking the vehicle position corresponding to the second timestamp as the second position where the vehicle exits the intersection range.

[0011] In one embodiment, the method further includes: Obtaining an identification result; the identification result includes the position information and type information of the road edge, and the position information is the pixel coordinate position of the road edge; Performing pixel coordinate to longitude and latitude calculation on the pixel coordinate position of the road edge to obtain the actual coordinate position of the road edge; Updating the high-precision map using the actual coordinate position and type information of the road edge.

[0012] A second aspect of the present application provides a road edge detection device, and the device includes: A front image data acquisition module, configured to acquire front image data collected by a front imaging device during vehicle driving; An intersection range determination module, configured to determine whether the vehicle is within the intersection range according to the recognition of the front image data; A first detection module, configured to perform road edge detection using the right image data collected by the right imaging device of the vehicle if the vehicle is within the intersection range.

[0013] A third aspect of the present application provides an electronic device, including: A processor; and A memory, on which executable code is stored, and when the executable code is executed by the processor, the processor is caused to execute the method as described above.

[0014] The fourth aspect of this application provides a computer-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method described above.

[0015] The technical solution provided by this application may include the following beneficial effects: The solution provided by this application obtains the front image data collected by the front imaging device during the vehicle driving process; determines whether the vehicle is within the range of an intersection according to the recognition of the front image data; if the vehicle is within the range of the intersection, the right image data collected by the right imaging device of the vehicle is used for road edge detection. By using the right image data collected by the right imaging device to detect the road edge within the range of the intersection, since the road edge within the range of the intersection does not exceed the field of view of the right imaging device, the accuracy of the road edge within the range of the intersection can be improved, and there is no need for manual marking, reducing the labor cost.

[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By describing the exemplary embodiments of this application in more detail in conjunction with the drawings, the above and other objects, features, and advantages of this application will become more obvious. Among them, in the exemplary embodiments of this application, the same reference numerals generally represent the same components.

[0018] Figure 1 is a flowchart of the road edge detection method shown in the embodiments of this application; Figure 2 is another flowchart of the road edge detection method shown in the embodiments of this application; Figure 3 is a schematic diagram of the range of a cross-shaped intersection shown in the embodiments of this application; Figure 4 is a schematic diagram of the structure of the road edge detection device shown in the embodiments of this application; Figure 5 is a schematic diagram of the structure of the electronic device shown in the embodiments of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The embodiments of this application will be described in more detail below with reference to the drawings. Although the embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0020] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, the meaning of "a plurality" is two or more unless otherwise specifically defined.

[0022] In the related art, the road edge is determined by a front imaging device to identify the drivable area within the intersection range. However, there are the following problems in detecting the road edge within the intersection range by the front imaging device: 1) For the scenario where the right side of the road is a non-motor vehicle lane, the road edge within the intersection range will exceed the field of view of the front imaging device, resulting in the road edge not being detectable.

[0023] 2) For the scenario with a feeder road, the road edge within the intersection range is located at the boundary of the field of view of the front imaging device, resulting in a large error in the calculation accuracy of the road edge.

[0024] 3) For the situation where the position of the road edge cannot be calculated, manual marking is required. For example, it is necessary to manually use a handheld rtk for marking to supplement the accuracy of the road edge, which will undoubtedly incur a huge amount of labor costs.

[0025] In view of the above problems, the embodiment of this application provides a road edge detection method, which detects the road edge within the intersection range by using the right image data collected by the right imaging device. Since the road edge within the intersection range does not exceed the field of view of the right imaging device, the accuracy of the road edge within the intersection range can be improved, and manual marking is not required, reducing labor costs.

[0026] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0027] Figure 1 It is a schematic flow chart of the road edge detection method shown in the embodiment of this application.

[0028] SeeFigure 1 , the road edge detection method of the present application includes: S110, obtaining the front image data collected by the front imaging device during the vehicle driving process.

[0029] In the embodiment of the present application, imaging devices for image data collection can be respectively configured at different positions of the vehicle. For example, a front imaging device can be configured in front of the center of the roof, a left imaging device can be configured on the left side of the B-pillar of the roof, and a right imaging device can be configured on the right side of the B-pillar of the roof. In addition, imaging devices can also be configured at other positions of the vehicle according to actual needs. For example, a rear imaging device can also be configured behind the center of the roof. The embodiment of the present application does not make any limitation on this.

[0030] In the embodiment of the present application, the imaging devices of the vehicle at least include a front imaging device, a left imaging device, and a right imaging device.

[0031] It should be noted that the B-pillar refers to the vertical column between the front door and the rear door of the vehicle, extending from the roof to the bottom of the vehicle.

[0032] During the vehicle driving process, the front imaging device, the left imaging device, and the right imaging device respectively collect image data of different perspectives of the vehicle. Among them, the front imaging device collects the image data of the front perspective of the vehicle, the left imaging device collects the image data of the left perspective of the vehicle, and the right imaging device collects the image data of the right perspective of the vehicle. For the convenience of description, the image data collected by the front imaging device can be called the front image data, the image data collected by the left imaging device can be called the left image data, and the image data collected by the right imaging device can be called the right image data.

[0033] Among them, the front image data can be used to reflect the road conditions of the front perspective of the vehicle, the left image data can be used to reflect the road conditions of the left perspective of the vehicle, and the right image data can be used to reflect the road conditions of the right perspective of the vehicle. Therefore, the embodiment of the present application can obtain the front image data collected by the front imaging device so as to determine the road conditions of the front perspective of the vehicle through the front image data.

[0034] S120, judging whether the vehicle is within the range of the intersection according to the recognition of the front image data.

[0035] Embodiments of the present application can utilize AI (Artificial Intelligence) technology to identify road elements contained in the front image data. The road elements can include at least one of indicative markings, prohibitory markings, and warning markings. Among them, the indicative markings can include at least one of stop lines, zebra crossings, and lane lines. The prohibitory markings can include at least one of no-overtaking lines, no-parking lines, and mesh lines. The warning markings can include at least one of deceleration reminder lines, diversion lines, and elevation markings.

[0036] Embodiments of the present application can determine whether the vehicle is within the range of an intersection based on the road elements contained in the identified front image data. Among them, an intersection refers to the place where two or more roads intersect, which is a key node for the convergence and turning of traffic flow. The shape of the intersection can include at least one of cross-shaped, grid-shaped, T-shaped, and Y-shaped.

[0037] S130, if the vehicle is within the range of the intersection, then the right image data collected by the right imaging device of the vehicle is used for road edge detection.

[0038] Generally, a complete road at least includes a left road and a right road. According to the driving direction of the vehicle, the vehicle generally drives on the right road. Therefore, the left road edge of the vehicle is generally the dividing line between the left road and the right road. For the range of the intersection, the right road edge can be of types such as curbstones and guardrails, which can be used to define the physical boundary of the intersection range. While the left road edge is generally classified as a green belt type. Therefore, in the embodiments of the present application, only the right road edge of the vehicle needs to be concerned within the range of the intersection, and there is no need to concern the left road edge of the vehicle.

[0039] When the vehicle is driving within the range of the intersection, the right road of the vehicle is generally a non-motor vehicle lane or a service road. Therefore, the right road edge of the vehicle often exceeds the field of view of the front imaging device and the left imaging device, but does not exceed the field of view of the right imaging device. Therefore, when it is determined that the vehicle is within the range of the intersection, the embodiments of the present application can obtain the right image data collected by the right imaging device to detect the right road edge within the range of the intersection using the right image data.

[0040] As can be seen from this example, the solution provided by this application is to obtain the front image data collected by the front imaging device during the vehicle's driving; based on the recognition of the front image data, determine whether the vehicle is within the range of an intersection; if the vehicle is within the range of the intersection, use the right-side image data collected by the right-side imaging device of the vehicle to perform road edge detection. By using the right-side image data collected by the right-side imaging device to detect the road edge within the range of the intersection, since the road edge within the range of the intersection does not exceed the field of view of the right-side imaging device, the accuracy of the road edge within the range of the intersection can be improved, and there is no need for manual marking, reducing labor costs.

[0041] Figure 2 It is another schematic flowchart of the road edge detection method shown in the embodiments of this application.

[0042] See Figure 2 , the road edge detection method of this application includes: S210, obtain the front image data collected by the front imaging device during the vehicle's driving.

[0043] This step can refer to the description in S110 and will not be elaborated here.

[0044] S220, based on the recognition of the front image data, determine whether the vehicle is within the range of an intersection.

[0045] This step can refer to the description in S120 and will not be elaborated here.

[0046] In an embodiment, based on the recognition of the front image data, determining whether the vehicle is within the range of an intersection may include: Recognize the road elements in the front image data; when the road elements include a stop line and a zebra crossing, extract the first lane line group in the first time period and the second lane line group in the second time period from the road elements; the first time period and the second time period are adjacent time periods; determine whether the first lane line group is in a disappearing state and determine whether the second lane line group is in a reappearing state; if the first lane line group is in a disappearing state and the second lane line group is in a reappearing state, it is determined that the vehicle is within the range of the intersection.

[0047] After obtaining the front image data collected by the front imaging device, the embodiments of this application can use AI technology to recognize the road elements included in the front image data. When the road elements include a stop line and a zebra crossing, the vehicle may start to enter the range of the intersection. However, the stop line and the zebra crossing exist not only within the range of the intersection but also within the non-intersection range. Therefore, the embodiments of this application need to make a further judgment.

[0048] It should be noted that the types of stop lines may include at least one of ordinary stop lines, stop-and-go lines, yield lines, and tidal lane lines.

[0049] When the road elements include a stop line and a zebra crossing, the embodiment of the present application can further extract a first lane line group in a first time period and a second lane line group in a second time period from the road elements. Among them, both the first lane line and the second lane line include at least one lane line. Since the first time period and the second time period are adjacent time periods, the lane lines included in the first lane line and the second lane line are continuous. The embodiment of the present application can determine whether the first lane line group is in a disappearing state and determine whether the second lane line group is in a reappearing state. Among them, the disappearing state may refer to the state where the lane line appears first and then disappears, and the reappearing state may refer to the state where the lane line disappears first and then appears. Since there are no clear lane lines to assist traffic driving within the intersection range, if the first lane line group is in a disappearing state and the second lane line group is in a reappearing state, it means that the vehicle first enters the intersection range and then exits the intersection range. Therefore, it can be determined that the vehicle is located within the intersection range.

[0050] In one embodiment, the first lane line group includes at least one lane line, and the front image data includes N image frames in the first time period, where N is an integer greater than 2; determining whether the first lane line group is in a disappearing state may include: In the N image frames, when it is recognized that the first i image frames include lane lines and the subsequent j image frames do not include lane lines, it is determined that the first lane line group is in a disappearing state; where both i and j are positive integers, and i < j.

[0051] Among them, the first lane line may include at least one lane line, and the front image data may include N consecutive image frames in the first time period, where N > 2 and N is an integer.

[0052] The embodiment of the present application can respectively recognize N consecutive image frames. If it is recognized that the first i image frames include lane lines and the subsequent j image frames do not include lane lines, it means that the lane lines in the N image frames appear first and then disappear. Therefore, it can be determined that the first lane line group is in a disappearing state, and further it can be determined that the vehicle enters the intersection range. Where both i and j are positive integers, and i + j = N, i < j.

[0053] In one example, assuming N is 10 and i = 1, then j = 9. The embodiment of the present application can respectively recognize 10 consecutive image frames. If a lane line is recognized in the previous image frame and the lane line disappears in the subsequent 9 consecutive image frames, it means that the entire group of lane lines in the 10 image frames disappears. Therefore, it can be determined that the first lane line group is in a disappearing state, and further it can be determined that the vehicle enters the intersection range.

[0054] In one embodiment, the second lane line group includes at least one lane line, and the front image data further includes M image frames in a second time period, where M is an integer greater than 2; determining whether the second lane line group is in a reproduced state may include: In the M image frames, when it is recognized that the first x image frames do not include lane lines and the subsequent y image frames include lane lines, it is determined that the second lane line group is in a reproduced state; where x and y are both positive integers, and x > y.

[0055] Wherein, the second lane line may include at least one lane line, and the front image data may include consecutive M image frames in a second time period, M > 2 and M is an integer.

[0056] In the embodiments of the present application, consecutive M image frames can be respectively recognized. If it is recognized that the first x image frames do not include lane lines and the subsequent y image frames include lane lines, it indicates that the lane lines in the M image frames disappear first and then appear. Therefore, it can be determined that the second lane line group is in a reproduced state, and further it can be determined that the vehicle has driven out of the intersection range. Where x and y are both positive integers, and x + y = M, x > y.

[0057] In another example, assuming M is 10 and x = 9, then y = 1. In the embodiments of the present application, consecutive 10 image frames can be respectively recognized. If no lane lines are recognized in the first consecutive 9 image frames and lane lines are re-recognized in the subsequent 1 image frame, it indicates that the entire group of lane lines in the 10 image frames appears. Therefore, it can be determined that the second lane line group is in a reproduced state, and further it can be determined that the vehicle has driven out of the intersection range.

[0058] It should be noted that if it is determined that the vehicle is within the intersection range, step S230 can be entered; if it is determined that the vehicle is not within the intersection range, step S240 can be entered.

[0059] S230, if the vehicle is within the intersection range, the right image data collected by the right imaging device of the vehicle is used for road edge detection.

[0060] This step can refer to the description in S130 and will not be elaborated here.

[0061] In one embodiment, if the vehicle is within the intersection range, using the right image data collected by the right imaging device of the vehicle for road edge detection may include: If the vehicle is within the intersection range, a first position of the vehicle entering the intersection range is obtained, and a second position of the vehicle exiting the intersection range is obtained; the area between the first position and the second position is determined as a first driving area of ​​the vehicle within the intersection range; right-side image data collected by a right-side imaging device of the vehicle within the first driving area is obtained; the right-side image data is input into a pre-trained semantic segmentation model to identify the position information of the road edge in the right-side image data through the semantic segmentation model; and the right-side image data is input into a pre-trained target detection model to identify the type information of the road edge in the right-side image data through the target detection model.

[0062] If it is determined that the vehicle is within the intersection range, the embodiment of the present application can obtain the first position of the vehicle entering the intersection range, and obtain the second position of the vehicle exiting the intersection range. The first position and the second position both belong to the intersection range, so the area between the first position and the second position can constitute the first driving area of ​​the vehicle within the intersection range, so as to obtain the right-side image data collected by the right-side imaging device of the vehicle within the first driving area.

[0063] The embodiment of the present application can pre-train two models: a semantic segmentation model and a target detection model. Among them, the semantic segmentation model aims to divide each pixel in the image frame into a corresponding semantic category to achieve fine segmentation at the pixel level. The target detection model aims to automatically identify objects in the image frame through computer algorithms and determine their categories.

[0064] In actual applications, the embodiments of the present application can directly call pre-trained semantic segmentation models and target detection models. The right image data may include multiple continuous image frames. The embodiments of the present application can input the multiple continuous image frames into the semantic segmentation model in sequence so as to identify the position information of the road edge in each image frame of the right image data through the semantic segmentation model, and can input the multiple continuous image frames into the target detection model in sequence so as to identify the type information of the road edge in each image frame of the right image data through the target detection model.

[0065] It should be noted that since the right image data only includes the right road edge of the vehicle, and the embodiment of the present application does not focus on the left road edge within the intersection range, the recognition result output by the semantic segmentation model for the intersection range scene is the position information of the right road edge of the vehicle within the intersection range, and the recognition result output by the target detection model for the intersection range scene is the type information of the right road edge of the vehicle within the intersection range.

[0066] In one implementation, obtaining the first position of the vehicle entering the intersection range may include: If the first lane line group is in a disappearing state, it is determined that the vehicle has entered the intersection range; obtain the first timestamp corresponding to the first image frame among N image frames; use the vehicle position corresponding to the first timestamp as the first position where the vehicle enters the intersection range.

[0067] If the first lane line group is in a disappearing state, that is, the lane lines in the N image frames appear first and then disappear, it can be determined that the vehicle has entered the intersection range. Among them, each image frame in the N image frames has a corresponding timestamp. Since the timestamp corresponding to the first image frame among the N image frames can relatively accurately reflect the moment when the vehicle just enters the intersection range, the embodiments of the present application can obtain the timestamp corresponding to the first image frame among the N image frames. For the convenience of description, the timestamp corresponding to the first image frame is defined as the first timestamp. The embodiments of the present application can use the vehicle position corresponding to the first timestamp as the first position where the vehicle just enters the intersection range.

[0068] In one embodiment, obtaining the second position where the vehicle exits the intersection range may include: If the second lane line group is in a reappearing state, it is determined that the vehicle has exited the intersection range; obtain the second timestamp corresponding to the last image frame among M image frames; use the vehicle position corresponding to the second timestamp as the second position where the vehicle exits the intersection range.

[0069] If the second lane line group is in a reappearing state, that is, the lane lines in the M image frames disappear first and then appear, it can be determined that the vehicle has exited the intersection range. Among them, each image frame in the M image frames has a corresponding timestamp. Since the timestamp corresponding to the last image frame among the M image frames can relatively accurately reflect the moment when the vehicle just exits the intersection range, the embodiments of the present application can obtain the timestamp corresponding to the last image frame among the M image frames. For the convenience of description, the timestamp corresponding to the last image frame is defined as the second timestamp. The embodiments of the present application can use the vehicle position corresponding to the second timestamp as the second position where the vehicle just exits the intersection range.

[0070] In one embodiment, before obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area, the method may further include: Obtain the driving trajectory of the vehicle in the first driving area; if the driving trajectory is a right-turn trajectory, then perform the step of obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area; or, if the driving trajectory is a straight-ahead trajectory, then when the distance between the right-side imaging device of the vehicle and the right-side road edge is greater than or equal to a preset distance threshold, perform the step of obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area; or, if the driving trajectory is a left-turn trajectory, then do not perform the step of obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area.

[0071] Before obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area, the embodiments of the present application may obtain the driving trajectory of the vehicle in the first driving area, so as to determine whether to perform the above-mentioned step of obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area based on this driving trajectory.

[0072] In one example, as Figure 3 shown, the vehicle is currently driving on Road A within the range of an intersection. If the driving trajectory is a right-turn trajectory, it means that the vehicle turns right from Road A to Road B within the intersection range. It can be seen that when the vehicle turns right, the right-side road edge of the vehicle will not exceed the field of view of the right-side imaging device, and the vehicle can meet the dynamic curvature during the turning process, so that the distance between the right-side imaging device and the right-side road edge is greater than or equal to the preset distance threshold. Therefore, the embodiments of the present application may perform the above-mentioned step of obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area.

[0073] In another example, as Figure 3 shown, the vehicle is currently driving on Road A within the range of an intersection. If the driving trajectory is a straight-ahead trajectory, it means that the vehicle goes straight from Road A to Road C within the intersection range. Since the imaging device is configured at the roof position, a certain distance needs to be maintained between the imaging device and the road edge so that the imaging device can capture the road edge. Therefore, when the vehicle is going straight, the distance between the right-side imaging device of the vehicle and the right-side road edge can be obtained. If this distance is greater than or equal to the preset distance threshold, it means that the right-side imaging device can capture the right-side road edge at this time. Therefore, the embodiments of the present application may perform the above-mentioned step of obtaining the right-side image data collected by the right-side imaging device of the vehicle in the first driving area.

[0074] In yet another example, as Figure 3As shown in the figure, the vehicle is currently driving on Road A within the intersection range. If the driving trajectory is a left-turn trajectory, it means that the vehicle turns left from Road A to Road D within the intersection range. It can be seen that when the vehicle turns left, the right road edge of the vehicle will exceed the field of view of the right imaging device. Therefore, in the embodiments of the present application, the step of obtaining the right image data collected by the right imaging device of the vehicle in the first driving area may not be executed.

[0075] Therefore, when the vehicle is driving within the intersection range, the above step of obtaining the right image data collected by the right imaging device of the vehicle in the first driving area is only executed when the vehicle turns right, or when the vehicle goes straight and the distance between the right imaging device of the vehicle and the right road edge is greater than or equal to the preset distance threshold.

[0076] It should be noted that the embodiments of the present application can only focus on the right road edge within the intersection range and do not need to focus on the left road edge within the intersection range.

[0077] It should be noted that the embodiments of the present application can set the preset distance threshold according to the actual situation. For example, the preset distance threshold can be set to 4m, or set to other values. The embodiments of the present application do not make any limitations in this regard.

[0078] S240. If the vehicle is not within the intersection range, the front image data collected by the front imaging device of the vehicle and / or the left image data collected by the left imaging device are used for road edge detection.

[0079] When the vehicle is not within the intersection range, it is easy to approach the right road edge, resulting in the distance between the right imaging device of the vehicle and the right road edge being less than the preset distance threshold, so that the right imaging device cannot collect the right road edge. Therefore, in the embodiments of the present application, the right imaging device does not need to be used within the non-intersection range, and only the front imaging device and the left imaging device need to be used.

[0080] Specifically, since the traffic conditions within the non-intersection range are relatively simple, the left road edge of the vehicle will not exceed the field of view of the left imaging device, nor will it exceed the field of view of the front imaging device. Therefore, when it is determined that the vehicle is not within the intersection range, that is, when it is determined that the vehicle is driving within the non-intersection range, the embodiments of the present application can obtain the front image data collected by the front imaging device or the left image data collected by the left imaging device, so as to detect the left road edge within the non-intersection range by using the front image data and / or the left image data.

[0081] That is to say, within the non-intersection range, the front imaging device can be used alone, or the left imaging device can be used alone. If the front imaging device fails to collect the left road edge within the non-intersection range, the left imaging device can be used for supplementation.

[0082] In one embodiment, if the vehicle is not within the intersection range, the front image data collected by the front imaging device of the vehicle and / or the left image data collected by the left imaging device can be used for road edge detection, which may include: If the vehicle is not within the intersection range, determine the second driving area of the vehicle within the non-intersection range; obtain the front image data collected by the front imaging device of the vehicle within the second driving area, and / or obtain the left image data collected by the left imaging device of the vehicle within the second driving area; input the front image data and / or the left image data into a pre-trained semantic segmentation model to identify the position information of the road edge in the front image data and / or the left image data through the semantic segmentation model; and input the front image data and / or the left image data into a pre-trained object detection model to identify the type information of the road edge in the front image data and / or the left image data through the object detection model.

[0083] If it is determined that the vehicle is within the non-intersection range, the embodiments of the present application can determine the second driving area of the vehicle within the non-intersection range. Specifically, the total driving area of the vehicle can be obtained. The total driving area can refer to the entire driving area of the vehicle from the starting point to the ending point. Among them, the total driving area is composed of the first driving area of the vehicle within the intersection range and the second driving area within the non-intersection range. Therefore, the second driving area can be obtained by calculating the difference between the total driving area and the first driving area, so as to obtain the front image data collected by the front imaging device of the vehicle within the second driving area, and / or obtain the left image data collected by the left imaging device of the vehicle within the second driving area.

[0084] In one example, the embodiments of the present application can also call a pre-trained semantic segmentation model and an object detection model. The front image data can include a continuous multi-frame image sequence. The embodiments of the present application can sequentially input the continuous multi-frame image sequence into the semantic segmentation model to identify the position information of the road edge in each frame of the front image data through the semantic segmentation model, and can sequentially input the continuous multi-frame image sequence into the object detection model to identify the type information of the road edge in each frame of the front image data through the object detection model.

[0085] In another example, the embodiments of the present application can also call a pre-trained semantic segmentation model and an object detection model. The left image data can include a series of consecutive image frames. The embodiments of the present application can sequentially input the series of consecutive image frames into the semantic segmentation model to identify the position information of the road edge in each image frame of the left image data through the semantic segmentation model, and can sequentially input the series of consecutive image frames into the object detection model to identify the type information of the road edge in each image frame of the left image data through the object detection model.

[0086] It should be noted that the embodiments of the present application do not pay attention to the right road edge within the non-intersection range. Therefore, the recognition result output by the semantic segmentation model for the non-intersection range scenario is the position information of the left road edge of the vehicle within the non-intersection range, and the recognition result output by the object detection model for the non-intersection range scenario is the type information of the left road edge of the vehicle within the non-intersection range.

[0087] In one embodiment, the method may further include: Obtain the recognition result; the recognition result includes the position information of the road edge and the type information of the road edge, and the position information is the pixel coordinate position of the road edge; perform pixel point longitude and latitude calculation on the pixel coordinate position of the road edge to obtain the actual coordinate position of the road edge; update the high-precision map using the actual coordinate position of the road edge and the type information of the road edge.

[0088] The embodiments of the present application can obtain the recognition results output for the intersection range scenario and the recognition results output for the non-intersection range scenario from the semantic segmentation model, and the embodiments of the present application can obtain the recognition results output for the intersection range scenario and the recognition results output for the non-intersection range scenario from the object detection model. Among them, the recognition result output for the intersection range scenario may include the position information of the right road edge and the type information of the right road edge, and the recognition result output for the non-intersection range scenario may include the position information of the left road edge and the type information of the left road edge.

[0089] Since the position information is the pixel coordinate position of the road edge in the image data. For example, the position information of the right road edge is the pixel coordinate position of the right road edge in the right image data, and the position information of the left road edge is the pixel coordinate position of the left road edge in the front image data, or the pixel coordinate position of the left road edge in the left image data. In this regard, the embodiments of the present application can perform pixel point longitude and latitude calculation on the pixel coordinate position of the road edge to obtain the actual coordinate position of the road edge.

[0090] In a specific implementation, the embodiment of the present application extracts target image frames capable of recognizing road edges from multiple image frames, then reads the target timestamps corresponding to the target image frames, and then searches for the vehicle position and vehicle pose under the corresponding hardware of the target timestamps, so as to calculate the actual coordinate positions of the road edges by combining the internal and external parameters of the imaging device with the corresponding vehicle position and vehicle pose. Among them, the actual coordinate positions of the road edges can be evenly spaced continuous points in the format of shp (shapefile, vector data storage format).

[0091] For the intersection range scenario, the embodiment of the present application can update the intersection range and isolation facility elements in the high-precision map with the actual coordinate positions of the right road edge and the type information of the right road edge (such as curb, guardrail, etc.).

[0092] For the non-intersection range scenario, the embodiment of the present application can update the non-intersection range and isolation facility elements in the high-precision map with the actual coordinate positions of the left road edge and the type information of the left road edge (such as lane demarcation lines).

[0093] From this example, it can be seen that the solution provided by the present application obtains the front image data collected by the front imaging device during the vehicle driving; determines whether the vehicle is within the intersection range according to the recognition of the front image data; if the vehicle is within the intersection range, the right image data collected by the right imaging device of the vehicle is used for road edge detection; if the vehicle is not within the intersection range, the front image data collected by the front imaging device of the vehicle and / or the left image data collected by the left imaging device of the vehicle are used for road edge detection. The present application improves the accuracy of the road edge within the intersection range by using the right image data collected by the right imaging device to detect the road edge within the intersection range. Since the road edge within the intersection range does not exceed the field of view of the right imaging device, manual marking is not required, reducing labor costs. Further, the present application improves the accuracy of the road edge within the non-intersection range by using the front image data collected by the front imaging device and / or the left image data collected by the left imaging device to detect the road edge within the non-intersection range. Since the road edge within the non-intersection range does not exceed the field of view of the front imaging device and the left imaging device, manual marking is not required, reducing labor costs.

[0094] Further, the present application uses road elements such as stop lines, zebra crossings, and lane lines recognized by AI technology to comprehensively judge the intersection range, which can improve the judgment accuracy.

[0095] Corresponding to the foregoing embodiments of the application function implementation method, the present application further provides a road edge detection device, an electronic device, a computer-readable storage medium, and corresponding embodiments.

[0096] Figure 4 It is a schematic structural diagram of the road edge device shown in the embodiments of the present application.

[0097] See Figure 4 , a road edge device provided by the present application, the device may include: A front image data acquisition module 410, configured to acquire front image data collected by a front imaging device during vehicle driving; An intersection range determination module 420, configured to determine whether the vehicle is within the intersection range according to the recognition of the front image data; A first detection module 430, configured to perform road edge detection using the right image data collected by the right imaging device of the vehicle if the vehicle is within the intersection range.

[0098] In an embodiment, the device may further include: A second detection module, configured to perform road edge detection using the front image data collected by the front imaging device of the vehicle and / or the left image data collected by the left imaging device of the vehicle if the vehicle is not within the intersection range.

[0099] In an embodiment, the intersection range determination module 420 may include: A road element recognition sub-module, configured to recognize road elements in the front image data; A lane line group extraction sub-module, configured to extract a first lane line group in a first time period and a second lane line group in a second time period from the road elements when the road elements include a stop line and a zebra crossing; the first time period and the second time period are adjacent time periods; A state determination sub-module, configured to determine whether the first lane line group is in a disappearing state and determine whether the second lane line group is in a reappearing state; An intersection range determination sub-module, configured to determine that the vehicle is within the intersection range if the first lane line group is in a disappearing state and the second lane line group is in a reappearing state.

[0100] In an embodiment, the first detection module 430 may include: An intersection range boundary position acquisition sub-module, configured to acquire a first position where the vehicle enters the intersection range and a second position where the vehicle exits the intersection range if the vehicle is within the intersection range; The first driving area determination sub-module is configured to determine the area between the first position and the second position as the first driving area of the vehicle within the intersection range; The imaging data acquisition sub-module within the first driving area is configured to acquire the right-side image data collected by the right-side imaging device of the vehicle within the first driving area; The position recognition sub-module of the road edge within the intersection range is configured to input the right-side image data into a pre-trained semantic segmentation model to identify the position information of the road edge in the right-side image data through the semantic segmentation model; and, The type recognition sub-module of the road edge within the intersection range is configured to input the right-side image data into a pre-trained object detection model to identify the type information of the road edge in the right-side image data through the object detection model.

[0101] In one embodiment, the second detection module may include: The second driving area determination sub-module is configured to determine the second driving area of the vehicle outside the intersection range if the vehicle is not within the intersection range; The imaging data acquisition sub-module within the second driving area is configured to acquire the front image data collected by the front imaging device of the vehicle within the second driving area, and / or, acquire the left-side image data collected by the left-side imaging device of the vehicle within the second driving area; The position recognition sub-module of the road edge outside the intersection range is configured to input the front image data and / or the left-side image data into a pre-trained semantic segmentation model to identify the position information of the road edge in the front image data and / or the left-side image data through the semantic segmentation model; and, The type recognition sub-module of the road edge outside the intersection range is configured to input the front image data and / or the left-side image data into a pre-trained object detection model to identify the type information of the road edge in the front image data and / or the left-side image data through the object detection model.

[0102] In one embodiment, the first lane line group includes at least one lane line, the front image data includes N image frames in the first time period, and N is an integer greater than 2; the state judgment sub-module may include: The disappearance state determination unit is configured to determine that the first lane line group is in a disappearance state when, among the N image frames, it is recognized that the first i image frames contain lane lines and the subsequent j image frames do not contain lane lines; where i and j are both positive integers, and i < j; The intersection range boundary position acquisition sub-module may include: The driving into the intersection range determination unit is configured to determine that the vehicle drives into the intersection range if the first lane line group is in a disappearance state; A first timestamp acquisition unit, configured to acquire a first timestamp corresponding to the first image frame among N image frames; A first position definition unit, configured to use the vehicle position corresponding to the first timestamp as the first position where the vehicle enters the intersection range.

[0103] In an embodiment, the second lane line group includes at least one lane line, and the front image data further includes M image frames in a second time period, where M is an integer greater than 2; the state judgment sub-module may include: A reproduction state determination unit, configured to determine that the second lane line group is in a reproduction state when it is recognized that the first x image frames among the M image frames do not include lane lines and the subsequent y image frames include lane lines; where x and y are both positive integers, and x > y; The intersection range boundary position acquisition sub-module may include: An intersection range exit determination unit, configured to determine that the vehicle exits the intersection range if the second lane line group is in a reproduction state; A second timestamp acquisition unit, configured to acquire a second timestamp corresponding to the last image frame among the M image frames; A second position definition unit, configured to use the vehicle position corresponding to the second timestamp as the second position where the vehicle exits the intersection range.

[0104] In an embodiment, the device may further include: An identification result acquisition module, configured to acquire an identification result; the identification result includes the position information and type information of the road edge, and the position information is the pixel coordinate position of the road edge; A pixel point longitude and latitude calculation module, configured to perform pixel point longitude and latitude calculation on the pixel coordinate position of the road edge to obtain the actual coordinate position of the road edge; A high-precision map update module, configured to update the high-precision map by using the actual coordinate position and type information of the road edge.

[0105] It can be seen from this example that the solution provided by this application acquires the front image data collected by the front imaging device during the vehicle driving process; determines whether the vehicle is within the intersection range according to the recognition of the front image data; if the vehicle is within the intersection range, the right image data collected by the right imaging device of the vehicle is used for road edge detection. By using the right image data collected by the right imaging device to detect the road edge within the intersection range, since the road edge within the intersection range does not exceed the field of view of the right imaging device, the accuracy of the road edge within the intersection range can be improved, and manual marking is not required, reducing the labor cost.

[0106] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0107] Figure 5 It is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

[0108] Refer to Figure 5 , the electronic device 500 includes a memory 510 and a processor 520.

[0109] The processor 520 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] The memory 510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, the ROM may store static data or instructions required by the processor 520 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device employs a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all of the instructions and data required by the processor during operation. In addition, the memory 510 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 510 may include a removable storage device that is readable and / or writable, such as a compact disc (CD), read-only digital versatile disc (such as DVD-ROM, dual-layer DVD-ROM), read-only Blu-ray disc, super density disc, flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and instantaneous electronic signals transmitted wirelessly or by wire.

[0111] Executable code is stored on the memory 510, and when the executable code is processed by the processor 520, it can cause the processor 520 to execute some or all of the methods described above.

[0112] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps of the above method according to the present application.

[0113] Alternatively, the present application may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium), on which executable code (or a computer program or computer instruction code) is stored. When the executable code (or the computer program or computer instruction code) is executed by a processor of an electronic device (or a server, etc.), it causes the processor to execute some or all of the steps of the above method according to the present application.

[0114] The embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A road edge detection method, characterized in that: The method comprises: Acquire the front image data collected by the front imaging device during the vehicle's driving; Determining whether the vehicle is located within an intersection range based on the recognition of the front image data; If the vehicle is located within the intersection, the right image data collected by the right imaging device of the vehicle is used to perform road edge detection.

2. The method according to claim 1, characterized in that The method further comprises: If the vehicle is not located within the intersection, road edge detection is performed using front image data captured by the front imaging device of the vehicle and / or left image data captured by the left imaging device.

3. The method according to claim 1, characterized in that The determining whether the vehicle is located within the intersection range according to the recognition of the front image data includes: identifying road elements in the front image data; When the road element includes a stop line and a zebra crossing, extracting a first lane line group in a first time period and a second lane line group in a second time period from the road element; the first time period and the second time period are adjacent time periods; Determining whether the first lane line group is in a disappearing state, and determining whether the second lane line group is in a reappearing state; If the first lane line group is in a disappearing state and the second lane line group is in a reappearing state, it is determined that the vehicle is located within the intersection range.

4. The method according to claim 3, characterized in that: If the vehicle is located within the intersection, the right image data collected by the right imaging device of the vehicle is used to perform road edge detection, including: If the vehicle is located within the intersection range, obtaining a first position of the vehicle entering the intersection range, and obtaining a second position of the vehicle exiting the intersection range; determining an area between the first position and the second position as a first driving area of ​​the vehicle within the intersection; Acquire right side image data collected by a right side imaging device of the vehicle in the first driving area; Inputting the right image data into a pre-trained semantic segmentation model to identify the position information of the road edge in the right image data through the semantic segmentation model; and, The right image data is input into a pre-trained target detection model so as to identify type information of the road edge in the right image data through the target detection model.

5. The method according to claim 4, characterized in that The first lane line group includes at least one lane line, the front image data includes N image frames in the first time period, and N is an integer greater than 2; and determining whether the first lane line group is in a disappeared state includes: In the N frames of image frames, when it is recognized that the first i frames of the image frames contain the lane line and the next j frames of the image frames do not contain the lane line, it is determined that the first lane line group is in a disappeared state; wherein both i and j are positive integers, and i<j; The obtaining of the first position of the vehicle entering the intersection range includes: If the first lane line group is in a disappeared state, it is determined that the vehicle enters the intersection range; Obtain a first timestamp corresponding to a first image frame among the N image frames; The vehicle position corresponding to the first timestamp is used as the first position of the vehicle entering the intersection.

6. The method according to claim 4, characterized in that The second lane line group includes at least one lane line, the front image data also includes M image frames in the second time period, and M is an integer greater than 2; and determining whether the second lane line group is in a reappearing state includes: In the M-frame image frames, when it is recognized that the first x-frame image frames do not include the lane line, and the next y-frame image frames include the lane line, it is determined that the second lane line group is in a reappearing state; wherein both x and y are positive integers, and x>y; The obtaining of the second position of the vehicle when it exits the intersection includes: If the second lane line group is in a reappearing state, determining that the vehicle has driven out of the intersection range; Obtain a second timestamp corresponding to the last image frame among the M frames; The vehicle position corresponding to the second timestamp is used as the second position where the vehicle exits the intersection.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a recognition result; the recognition result includes the position information of the road edge and the type information of the road edge, and the position information is the pixel coordinate position of the road edge; Calculating the longitude and latitude of the pixel coordinates of the road edge to obtain the actual coordinates of the road edge; The high-precision map is updated using the actual coordinate position of the road edge and the type information of the road edge.

8. A road edge detection device, characterized in that: The device comprises: A front image data acquisition module is used to acquire the front image data collected by the front imaging device during the vehicle's driving process; An intersection range determination module, used for determining whether the vehicle is within the intersection range based on the recognition of the front image data; The first detection module is used to perform road edge detection using right side image data collected by a right side imaging device of the vehicle if the vehicle is located within the intersection.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.

Images