Road boundary detection method and device, electronic device, storage medium, and computer program product

Abstract

An embodiment of the present invention provides a road boundary detection method and an apparatus, electronic device, storage medium, and computer program product, which includes identifying a road image collected by an image collection device mounted on a vehicle, determining a plurality of road boundaries in the road image, and selecting, from the plurality of road boundaries, a road boundary that the vehicle can enter.

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on a Chinese patent application filed on March 24, 2022, bearing application number 202210303727.1 and entitled "Road boundary detection method and apparatus, electronic device, and storage medium", and claims priority to the Chinese patent application, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of computer vision technology, including, but not limited to, a road boundary detection method and apparatus, an electronic device, a storage medium, and a computer program product. [Background technology]

[0003] In the process of an autonomous vehicle turning at an intersection based on camera sensing, in order to provide sufficient basis for the autonomous vehicle to decide to turn at the intersection, it is necessary not only to detect the location of the road boundary, but also to detect the road boundary that the vehicle can enter. Currently, in the scenario of the road boundary, it is not possible to determine the road boundary that the vehicle can enter. Summary of the Invention [Problem to be solved by the invention]

[0004] To solve the existing technical problems, the embodiments of the present invention provide a road boundary detection method and apparatus, an electronic device, a storage medium, and a computer program product. [Means for solving the problem]

[0005] To achieve the above objectives, the technical solutions of the embodiments of the present invention are realized as follows:

[0006] An embodiment of the present invention provides a road boundary detection method, the method comprising: identifying a road image collected by an image collection device mounted on a vehicle and determining a plurality of road boundaries within the road image; selecting, from among the plurality of road boundaries, a road boundary into which the vehicle may enter.

[0007] In some embodiments, selecting a road boundary that the vehicle can enter from among the plurality of road boundaries includes determining an ego road on which the vehicle is located based on the road image; and determining, based on an own roadway on which the vehicle is located, a road boundary into which the vehicle can enter from among the plurality of road boundaries.

[0008] In some embodiments, determining an ego road on which the vehicle is located based on the road image comprises: identifying traffic signs within the road image; and determining an ego road on which the vehicle is located based on the traffic sign.

[0009] In some embodiments, determining an ego road on which the vehicle is located based on the road image comprises: identifying the direction of travel of other vehicles in the road image; and determining an ego road on which the vehicle is located based on the driving direction of the other vehicle.

[0010] In some embodiments, determining an ego roadway on which the vehicle is located based on the traffic sign comprises: and in response to the traffic sign indicating that the roadway on which the vehicle is located is not a one-way road and the traffic sign including a designated road marking, determining an own roadway on which the vehicle is located based on the designated road marking.

[0011] In some embodiments, determining an ego roadway on which the vehicle is located based on the driving direction of the other vehicle includes, in response to the driving direction of the other vehicle being opposite to the driving direction of the vehicle, determining an ego roadway on which the vehicle is located based on a location roadway of the other vehicle.

[0012] In some embodiments, determining a road boundary from among the plurality of road boundaries that the vehicle can enter based on an ego road on which the vehicle is located includes determining a road boundary from among the plurality of road boundaries that the vehicle can enter based on the traffic sign and the ego road on which the vehicle is located.

[0013] In some embodiments, determining a road boundary from among the multiple road boundaries that the vehicle can enter based on a vehicle road on which the vehicle is located includes acquiring position information on which the vehicle is located, determining map sub-data related to the position information from previously acquired map data, and determining a road boundary from among the multiple road boundaries that the vehicle can enter based on the map sub-data, wherein the map data includes at least road data, road sign data, and traffic sign board data.

[0014] In some embodiments, determining a plurality of road boundaries in the road image includes detecting a plurality of roadways in the road image and determining the plurality of road boundaries by connecting edges of each roadway.

[0015] In some embodiments, determining a plurality of road boundaries in the road image includes detecting a drivable area in the road image and determining a plurality of road boundaries in the image based on a contour of the drivable area.

[0016] In some embodiments, the method further includes determining a driving path for the vehicle based on road boundaries that the vehicle can enter, and controlling the vehicle to drive according to the driving path.

[0017] In some embodiments, the method further includes setting a first area of ​​interest based on a road boundary that the vehicle can enter, and acquiring an image corresponding to the first area of ​​interest according to a first resolution, where the road image is acquired according to a second resolution, the second resolution being smaller than the first resolution.

[0018] In some embodiments, the method further includes setting a second region of interest based on a road boundary that the vehicle can enter, and acquiring an image corresponding to the second region of interest according to a first frame rate, where the road image is acquired according to a second frame rate, the second frame rate being smaller than the first frame rate.

[0019] An embodiment of the present invention further provides a road boundary detection apparatus, the apparatus comprising a detection unit and a selection unit, wherein: The detection unit is configured to identify road images collected by an image collection device provided on a vehicle and determine a plurality of road boundaries within the road images; The selection unit is configured to select a road boundary that the vehicle can enter from the plurality of road boundaries.

[0020] An embodiment of the present invention further provides a computer readable storage medium having stored thereon a computer program for causing a processor to perform the steps of the method according to an embodiment of the present invention.

[0021] An embodiment of the present invention further provides an electronic device, the electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the processor executing the program to implement steps of the method according to the embodiment of the present invention.

[0022] An embodiment of the present invention further provides a computer program product comprising a computer program or instructions for causing an electronic device to perform the steps of the method according to an embodiment of the present invention. Effect of the Invention

[0023] According to the road boundary detection method of an embodiment of the present invention, it is possible to determine road boundaries that a vehicle can enter based on the identified road boundaries, and in particular in scenarios where road boundaries are not visible, determining the road boundaries that a vehicle can enter can provide sufficient basis for the vehicle's decision to turn at an intersection.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. [Brief description of the drawings]

[0025] [Figure 1a] 2 is a schematic diagram of a road boundary in the road boundary detection method according to the embodiment of the present invention; [Figure 1b] 2 is a schematic diagram of an approachable road boundary in the road boundary detection method according to the embodiment of the present invention; [Figure 2a] FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present invention. [Figure 2b] FIG. 2 is a schematic diagram of an application scenario of an embodiment of the present invention. [Diagram 3] 1 is an exemplary flowchart 1 of a road boundary detection method according to an embodiment of the present invention. [Figure 4a] FIG. 1 is a schematic diagram of a scenario of a vehicle road in a road boundary detection method according to an embodiment of the present invention; [Figure 4b] FIG. 2 is a schematic diagram 2 of a scenario of a vehicle road in the road boundary detection method according to an embodiment of the present invention; [Diagram 5] 1 is an exemplary structural diagram of a road boundary detection device according to an embodiment of the present invention; [Figure 6] FIG. 2 is an exemplary structural diagram of a road boundary detection device according to an embodiment of the present invention. [Figure 7] FIG. 3 is an exemplary structural diagram of a road boundary detection device according to an embodiment of the present invention. [Figure 8] FIG. 2 is an exemplary structural diagram of a hardware configuration of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] In order to describe the technical solutions of the embodiments of the present invention more clearly, the drawings necessary for the embodiments of the present invention have been described above.

[0027] The drawings herein are incorporated in and constitute a part of this specification, and illustrate embodiments consistent with the present application, and together with this specification are used to explain technical solutions of the embodiments of the present invention.

[0028] In the following, the present application will be explained in more detail with reference to the drawings and specific embodiments.

[0029] Before describing the road boundary detection technique solution of the embodiment of the present invention, some concepts will be briefly explained first.

[0030] Figures 1a and 1b are respectively schematic diagrams of road boundaries and enterable road boundaries in a road boundary detection method of an embodiment of the present invention, in which the road boundaries include, in addition to the boundaries 110 on both sides of the roadway where the vehicle 130 is located, the vertical boundaries 120 on both sides of the roadway, as shown in Figure 1a, and in the following embodiments, the boundaries are collectively referred to as road boundaries, and eight road boundaries are shown at the intersection shown in the scenario.

[0031] As shown in Fig. 1b, the roadway is a two-way roadway, and the road boundary corresponding to the vehicle's driving direction is the road boundary that the vehicle can enter. Assuming that the vehicle 130 is driving on the left side, for two road boundaries 120 in each direction, the left road boundary is the road boundary 140 that the vehicle can enter.

[0032] FIG. 2a is a schematic diagram 1 of an application scenario of an embodiment of the present invention. As shown in FIG. 2a, in the intersection scenario shown in FIG. 1a and FIG. 1b, assuming that there is an obstacle 210 at the southwest corner of the intersection (the top, bottom, left, and right of the image correspond to the north, south, west, and east, respectively), in a normal case, the obstacle 210 will block the view of a vehicle traveling from south to north. FIG. 2b is a schematic diagram 2 of an application scenario of an embodiment of the present invention. As shown in FIG. 2b, a driver or a sensor in a vehicle 230 will be blocked by the obstacle 210. In the scenario, information on some of the areas cannot be obtained, and these areas are called unknown areas 220; of the eight road boundaries present in the scenario, those that are blocked by obstacles or cannot be detected by the driver or sensors in the vehicle 230 for other reasons (e.g., because they are too far away) are called invisible road boundaries 250 (thick solid lines in Figures 2a and 2b), and those that can be detected by the driver or sensors in the vehicle 230 are visible road boundaries 240 (thick dashed lines in Figure 2a).

[0033] In order to solve the above problems, in an embodiment of the present invention, road images collected by an image collection device installed in a vehicle are identified, a plurality of road boundaries in the road images are determined, and road boundaries that the vehicle can enter from the plurality of road boundaries are selected, thereby making it possible to identify road boundaries (in particular, invisible road boundaries) and determine road boundaries that the vehicle can enter.

[0034] It should be noted that in the embodiments of the present invention, the terms "comprise", "include" or any other variations thereof are intended to cover a non-exclusive inclusiveness, whereby a method or apparatus including a set of elements not only includes the elements explicitly described, but also includes other elements not explicitly listed, or includes elements inherent to implementing the method or apparatus. Unless otherwise limited, an element limited by the phrase "comprises" does not exclude the presence of another related element (e.g., a step in a method or a part in an apparatus, where an example part may be a partial circuit, a partial processor, a partial program or software, etc.) in the method or apparatus including the element.

[0035] For example, the road boundary detection method according to the embodiment of the present invention includes a series of steps, but the road boundary detection method according to the embodiment of the present invention is not limited to the steps described. Similarly, the road boundary detection device according to the embodiment of the present invention includes a series of modules, but the device according to the embodiment of the present invention is not limited to including the modules explicitly described, and may further include a module for obtaining related information or a module for processing based on the information.

[0036] The term "and / or" in this specification describes an association relationship between related objects and indicates that three relationships may exist, for example, A and / or B can represent three cases: A is present independently, A and B are both present, and B is present independently. Furthermore, the term "at least one" in this specification indicates any combination of one of a plurality, or at least two of a plurality, for example, including at least one of A, B, and C indicates including any one or more elements selected from the set consisting of A, B, and C.

[0037] An embodiment of the present invention provides a road boundary detection method. Figure 3 is an exemplary flow chart 1 of the road boundary detection method of the embodiment of the present invention. As shown in Figure 3, the method includes the following steps:

[0038] In step S301, a road image collected by an image collection device mounted on a vehicle is identified, and a number of road boundaries in the road image are determined.

[0039] In step S302, a road boundary into which the vehicle can enter is selected from the plurality of road boundaries.

[0040] The road boundary detection method of the embodiment of the present invention is applied by an electronic device, which may be an in-vehicle device, a cloud platform, or other computer device. For example, the in-vehicle device may be a thin client, a thick client, a microprocessor-based system, a small computer system, etc. implemented in a vehicle, and the cloud platform may be a distributed cloud computing technology environment including a small computer system or a large computer system, etc.

[0041] In an embodiment of the present invention, the on-board device can be communicatively connected to a sensor, a positioning device, etc. of the vehicle, and the on-board device can obtain data collected by the sensor of the vehicle and geographical location information reported by the positioning device, etc. through the communication connection. Exemplarily, the sensor of the vehicle can be at least one of devices such as a millimeter wave radar, a laser radar (LIDAR), a camera, etc., and the positioning device can be a device that provides a positioning service based on at least one positioning system of a global positioning system (GPS), a Beidou satellite navigation system, or a Galileo satellite navigation system.

[0042] In some embodiments, the in-vehicle device may be an Advanced Driving Assistant System (ADAS), where the ADAS is installed in a vehicle and can obtain real-time position information of the vehicle from a positioning device of the vehicle, and / or the ADAS can obtain image data, radar data, etc. representing vehicle surrounding environment information from a sensor of the vehicle. Here, the ADAS can transmit vehicle driving data including the real-time position information of the vehicle to a cloud platform, and in this way, the cloud platform can receive the real-time position information of the vehicle and / or the image data, radar data, etc. representing vehicle surrounding environment information.

[0043] In an embodiment of the present invention, road images are acquired via an image acquisition device (i.e., the above-mentioned sensor such as a camera) provided on a vehicle, and the image acquisition device collects road images or environmental images around the vehicle in real time according to the movement of the vehicle, and further, by detecting and identifying the road images, a plurality of road boundaries associated with the vehicle in the road images are determined, and a road boundary that the vehicle can enter is selected from the plurality of road boundaries.

[0044] By adopting the technical solution of the embodiment of the present invention, the electronic device can determine the road boundary that the vehicle can enter based on the identified road boundary, and especially in a scenario where the road boundary is not visible, determining the road boundary that the vehicle can enter can provide sufficient basis for the vehicle's decision to turn at the intersection.

[0045] In some embodiments of the present invention, determining a plurality of road boundaries in the road image comprises detecting a plurality of roadways in the road image and determining the plurality of road boundaries by connecting ends of each roadway.

[0046] In an embodiment of the present invention, the first network can detect multiple roads in a road image, i.e., detect multiple road lines in a road image. Illustratively, the first network processes the road image to obtain road lines in the road image, and further obtains multiple road boundaries associated with a vehicle by connecting edges of the road lines.

[0047] In other embodiments, other image detection techniques can be adopted to detect multiple roadways in a road image. For example, first, a grayscale process is performed on the road image, roadway edges are detected in the road image after the grayscale process, and edge detection is performed using, for example, an edge detection operator, and then a binarization process is performed on the processed image to obtain roadway lines in the road image.

[0048] In some other embodiments, determining a plurality of road boundaries in the road image includes detecting a drivable area in the road image and determining a plurality of road boundaries in the image based on a contour of the drivable area.

[0049] In an embodiment of the present invention, the second network can detect a free space in a road image, and the free space is also called a passable area, and represents an area where a vehicle can travel or an area where a vehicle can travel. A road image usually includes other vehicles, pedestrians, trees, road edges, etc. in addition to the current vehicle, and the areas where the other vehicles, pedestrians, trees, and road edges are located are all areas where the current vehicle cannot travel. Therefore, by processing the road image with the second network, the areas where other vehicles, pedestrians, trees, road edges, etc. are located in the road image are removed to obtain the free space for the vehicle.

[0050] In some further embodiments, determining a plurality of road boundaries within the road image includes utilizing a third network to detect the road image and determine a plurality of road boundaries associated with the vehicle.

[0051] In an embodiment of the present invention, a pre-trained third network may be utilized to process road images to obtain a number of road boundaries associated with the vehicle.

[0052] Here, the above first network, second network, and third network may all be deep neural networks (DNNs).

[0053] In some embodiments of the present invention, selecting a road boundary from the plurality of road boundaries that the vehicle can enter includes determining an ego road on which the vehicle is located based on the road image, and determining a road boundary from the plurality of road boundaries that the vehicle can enter based on the ego road on which the vehicle is located.

[0054] In an embodiment of the present invention, after determining the road boundary shown in Figure 1a, the vehicle road on which the vehicle is located is determined, and based on the vehicle road on which the vehicle is located, a road boundary that the vehicle can enter from the multiple road boundaries can be determined.

[0055] Here, the road boundary corresponding to the vehicle travel direction is the road boundary that the vehicle can enter. For example, as shown in FIG. 1b, when the vehicle travels on the left side, the road boundary in the left roadway is the road boundary that the vehicle can enter, and the road boundary in the right roadway is the road boundary that the vehicle cannot enter; correspondingly, when the vehicle travels on the right side, the road boundary in the right roadway is the road boundary that the vehicle can enter, and the road boundary in the left roadway is the road boundary that the vehicle cannot enter. Here, the above "left side" and "right side" are relative, and when a person faces the road boundary shown in FIG. 1a along the vehicle travel direction, in two roadways divided by a solid line of the roadway, the left roadway is called the left roadway, and the right roadway is called the right roadway.

[0056] In some embodiments, determining an ego road on which the vehicle is located based on the road image includes identifying traffic signs in the road image and determining an ego road on which the vehicle is located based on the traffic signs.

[0057] In some other embodiments, determining the ego road on which the vehicle is located based on the road image includes identifying the driving direction of other vehicles in the road image, and determining the ego road on which the vehicle is located based on the driving direction of the other vehicles.

[0058] In an embodiment of the present invention, the electronic device can determine the roadway on which the vehicle is located based on identified traffic signs and / or the direction of travel of other vehicles.

[0059] Here, for example, the traffic signs include at least one of signs indicated by traffic sign boards and road signs. Here, the traffic sign boards are used as graphic symbols indicating traffic regulations and road information, and are usually installed at intersections and road edges to manage traffic and indicate the direction of travel, thereby ensuring smooth flow of traffic and safe driving on the road. The road signs are, for example, line signs on the road (e.g., white solid lines, white broken lines, yellow solid lines, double yellow solid lines, etc.), signs on the road indicating attributes of the road (e.g., direct signs, turn signs, speed limit signs, bus-only signs, etc., that is, signs artificially drawn on the road).

[0060] In an embodiment of the present invention, the electronic device can determine the ego roadway on which the vehicle is located based on the driving directions of other vehicles detected in the road image. The electronic device can also determine the ego roadway on which the vehicle is located based on traffic signs detected in the road image. The electronic device can also determine the ego roadway on which the vehicle is located based on traffic signs detected in the road image and the driving directions of other vehicles.

[0061] In some examples, determining an ego roadway on which the vehicle is located based on the traffic sign includes, in response to the traffic sign indicating that the roadway on which the vehicle is located is not a one-way road and the traffic sign including a designated road marking, determining an ego roadway on which the vehicle is located based on the designated road marking.

[0062] In an embodiment of the present invention, the designated road markings are used to indicate traffic flows traveling in the same direction or to separate traffic flows traveling in opposite directions. Illustratively, the designated road markings are, for example, solid lines (e.g., solid yellow lines, double solid yellow lines, etc.), dotted lines (e.g., dashed white lines).

[0063] In some other embodiments, determining an ego roadway on which the vehicle is located based on the driving direction of the other vehicle includes, in response to the driving direction of the other vehicle being opposite to the driving direction of the vehicle, determining an ego roadway on which the vehicle is located based on the location roadway of the other vehicle.

[0064] As an example, Fig. 4a is a schematic diagram 1 of the scenario of the ego road in the road boundary detection method of the embodiment of the present invention, as shown in Fig. 4a, if the road is determined to be a non-one-way road through the identification of traffic signs, and a solid line (e.g., the thick solid line 410 in Fig. 4a) is identified in the road image, and the vehicle 400 is traveling on the left side, then the road to the left of the solid line can be determined to be the ego road. In this example, the thick solid line 410 is the above-mentioned specified road marking.

[0065] As another example, Figure 4b is a schematic diagram 2 of a scenario of the ego road in the road boundary detection method of an embodiment of the present invention. As shown in Figure 4b, when a dotted line (e.g., the thick dashed line 420 in Figure 4b) is identified through road image identification and the vehicle 400 is traveling on the left side, it can be determined that the road to the right of the dotted line may be the ego road, and the ego road can be determined based on the identification results of other traffic signs or road images.

[0066] As another example, when it is identified by the road image that there is another vehicle traveling in the opposite direction to the current vehicle in the road image, it can be determined that the roadway on which the other vehicle is located is not the vehicle's own roadway. Furthermore, the roadway on which the other vehicle (vehicle traveling in the opposite direction to the current vehicle) is located can be removed from the roadways to obtain the vehicle's own roadway. In another embodiment, after it is determined that the roadway on which the other vehicle is located is not the vehicle's own roadway, a road boundary corresponding to the roadway on which the other vehicle is located can be determined, and the road boundary corresponding to the roadway on which the other vehicle (vehicle traveling in the opposite direction to the current vehicle) is located can be removed from the multiple road boundaries determined in step S301 to obtain the road boundary that the vehicle can enter.

[0067] In some embodiments of the present invention, determining a road boundary from among the plurality of road boundaries that the vehicle can enter based on an own roadway on which the vehicle is located includes determining a road boundary from among the plurality of road boundaries that the vehicle can enter based on the traffic sign and the own roadway on which the vehicle is located.

[0068] In an embodiment of the present invention, the electronic device can identify traffic signs in a road image in real time and, in combination with the vehicle road on which the vehicle is located, determine which road boundaries from among the multiple road boundaries the vehicle can enter.

[0069] For example, the traffic signs may include at least one of a one-way street sign, a roundabout right turn traffic sign, a no driving in a specified direction sign, a no entry sign, a road closed sign, a no vehicle crossing sign, a no turn sign, a pedestrian only sign, a bicycle only sign, a bicycle and pedestrian only sign, a stop line, a roadway line, and the like.

[0070] In an embodiment of the present invention, the electronic device can determine a number of road boundaries associated with the vehicle and determine the roadway on which the vehicle is located, and then determine the road boundaries that the vehicle can enter based on each traffic sign around the vehicle.

[0071] In some other embodiments of the present invention, determining a road boundary that the vehicle can enter from among the multiple road boundaries based on the vehicle road on which the vehicle is located includes acquiring position information on the location of the vehicle, determining map sub-data related to the position information from previously acquired map data, and determining a road boundary that the vehicle can enter from among the multiple road boundaries based on the map sub-data, wherein the map data includes at least road data, road sign data, and traffic sign board data.

[0072] In an embodiment of the present invention, the electronic device can acquire map data in advance, and the map data may be data including a priori information such as road information and traffic sign information. The electronic device can determine the driving direction of the vehicle based on the position information of the vehicle, determine routes on which the vehicle can travel based on the position information of the vehicle and the driving direction, and determine road boundaries on which the vehicle can enter or cannot enter based on the routes on which the vehicle can travel.

[0073] In some embodiments of the present invention, the method further comprises determining a driving path for the vehicle based on road boundaries that the vehicle can enter, and controlling the vehicle to drive according to the driving path.

[0074] In an embodiment of the present invention, for a road boundary that the vehicle can enter, the electronics can determine a driving route for the vehicle, and the electronics can control the vehicle to drive according to the driving route.

[0075] In some embodiments of the present invention, the method further includes setting a first area of ​​interest based on a road boundary that the vehicle can enter, and acquiring an image corresponding to the first area of ​​interest according to a first resolution, where the road image is acquired according to a second resolution, the second resolution being smaller than the first resolution.

[0076] In some other embodiments of the present invention, the method further includes setting a second area of ​​interest based on a road boundary that the vehicle can enter, and acquiring an image corresponding to the second area of ​​interest according to a first frame rate, where the road image is acquired according to a second frame rate, and the second frame rate is smaller than the first frame rate.

[0077] In an embodiment of the present invention, the electronic device sets a region of interest (ROI), i.e., the above-mentioned first and second regions of interest, based on road boundaries that a vehicle can enter. In the process of acquiring a road image of a road environment, the electronic device can acquire an image at a second resolution (also called a low resolution), and for the first region of interest, can adopt a first resolution (also called a high resolution) higher than the second resolution to acquire the image, and collect a higher quality image for the first region of interest, and subsequently perform object recognition of the image corresponding to the first region of interest. Meanwhile, in the process of acquiring a road image of a road environment, the electronic device can acquire an image using a second frame rate (also called a low frame rate), and for the second region of interest, can acquire an image using a first frame rate (also called a high frame rate) higher than the second frame rate.

[0078] Based on the above embodiment, the embodiment of the present invention further provides a road boundary detection device. Figure 5 is an exemplary structural diagram 1 of the configuration of the road boundary detection device of the embodiment of the present invention. As shown in Figure 5, the device includes a detection unit 51 and a selection unit 52, where: The detection unit 51 is configured to identify a road image collected by an image collection device mounted on a vehicle and to determine a plurality of road boundaries in the road image; The selection unit 52 is configured to select, from the plurality of road boundaries, a road boundary into which the vehicle can enter.

[0079] In some embodiments of the present invention, the selection unit 52 is configured to determine an own roadway on which the vehicle is located based on the road image, and to determine a road boundary which the vehicle can enter from among the multiple road boundaries based on the own roadway on which the vehicle is located.

[0080] In some embodiments of the present invention, the selection unit 52 is configured to identify traffic signs in the road image and to determine, based on the traffic signs, an ego road on which the vehicle is located.

[0081] In some embodiments of the present invention, the selection unit 52 is configured to identify the driving directions of other vehicles in the road image and determine the ego-road on which the vehicle is located based on the driving directions of the other vehicles.

[0082] In some embodiments of the present invention, the selection unit 52 is configured, in response to the traffic sign indicating that the roadway on which the vehicle is located is not a one-way road and the traffic sign includes a designated road marking, to determine the own roadway on which the vehicle is located based on the designated road marking.

[0083] In some embodiments of the present invention, the selection unit 52 is configured to determine the vehicle roadway on which the vehicle is located based on the other vehicle's roadway in response to the other vehicle's direction of travel being opposite to the vehicle's direction of travel.

[0084] In some embodiments of the present invention, the selection unit 52 is configured to determine, based on the traffic signs and the roadway on which the vehicle is located, a road boundary which the vehicle can enter from among the plurality of road boundaries.

[0085] In some embodiments of the present invention, the selection unit 52 is configured to acquire location information where the vehicle is located, determine map sub-data related to the location information from previously acquired map data, and determine a road boundary that the vehicle can enter from among the multiple road boundaries based on the map sub-data, and the map data includes at least road data, road sign data, and traffic sign board data.

[0086] In some embodiments of the present invention, the detection unit 51 is configured to detect a plurality of roadways in the road image and determine a plurality of road boundaries associated with the vehicle by connecting the edges of each roadway.

[0087] In some embodiments of the present invention, the detection unit 51 is configured to detect a drivable area in the road image and determine a number of road boundaries associated with the vehicle based on a contour of the drivable area.

[0088] In some embodiments of the present invention, as shown in FIG. 6 , the device further includes a first control unit 53, which is configured to determine a driving route for the vehicle based on road boundaries that the vehicle can enter, and to control the vehicle to drive according to the driving route.

[0089] In some embodiments of the present invention, as shown in FIG. 7, the device further includes a second control unit 54, which is configured to set a first area of ​​interest based on a road boundary where the vehicle can enter, and to acquire an image corresponding to the first area of ​​interest according to a first resolution, where the road image is acquired according to a second resolution, and the second resolution is smaller than the first resolution.

[0090] In some embodiments of the present invention, as shown in FIG. 7, the device further includes a second control unit 54, which is configured to set a second area of ​​interest based on a road boundary where the vehicle can enter, and to acquire an image corresponding to the second area of ​​interest according to a first frame rate, where the road image is acquired according to a second frame rate, and the second frame rate is smaller than the first frame rate.

[0091] In an embodiment of the present invention, the device is applied to an electronic device. In practical application, the detection unit 51, the selection unit 52, the first control unit 53, and the second control unit 54 in the device can be realized by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a field-programmable gate array (FPGA).

[0092] Although the above embodiment of the road boundary detection apparatus has been described by taking only the division of each of the above program modules as an example when detecting road boundaries, in actual applications, the above processes may be assigned to different program modules as necessary, that is, the internal structure of the apparatus may be divided into different program modules to perform all or part of the above processes. Furthermore, the road boundary detection apparatus according to the above embodiment belongs to the same concept as the embodiment of the road boundary detection method, and the specific implementation process thereof may be referred to the embodiment of the method.

[0093] An embodiment of the present invention further provides an electronic device, and Figure 8 is an exemplary structural diagram of the hardware configuration of the electronic device of the embodiment of the present invention. As shown in Figure 8, the electronic device includes a memory 82, a processor 81, and a computer program stored in the memory 82 and executable by the processor 81, and the processor 81 realizes the steps of the road boundary detection method described in the embodiment of the present invention by executing the program.

[0094] In some embodiments, the electronic device may further include a user interface 83 and a network interface 84. Here, the user interface 83 may include a display, a keyboard, a mouse, a trackball, a click wheel, keystrokes, buttons, a touch pad or a touch screen, etc.

[0095] In some embodiments, each component in the electronic device is coupled via a bus system 85. It can be understood that the bus system 85 is used to realize the connection communication between these components. In addition to the data bus, the bus system 85 further includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the various buses are labeled as the bus system 85 in FIG. 8 .

[0096] It can be understood that memory 82 may be volatile or nonvolatile memory, or may include both volatile and nonvolatile memory. Here, the nonvolatile memory may be read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disk, or compact disc read-only memory (CD-ROM), and the magnetic surface memory may be magnetic disk memory or magnetic tape memory. The volatile memory may be random access memory (RAM) used as an external cache.By way of illustrative but non-limiting example, many forms of RAM are available, such as, for example, static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), SyncLink dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM), etc. Memory 82 as described in embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0097] The methods disclosed in the above embodiments of the present invention can be applied to or realized by the processor 81. The processor 81 can be an integrated circuit chip with signal processing capabilities. In the realization process, each step of the above methods can be performed by an integrated logic circuit of hardware in the processor 81 or an instruction in the form of software. The processor 81 can be a general-purpose processor, a DSP, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The processor 81 can realize or execute each method, step, and logic block diagram disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor, or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly performed by a hardware decoding processor, or can be performed by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, which is located in the memory 82, and the processor 81 reads the information in the memory 82 and performs the steps of the above methods in combination with the hardware.

[0098] In an exemplary embodiment, the electronic device may be implemented with one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, microprocessors, or other electronic elements to perform the above methods.

[0099] In an exemplary embodiment, the present invention further provides a computer readable storage medium, such as a memory 82, containing a computer program, which can be executed by a processor 81 of the electronic device to perform the steps of the above-mentioned method. The computer readable storage medium may be a memory, such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM, or may be a variety of devices including any of the above memories or any combination thereof.

[0100] A computer-readable storage medium according to an embodiment of the present invention stores a computer program for causing a processor to execute steps of a road boundary detection method according to an embodiment of the present invention.

[0101] An embodiment of the present invention further provides a computer program product comprising a computer program or instructions for causing an electronic device to perform the steps of the road boundary detection method according to the embodiment of the present invention.

[0102] The methods disclosed in the several method embodiments provided in the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0103] Features disclosed in the several product embodiments provided herein may be combined in any non-conflicting manner to obtain new product embodiments.

[0104] Features disclosed in any method or apparatus embodiment provided in the present invention may be combined in any non-conflicting manner to obtain new method or apparatus embodiments.

[0105] In some embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be realized in other ways. The above-described device embodiments are merely illustrative, and the division of the units is merely a division of logical functions. In actual implementation, there may be other division methods, for example, multiple units or components may be combined or integrated into another system, and some features may be ignored or not implemented. In addition, the coupling or direct coupling or communication connection between each component shown or discussed may be an indirect coupling or communication connection via some interfaces, devices or units, and may be in an electrical, mechanical, or other form.

[0106] The parts described as separate parts may or may not be physically separated, and the parts shown as parts may or may not be physical parts, located in one place or distributed across multiple network parts, and some or all of the parts may be selected according to actual needs to achieve the objective of the technical solutions in the embodiments of the present invention.

[0107] In addition, the functional parts in each embodiment of the present invention may all be integrated into a single processing part, or each part may be separated into an independent part, or two or more parts may be integrated into a single part, and the above integrated parts may be realized as hardware or as a combination of hardware and software.

[0108] As can be understood by those skilled in the art, all or some of the steps of the above method embodiments can be performed by hardware associated with program instructions, and the program can be stored in a computer-readable storage medium, and when the program is executed, the steps of the above method embodiments are executed, and the storage medium includes a medium capable of storing program code, such as a mobile storage, a ROM, a RAM, a magnetic memory or an optical disk.

[0109] Or, when the above integrated parts of the present invention are realized in the form of software functional modules and sold or used as an independent product, they can be stored in one computer-readable storage medium. Based on this understanding, the essential part of the technical solution of the present invention, i.e., the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product is stored in one storage medium and includes several instructions for causing one computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as mobile storage, ROM, RAM, magnetic disk, or optical disk.

[0110] The above content is only a specific embodiment of the present invention, and the protection scope of the present invention is not limited thereto. Any modifications or replacements that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims. [Industrial Applicability]

[0111] An embodiment of the present invention provides a road boundary detection method and its device, electronic device, and storage medium. The method includes: identifying a road image collected by an image collection device installed in a vehicle, determining a plurality of road boundaries in the road image, and selecting a road boundary that the vehicle can enter from the plurality of road boundaries. By adopting the technical solution of the embodiment of the present invention, the road boundary that the vehicle can enter can be determined based on the identified road boundary, and especially in a scenario where the road boundary is not visible, the determination of the road boundary that the vehicle can enter can provide sufficient basis for the vehicle to decide to turn at the intersection.

Claims

1. 1. A road boundary detection method performed by an electronic device, comprising: identifying a road image collected by an image collection device mounted on a vehicle and determining a plurality of road boundaries within the road image; selecting, from the plurality of road boundaries, a road boundary into which the vehicle can enter.

2. Selecting a road boundary into which the vehicle can enter from among the plurality of road boundaries includes determining a vehicle road on which the vehicle is located based on the road image; determining a road boundary into which the vehicle can enter from among the plurality of road boundaries based on a vehicle road on which the vehicle is located; The road boundary detection method according to claim 1 .

3. Determining a vehicle road on which the vehicle is located based on the road image, identifying traffic signs within the road image; and determining an ego road on which the vehicle is located based on the traffic sign. The road boundary detection method according to claim 2 .

4. Determining a vehicle road on which the vehicle is located based on the road image, identifying the direction of travel of other vehicles in the road image; determining a vehicle road on which the vehicle is located based on a driving direction of the other vehicle; The road boundary detection method according to claim 2 .

5. Determining a vehicle road on which the vehicle is located based on the traffic sign in response to the traffic sign indicating that the roadway on which the vehicle is located is not a one-way road and the traffic sign including a designated road marking, determining an own roadway on which the vehicle is located based on the designated road marking. The road boundary detection method according to claim 3.

6. Determining a vehicle road on which the vehicle is located based on a traveling direction of the other vehicle, determining an own vehicle roadway on which the vehicle is located based on a roadway on which the other vehicle is located in response to a driving direction of the other vehicle being opposite to a driving direction of the vehicle. The road boundary detection method according to claim 4.

7. Determining a road boundary into which the vehicle can enter from among the plurality of road boundaries based on a vehicle road on which the vehicle is located, determining a road boundary into which the vehicle can enter from among the plurality of road boundaries based on the traffic sign and a roadway on which the vehicle is located; The road boundary detection method according to claim 3 or 5.

8. Determining a road boundary into which the vehicle can enter from among the plurality of road boundaries based on a vehicle road on which the vehicle is located, acquiring location information of the vehicle, determining map sub-data related to the location information from pre-acquired map data, and determining a road boundary into which the vehicle can enter from among the plurality of road boundaries based on the map sub-data, wherein the map data includes at least road data, road sign data, and traffic sign board data; A road boundary detection method according to any one of claims 2 to 6.

9. Determining a plurality of road boundaries within the road image comprises: detecting a plurality of roadways in the road image and determining the plurality of road boundaries by connecting edges of each roadway; A road boundary detection method according to any one of claims 1 to 8.

10. Determining a plurality of road boundaries within the road image comprises: detecting a driveable area in the road image, and determining a plurality of road boundaries in the image based on a contour of the driveable area; A road boundary detection method according to any one of claims 1 to 8.

11. The road boundary detection method includes: determining a driving route for the vehicle based on a road boundary that the vehicle can enter, and controlling the vehicle to travel according to the driving route; A road boundary detection method according to any one of claims 1 to 10.

12. The road boundary detection method includes: The method further includes: setting a first area of ​​interest based on a road boundary where the vehicle can enter; and acquiring an image corresponding to the first area of ​​interest according to a first resolution, the road image being acquired according to a second resolution, the second resolution being smaller than the first resolution; A road boundary detection method according to any one of the preceding claims.

13. The road boundary detection method includes: and further comprising: setting a second region of interest based on a road boundary where the vehicle can enter; and acquiring an image corresponding to the second region of interest according to a first frame rate, the road image being acquired according to a second frame rate, the second frame rate being smaller than the first frame rate. A road boundary detection method according to any one of the preceding claims.

14. A road boundary detection device comprising: a detection unit; and a selection unit, The detection unit is configured to identify road images collected by an image collection device provided on a vehicle and determine a plurality of road boundaries within the road images; A road boundary detection device, wherein the selection unit is configured to select, from the plurality of road boundaries, a road boundary into which the vehicle can enter.

15. A computer readable storage medium having stored thereon a computer program for causing a processor to carry out the steps of the road boundary detection method according to any one of claims 1 to 13.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the processor executing the program to realize the steps of the road boundary detection method described in any one of claims 1 to 13.

17. A computer program product comprising a computer program or instructions for causing an electronic device to carry out the steps of the road boundary detection method according to any one of claims 1 to 13.

Images