Road boundary detection method, road boundary detection device, computer-readable storage medium, electronic device and computer program

Abstract

An embodiment of the present invention provides a road boundary detection method and an apparatus, electronic device, storage medium, and computer program product, which includes identifying a road image collected by an image collection device mounted on a vehicle, determining a plurality of road boundaries in the road image, and selecting, from the plurality of road boundaries, a road boundary that the vehicle can enter.

Description

Technical Field

[0001] (Cross-reference to Related Applications) This application is filed based on a Chinese patent application filed on March 24, 2022, with an application number of 202210303727.1 and an invention title of "Road Boundary Detection Method and Its Device, Electronic Device, and Storage Medium", claims the priority of the Chinese patent application, and all the contents of the Chinese patent application are incorporated herein by reference.

[0002] This application relates to the field of computer vision technology, but is not limited thereto, and relates to a road boundary detection method and its device, an electronic device, a storage medium, and a computer program product.

Background Art

[0003] In the process of an autonomous vehicle turning at an intersection based on camera perception, in order to provide sufficient basis for determining the turning direction of the autonomous vehicle at the intersection, it is necessary to not only detect the position of the road boundary, but also detect the road boundary that the vehicle can enter. Currently, in the scenario of the road boundary, the road boundary that the vehicle can enter cannot be determined.

Summary of the Invention

Problems to be Solved by the Invention

[0004] To solve the existing technical problems, embodiments of the present invention provide a road boundary detection method and its device, an electronic device, a storage medium, and a computer program product.

Means for Solving the Problems

[0005] To achieve the above object, the technical solutions of the embodiments of the present invention are realized as follows.

[0006] Embodiments of the present invention provide a road boundary detection method, the method includes Identifying road images collected by image acquisition equipment installed on the vehicle, and determining multiple road boundaries within the road images, This includes selecting a road boundary from among the aforementioned multiple road boundaries that the vehicle can enter.

[0007] In some embodiments, selecting a road boundary into which the vehicle can enter from among the plurality of road boundaries is equivalent to determining the vehicle's own roadway based on the road image. This includes determining, from among the multiple road boundaries, which the vehicle can enter, based on the roadway in which the vehicle is located.

[0008] In some embodiments, determining the roadway on which the vehicle is located based on the road image is performed. Identifying traffic signs in the aforementioned road image, This includes determining the roadway in which the vehicle is located based on the aforementioned traffic signs.

[0009] In some embodiments, determining the roadway on which the vehicle is located based on the road image is performed. Identifying the direction of travel of other vehicles in the aforementioned road image, This includes determining the roadway in which the vehicle is located based on the direction of travel of the other vehicle.

[0010] In some embodiments, determining the roadway in which the vehicle is located based on the traffic sign is: The method includes determining the vehicle's own roadway based on the designated road markings, in response to the traffic sign indicating that the roadway in which the vehicle is located is not a one-way roadway and the traffic sign including a designated road marking.

[0011] In some embodiments, determining the roadway on which the vehicle is located based on the direction of travel of the other vehicle includes determining the roadway on which the vehicle is located based on the roadway on which the other vehicle is located in response to the direction of travel of the other vehicle being opposite to the direction of travel of the vehicle.

[0012] In some embodiments, determining which road boundary a vehicle can enter from among the plurality of road boundaries based on the roadway in which the vehicle is located includes determining which road boundary a vehicle can enter from among the plurality of road boundaries based on the traffic sign and the roadway in which the vehicle is located.

[0013] In some embodiments, determining which road boundary a vehicle can enter from among a plurality of road boundaries based on the roadway on which the vehicle is located includes acquiring location information of the vehicle, determining map subdata related to the location information from previously acquired map data, and determining which road boundary a vehicle can enter from among the plurality of road boundaries based on the map subdata, wherein the map data includes at least road data, road sign data, and traffic sign data.

[0014] In some embodiments, determining multiple road boundaries within a road image includes detecting multiple roadways within the road image and determining the multiple road boundaries by connecting the ends of each roadway.

[0015] In some embodiments, determining multiple road boundaries within the road image includes detecting a drivable area within the road image and determining multiple road boundaries within the image based on the contour lines of the drivable area.

[0016] In some embodiments, the method further includes determining the vehicle's travel path based on the road boundary to which the vehicle can enter, and controlling the vehicle to travel along the travel path.

[0017] In some embodiments, the method further includes defining a first region of interest based on a road boundary into which the vehicle can enter, and acquiring an image corresponding to the first region of interest according to a first resolution, wherein the road image is acquired according to a second resolution, the second resolution being smaller than the first resolution.

[0018] In some embodiments, the method further includes defining a second region of interest based on a road boundary into which the vehicle can enter, and acquiring an image corresponding to the second region of interest according to a first frame rate, wherein the road image is acquired according to a second frame rate, the second frame rate being smaller than the first frame rate.

[0019] An embodiment of the present invention further provides a road boundary detection device, the device comprising a detection unit and a selection unit, wherein, The detection unit is configured to identify road images collected by image acquisition equipment installed on the vehicle and to determine multiple road boundaries within the road images. The selection unit is configured to select a road boundary from the plurality of road boundaries into which the vehicle can enter.

[0020] Embodiments of the present invention further provide a computer-readable storage medium in which a computer program for causing a processor to perform steps according to the embodiment of the present invention is stored.

[0021] Embodiments of the present invention further provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the steps of the method according to the embodiment of the present invention by executing the program.

[0022] Embodiments of the present invention further provide a computer program product that includes a computer program or instructions for causing an electronic device to perform steps according to the method described in the embodiment of the present invention. [Effects of the Invention]

[0023] According to the road boundary detection method according to an embodiment of the present invention, based on the identified road boundary, a road boundary that a vehicle can enter can be determined. In particular, in a scenario where the road boundary cannot be visually recognized, by determining the road boundary that the vehicle can enter, sufficient grounds can be provided for determining the direction change of the vehicle at an intersection.

[0024] It should be understood that the above general description and the detailed description to be described later are merely illustrative and explanatory, and do not limit the present application.

Brief Description of the Drawings

[0025] [Figure 1a] It is a schematic diagram of a road boundary in the road boundary detection method according to an embodiment of the present invention. [Figure 1b] It is a schematic diagram of a road boundary that can be entered in the road boundary detection method according to an embodiment of the present invention. [Figure 2a] It is a schematic diagram 1 of an application scenario according to an embodiment of the present invention. [Figure 2b] It is a schematic diagram 2 of an application scenario according to an embodiment of the present invention. [Figure 3] It is an exemplary flowchart 1 of the road boundary detection method according to an embodiment of the present invention. [Figure 4a] It is a schematic diagram 1 of a scenario of the own lane in the road boundary detection method according to an embodiment of the present invention. [Figure 4b] It is a schematic diagram 2 of a scenario of the own lane in the road boundary detection method according to an embodiment of the present invention. [Figure 5] It is an exemplary structural diagram 1 of the configuration of a road boundary detection device according to an embodiment of the present invention. [Figure 6] It is an exemplary structural diagram 2 of the configuration of a road boundary detection device according to an embodiment of the present invention. [Figure 7] It is an exemplary structural diagram 3 of the configuration of a road boundary detection device according to an embodiment of the present invention. [Figure 8] It is an exemplary structural diagram of the hardware configuration of an electronic device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings necessary for the embodiments of the present invention have been described above.

[0027] The drawings herein are incorporated into this specification and constitute part thereof, and these drawings illustrate embodiments consistent with the present application and are used together with this specification to illustrate the technical solutions of embodiments of the present invention.

[0028] The present application will be described in more detail below with reference to the drawings and specific embodiments.

[0029] Before describing the proposed road boundary detection technology according to an embodiment of the present invention, we will first briefly explain some concepts.

[0030] Figures 1a and 1b are schematic diagrams of the road boundary and the accessible road boundary in the road boundary detection method of an embodiment of the present invention, respectively. The road boundary includes not only the boundaries 110 on both sides of the roadway where the vehicle 130 is located, but also the boundaries 120 perpendicular to both sides of the roadway. As shown in Figure 1a, the boundaries 120 perpendicular to both sides of the roadway are included. In the following embodiments, these boundaries are collectively referred to as road boundaries, and eight road boundaries are shown at the intersections shown in the scenario.

[0031] As shown in Figure 1b, the roadway is a two-way roadway, and the road boundary corresponding to the direction of vehicle travel is the road boundary into which the vehicle can enter. Assuming that vehicle 130 is traveling on the left side, for each of the two road boundaries 120 in each direction, the left-hand road boundary is the road boundary 140 into which the vehicle can enter.

[0032] Figure 2a is a schematic diagram 1 of an application scenario of an embodiment of the present invention, and as shown in Figure 2a, assuming that there is an obstacle 210 at the southwest corner of the intersection in the intersection scenario shown in Figures 1a and 1b (the top, bottom, left, and right of the image correspond to north, south, west, and east, respectively), under normal circumstances, the obstacle 210 obstructs the view of a vehicle traveling from south to north, and Figure 2b is a schematic diagram 2 of an application scenario of an embodiment of the present invention, and as shown in Figure 2b, the driver or sensor inside the vehicle 230 is obstructed by the obstacle 210. Information could not be obtained for some of the areas, and these areas are called unknown areas 220. Of the eight road boundaries present in the scenario, road boundaries that are obscured by obstacles or cannot be perceived by the driver or sensors in the vehicle 230 for other reasons (e.g., too far away) are called unseen road boundaries 250 (thick solid lines in Figures 2a and 2b), and road boundaries that can be perceived by the driver or sensors in the vehicle 230 are called visible road boundaries 240 (thick dashed lines in Figure 2a).

[0033] To solve the above problems, in an embodiment of the present invention, road images collected by an image acquisition device installed on a vehicle are identified, multiple road boundaries within the road image are determined, and a road boundary into which the vehicle can enter is selected from the multiple road boundaries, thereby enabling the identification of road boundaries (especially road boundaries that are not visible) and the determination of road boundaries into which the vehicle can enter.

[0034] In embodiments of the present invention, the terms “equipped with,” “included,” or any other variation thereof are intended to cover non-exclusive coverage, so that a method or apparatus comprising a set of elements may include not only the elements expressly described, but also other elements not expressly enumerated, or elements specific to carrying out the method or apparatus. Unless otherwise limited, the elements limited by the phrase “…included” do not exclude the presence of other related elements (for example, steps in a method or parts in an apparatus, the exemplary parts may be partial circuits, partial processors, partial programs or software, etc.) in a method or apparatus comprising such elements.

[0035] For example, the road boundary detection method according to an embodiment of the present invention includes a series of steps, but the road boundary detection method according to an embodiment of the present invention is not limited to the steps described. Similarly, the road boundary detection device according to an embodiment of the present invention comprises a series of modules, but the device according to an embodiment of the present invention is not limited to comprising the modules explicitly described, and may further comprise modules for acquiring relevant information or modules for processing based on the information.

[0036] In this specification, the term "and / or" describes the relationship between the related elements and indicates that there may be three possible relationships. For example, A and / or B can represent three cases: A existing independently, A and B existing together, or B existing independently. Furthermore, in this specification, the term "at least one" indicates one of several or any combination of at least two of several. For example, including at least one of A, B, and C indicates including any one or more elements selected from the set consisting of A, B, and C.

[0037] An embodiment of the present invention provides a road boundary detection method. Figure 3 is an illustrative flowchart 1 of the road boundary detection method according to an embodiment of the present invention, and as shown in Figure 3, the method includes the following steps.

[0038] In step S301, the road image collected by the image acquisition device installed on the vehicle is identified, and multiple road boundaries within the road image are determined.

[0039] In step S302, a road boundary into which the vehicle can enter is selected from the plurality of road boundaries.

[0040] The road boundary detection method according to embodiments of the present invention is applied by electronic equipment, which may be in-vehicle equipment, a cloud platform, or other computer equipment. Exemplary examples include in-vehicle equipment such as thin clients, thick clients, microprocessor-based systems, and small computer systems implemented in vehicles, and cloud platforms may include distributed cloud computing technology environments including small computer systems or large computer systems.

[0041] In embodiments of the present invention, the in-vehicle equipment can communicate with vehicle sensors, positioning devices, etc., and the in-vehicle equipment can acquire data collected by the vehicle sensors and geographic location information reported by the positioning device via the communication connection. Exemplarily, the vehicle sensors may be at least one of the following devices: millimeter-wave radar, laser radar (LIDAR), camera, etc., and the positioning device may be a device that provides positioning services based on at least one of the following positioning systems: Global Positioning System (GPS), Beidou satellite navigation system, or Galileo satellite navigation system.

[0042] In some embodiments, the in-vehicle equipment may be an Advanced Driving Assistant System (ADAS), which is installed in the vehicle and can acquire real-time vehicle location information from the vehicle's positioning device, and / or can acquire image data, radar data, etc., representing information about the vehicle's surrounding environment from the vehicle's sensors. Here, the ADAS can transmit vehicle driving data, including the vehicle's real-time location information, to a cloud platform, and in this way the cloud platform can receive the vehicle's real-time location information, and / or image data, radar data, etc., representing information about the vehicle's surrounding environment.

[0043] In embodiments of the present invention, road images are acquired via image acquisition equipment (i.e., the above-mentioned sensors such as cameras) installed on the vehicle, and the image acquisition equipment collects road images or environmental images around the vehicle in real time as the vehicle moves. Furthermore, based on detection and identification of the road images, a plurality of road boundaries related to the vehicle within the road images are determined, and a road boundary that the vehicle can enter is selected from among the plurality of road boundaries.

[0044] By employing the technical solutions of the embodiments of the present invention, an electronic device can determine a road boundary into which a vehicle can enter based on an identified road boundary, and in particular, in scenarios where the road boundary is not visible, determining the road boundary into which a vehicle can enter can provide sufficient grounds for deciding whether to turn around at an intersection.

[0045] In some embodiments of the present invention, determining a plurality of road boundaries in the road image includes detecting a plurality of roadways in the road image and determining the plurality of road boundaries by connecting the ends of each roadway.

[0046] In embodiments of the present invention, the first network can detect multiple roadways in a road image, that is, it can detect multiple roadway lines in a road image. Exemplarily, the first network processes the road image to obtain roadway lines in the road image and then connects the edges of the roadway lines to obtain multiple road boundaries related to vehicles.

[0047] In other embodiments, other image detection techniques can be employed to detect multiple roadways within a road image. For example, first, the road image is subjected to grayscale processing, roadway edges are detected within the grayscale-processed road image, edge detection is performed using, for example, an edge detection operator, and then the roadway lines within the road image are obtained by further performing binarization on the processed image.

[0048] In some other embodiments, determining multiple road boundaries within the road image includes detecting a drivable area within the road image and determining multiple road boundaries within the image based on the contour lines of the drivable area.

[0049] In embodiments of the present invention, a second network can detect drivable areas within a road image. These drivable areas (Freespace), also called passable areas, represent areas where a vehicle can travel or areas where a vehicle can travel. Road images typically include, in addition to the current vehicle, other vehicles, pedestrians, trees, road edges, etc. The areas where these other vehicles, pedestrians, trees, and road edges are located are all areas where the current vehicle cannot travel. Therefore, by processing the road image with the second network, the areas where other vehicles, pedestrians, trees, road edges, etc. are located within the road image are removed to obtain the drivable area for the vehicle.

[0050] In some other embodiments, determining multiple road boundaries within the road image includes using a third network to detect the road image and determine multiple road boundaries associated with the vehicle.

[0051] In embodiments of the present invention, a pre-trained third network can be used to process road images and obtain multiple road boundaries associated with the vehicle.

[0052] Here, the first network, the second network, and the third network described above can all be deep neural networks (DNNs).

[0053] In some embodiments of the present invention, selecting a road boundary from which the vehicle can enter from the plurality of road boundaries includes determining the vehicle's driveway based on the road image, and determining the road boundary from which the vehicle can enter from the plurality of road boundaries based on the vehicle's driveway.

[0054] In an embodiment of the present invention, after determining the road boundary shown in Figure 1a, the roadway on which the vehicle is located can be determined, and based on the roadway on which the vehicle is located, the road boundaries from which the vehicle can enter can be determined.

[0055] Here, the road boundary corresponding to the direction of vehicle travel is the road boundary that the vehicle can enter. For example, as shown in Figure 1b, when a vehicle is traveling on the left side, the road boundary within the left-hand carriageway is the road boundary that the vehicle can enter, and the road boundary within the right-hand carriageway is the road boundary that the vehicle cannot enter. Conversely, when a vehicle is traveling on the right side, the road boundary within the right-hand carriageway is the road boundary that the vehicle can enter, and the road boundary within the left-hand carriageway is the road boundary that the vehicle cannot enter. Here, the above terms "left side" and "right side" are relative, and when a person faces the road boundary shown in Figure 1a along the direction of vehicle travel, in the two carriageways divided by the solid line of the carriageway, the left carriageway is called the left-hand carriageway, and the right carriageway is called the right-hand carriageway.

[0056] In some embodiments, determining the roadway on which the vehicle is located based on the road image includes identifying traffic signs in the road image and determining the roadway on which the vehicle is located based on the traffic signs.

[0057] In some other embodiments, determining the roadway on which the vehicle is located based on the road image includes identifying the direction of travel of other vehicles in the road image and determining the roadway on which the vehicle is located based on the direction of travel of the other vehicles.

[0058] In embodiments of the present invention, the electronic device can determine the roadway in which the vehicle is located based on identified traffic signs and / or the direction of travel of other vehicles.

[0059] Here, exemplary, the traffic signs include at least one of the following: signs indicated by traffic sign boards and road signs. Here, the traffic sign boards are used as graphic symbols to indicate traffic regulations and road information, are usually installed at intersections and road edges, and are used for traffic management and direction indication to ensure smooth road flow and driving safety. The road signs are, for example, line markers on the road (e.g., solid white lines, dashed white lines, solid yellow lines, double solid yellow lines, etc.) and road signs indicating road attributes (e.g., straight signs, turn signs, speed limit signs, bus-only signs, etc., that is, signs artificially drawn on the road).

[0060] In embodiments of the present invention, an electronic device can determine the roadway on which a vehicle is located based on the direction of travel of other vehicles detected in a road image. The electronic device can also determine the roadway on which a vehicle is located based on traffic signs detected in a road image. The electronic device can also determine the roadway on which a vehicle is located based on traffic signs detected in a road image and the direction of travel of other vehicles.

[0061] In some embodiments, determining the roadway in which the vehicle is located based on the traffic sign includes determining the roadway in which the vehicle is located based on the designated road marking in response to the traffic sign indicating that the roadway in which the vehicle is located is not a one-way roadway and the traffic sign includes a designated road marking.

[0062] In embodiments of the present invention, the designated road markings are used to indicate traffic flow traveling in the same direction or to separate traffic flow traveling in opposite directions. Exemplary examples of the designated road markings are, for example, solid lines (e.g., solid yellow lines, double solid yellow lines, etc.) and dotted lines (e.g., dashed white lines).

[0063] In some other embodiments, determining the roadway on which the vehicle is located based on the direction of travel of the other vehicle includes determining the roadway on which the vehicle is located based on the roadway on which the other vehicle is located in response to the direction of travel of the other vehicle being opposite to the direction of travel of the vehicle.

[0064] As an example, Figure 4a is a schematic diagram 1 of the scenario for the vehicle's own roadway in the road boundary detection method of an embodiment of the present invention. As shown in Figure 4a, if it is determined that the roadway is not a one-way roadway by identifying a traffic sign, and a solid line (for example, the thick solid line 410 in Figure 4a) is identified in the road image, and the vehicle 400 is traveling on the left side, then it can be determined that the roadway to the left of the solid line is the vehicle's own roadway. In this example, the thick solid line 410 is the designated road marking mentioned above.

[0065] As another example, Figure 4b is a schematic diagram 2 of the self-vehicle roadway scenario in the road boundary detection method of an embodiment of the present invention. As shown in Figure 4b, if a dotted line (e.g., the thick dashed line 420 in Figure 4b) is identified by road image identification and vehicle 400 is traveling on the left side, it can be determined that the roadway to the right of the dotted line may be the self-vehicle roadway. Furthermore, the self-vehicle roadway can be determined based on other traffic signs or road image identification results.

[0066] As another example, if road image identification identifies the presence of another vehicle traveling in the opposite direction to the current vehicle, it can be determined that the roadway where the other vehicle is located is not the vehicle's own roadway. Furthermore, the roadway where the other vehicle (traveling in the opposite direction to the current vehicle) is located can be removed from the roadway to obtain the vehicle's own roadway. In another embodiment, after determining that the roadway where the other vehicle is located is not the vehicle's own roadway, the road boundary corresponding to the roadway where the other vehicle is located can be determined, and the road boundary corresponding to the roadway where the other vehicle (traveling in the opposite direction to the current vehicle) is located can be removed from the multiple road boundaries determined in step S301 to obtain a road boundary that the vehicle can enter.

[0067] In some embodiments of the present invention, determining which road boundary a vehicle can enter from among the plurality of road boundaries based on the roadway in which the vehicle is located includes determining which road boundary a vehicle can enter from among the plurality of road boundaries based on the traffic sign and the roadway in which the vehicle is located.

[0068] In embodiments of the present invention, the electronic device can identify traffic signs in a road image in real time and, in combination with the roadway on which the vehicle is located, determine from among the plurality of road boundaries that the vehicle can enter.

[0069] For example, the traffic signs may include at least one of the following signs: one-way signs, right-turn signs at roundabouts, signs prohibiting driving in a specified direction, no-entry signs, road closed signs, no-vehicle crossing signs, no-turn signs, pedestrian-only signs, bicycle-only signs, bicycle and pedestrian-only signs, stop lines, roadway lines, etc.

[0070] In embodiments of the present invention, the electronic device can determine a plurality of road boundaries related to the vehicle and the roadway in which the vehicle is located, and then determine the road boundaries that the vehicle can enter based on each traffic sign around the vehicle.

[0071] In some other embodiments of the present invention, determining a road boundary into which a vehicle can enter from among a plurality of road boundaries, based on the roadway in which the vehicle is located, includes acquiring location information of the vehicle, determining map subdata related to the location information from previously acquired map data, and determining a road boundary into which the vehicle can enter from among the plurality of road boundaries based on the map subdata, wherein the map data includes at least road data, road sign data, and traffic sign data.

[0072] In embodiments of the present invention, the electronic device can acquire map data in advance, and the map data may include prior information such as road information and traffic sign information. The electronic device can determine the direction of travel of the vehicle based on the location information of the vehicle, determine the route the vehicle can travel on based on the location information of the vehicle and the direction of travel, and determine the road boundaries that the vehicle can enter or cannot enter based on the route the vehicle can travel on.

[0073] In some embodiments of the present invention, the method further includes determining the vehicle's travel path based on the road boundary to which the vehicle can enter, and controlling the vehicle to travel along the travel path.

[0074] In embodiments of the present invention, with respect to a road boundary into which a vehicle can enter, the electronic device can determine the vehicle's travel path, and the electronic device can control the vehicle to travel according to the travel path.

[0075] In some embodiments of the present invention, the method further includes defining a first region of interest based on a road boundary into which the vehicle can enter, and acquiring an image corresponding to the first region of interest according to a first resolution, wherein the road image is acquired according to a second resolution, the second resolution being smaller than the first resolution.

[0076] In some other embodiments of the present invention, the method further includes defining a second region of interest based on a road boundary into which the vehicle can enter, and acquiring an image corresponding to the second region of interest according to a first frame rate, wherein the road image is acquired according to a second frame rate, the second frame rate being smaller than the first frame rate.

[0077] In embodiments of the present invention, the electronic device sets a Region of Interest (ROI), i.e., the first and second regions of interest described above, based on the road boundary into which a vehicle can enter. In the process of acquiring road images of the road environment, the electronic device can acquire images at a second resolution (also called low resolution), and in the case of the first region of interest, it can acquire images using a first resolution (also called high resolution) that is higher than the second resolution, thereby collecting higher quality images for the first region of interest and subsequently performing object identification of the images corresponding to the first region of interest. On the other hand, in the process of acquiring road images of the road environment, the electronic device can acquire images using a second frame rate (also called low frame rate), and in the case of the second region of interest, it can acquire images using a first frame rate (also called high frame rate) that is higher than the second frame rate.

[0078] Based on the above embodiments, the embodiments of the present invention further provide a road boundary detection device. Figure 5 is an illustrative structural diagram 1 of the configuration of the road boundary detection device according to the embodiments of the present invention, and as shown in Figure 5, the device comprises a detection unit 51 and a selection unit 52, where, The detection unit 51 is configured to identify road images collected by image acquisition equipment installed on the vehicle and to determine multiple road boundaries within the road images. The selection unit 52 is configured to select a road boundary from among the plurality of road boundaries that the vehicle can enter.

[0079] In some embodiments of the present invention, the selection unit 52 is configured to determine the roadway on which the vehicle is located based on the road image, and to determine from among the plurality of road boundaries that the vehicle can enter based on the roadway on which the vehicle is located.

[0080] In some embodiments of the present invention, the selection unit 52 is configured to identify traffic signs in the road image and to determine the roadway on which the vehicle is located based on the traffic signs.

[0081] In some embodiments of the present invention, the selection unit 52 is configured to identify the direction of travel of other vehicles in the road image and to determine the roadway on which the vehicle is located based on the direction of travel of the other vehicles.

[0082] In some embodiments of the present invention, the selection unit 52 is configured to determine the vehicle's own roadway based on the designated road markings in response to the traffic sign indicating that the roadway in which the vehicle is located is not a one-way roadway and the traffic sign includes designated road markings.

[0083] In some embodiments of the present invention, the selection unit 52 is configured to determine the roadway on which the vehicle is located based on the roadway on which the other vehicle is located, in response to the direction of travel of the other vehicle being opposite to the direction of travel of the vehicle.

[0084] In some embodiments of the present invention, the selection unit 52 is configured to determine, from among the plurality of road boundaries, a road boundary that the vehicle can enter, based on the traffic sign and the roadway on which the vehicle is located.

[0085] In some embodiments of the present invention, the selection unit 52 is configured to acquire location information of the vehicle, determine map subdata related to the location information from previously acquired map data, and determine a road boundary that the vehicle can enter from among the plurality of road boundaries based on the map subdata, wherein the map data includes at least road data, road sign data, and traffic sign data.

[0086] In some embodiments of the present invention, the detection unit 51 is configured to detect a plurality of roadways in the road image and determine a plurality of road boundaries related to the vehicle by connecting the ends of each roadway.

[0087] In some embodiments of the present invention, the detection unit 51 is configured to detect a drivable area in the road image and to determine a plurality of road boundaries related to the vehicle based on the contour lines of the drivable area.

[0088] In some embodiments of the present invention, as shown in Figure 6, the apparatus further comprises a first control unit 53, the first control unit 53 is configured to determine the vehicle's travel path based on the road boundary in which the vehicle can enter, and to control the vehicle to travel according to the travel path.

[0089] In some embodiments of the present invention, as shown in Figure 7, the apparatus further comprises a second control unit 54, the second control unit 54 configured to set a first region of interest based on the road boundary into which the vehicle can enter, and to acquire an image corresponding to the first region of interest according to a first resolution, where the road image is acquired according to a second resolution, the second resolution being smaller than the first resolution.

[0090] In some embodiments of the present invention, as shown in Figure 7, the apparatus further comprises a second control unit 54, the second control unit 54 configured to set a second region of interest based on the road boundary into which the vehicle can enter, and to acquire an image corresponding to the second region of interest according to a first frame rate, where the road image is acquired according to a second frame rate, the second frame rate being smaller than the first frame rate.

[0091] In embodiments of the present invention, the apparatus is applied to electronic equipment. The detection unit 51, selection unit 52, first control unit 53, and second control unit 54 in the apparatus can all be implemented in actual applications by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a field-programmable gate array (FPGA).

[0092] In the above embodiment, when the road boundary detection device performs road boundary detection, only the division of each program module as described above was given as an example. However, in actual applications, the above processing may be assigned to different program modules as needed, that is, the internal structure of the device may be divided into different program modules to perform all or part of the above processing. Furthermore, the road boundary detection device according to the above embodiment belongs to the same concept as the embodiment of the road boundary detection method, and its specific implementation process can be found by referring to the embodiment of the method.

[0093] Embodiments of the present invention further provide an electronic device, and Figure 8 is an exemplary structural diagram of the hardware configuration of the electronic device of an embodiment of the present invention, as shown in Figure 8, the electronic device comprises a memory 82, a processor 81, and a computer program stored in the memory 82 and executable by the processor 81, the processor 81 executing the program to realize the steps of the road boundary detection method described in an embodiment of the present invention.

[0094] In some embodiments, the electronic device may further include a user interface 83 and a network interface 84. Here, the user interface 83 may include a display, keyboard, mouse, trackball, click wheel, keystrokes, buttons, touchpad or touchscreen, etc.

[0095] In some embodiments, each component in an electronic device is coupled via a bus system 85. Understandably, the bus system 85 is used to facilitate communication between these components. In addition to the data bus, the bus system 85 further includes a power bus, a control bus, and a status signal bus. However, for clarity, in Figure 8, the various buses are labeled as the bus system 85.

[0096] It can be understood that memory 82 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Here, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disk, or CD-ROM (Compact Disc Read-Only Memory), and magnetic surface memory may be magnetic disk memory or magnetic tape memory. Volatile memory may be random access memory (RAM) used as an external cache.To illustrate, in an illustrative but non-limiting manner, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synclink dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memory 82 described in the embodiments of the present invention includes, but is not limited to, these and any other suitable types of memory.

[0097] The methods disclosed in the embodiments of the present invention described above can be applied to or implemented by a processor 81. The processor 81 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be performed by hardware integrated logic circuits or software-type instructions within the processor 81. The processor 81 may be a general-purpose processor, a DSP, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The processor 81 can implement or execute each method, step and logic block diagram disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention may be performed directly by a hardware decoding processor or by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 82, and the processor 81 reads information in memory 82 and performs the steps of the above method in combination with its hardware.

[0098] In exemplary embodiments, the electronic device can be implemented using one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the above method.

[0099] In exemplary embodiments, embodiments of the present invention further provide a computer-readable storage medium, such as a memory 82 containing a computer program, the computer program being executed by a processor 81 of an electronic device to perform the steps of the method described above. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM, or it may be a variety of devices containing any of the above memories or any combination thereof.

[0100] The computer-readable storage medium according to an embodiment of the present invention stores a computer program in which a processor is instructed to perform the steps of the road boundary detection method described in an embodiment of the present invention.

[0101] Embodiments of the present invention further provide a computer program product that includes a computer program or instructions for causing an electronic device to perform the steps of the road boundary detection method described in the embodiment of the present invention.

[0102] The methods disclosed in the embodiments of several methods provided in the present invention can be arbitrarily combined without conflict to obtain new embodiments of methods.

[0103] The features disclosed in the examples of some of the products provided in this invention can be arbitrarily combined without conflict to obtain new examples of products.

[0104] The features disclosed in some of the embodiments of the methods or apparatus provided in the present invention can be arbitrarily combined without conflict to obtain new embodiments of the methods or apparatus.

[0105] In some embodiments provided in the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments of the devices described above are illustrative only, and for example, the division of parts is merely a division of logical functions, and in actual implementation, there may be other methods of division, for example, multiple parts or components may be combined or integrated into another system, and some features may be ignored or not performed. Also, the coupling, direct coupling or communication connection between each component shown or discussed may be an indirect coupling or communication connection via several interfaces, devices or parts, and may be in the form of electrical, mechanical or other.

[0106] The parts described as individual components may or may not be physically separated, and the parts shown as components may or may not be physical components, and may be located in one place or distributed across multiple network sections. Depending on the actual needs, some or all of these components can be selected to achieve the objectives of the technical solutions in the embodiments of the present invention.

[0107] Furthermore, each functional part in each embodiment of the present invention may be integrated into a single processing part, each part may be separated as an independent part, or two or more parts may be integrated into one part, and the integrated part may be implemented as hardware or as a combination of hardware and software.

[0108] As those skilled in the art will understand, all or some of the steps of the embodiments of the above method can be performed by hardware relating to program instructions, the program can be stored in a computer-readable storage medium, and when the program is executed, the steps of the embodiments of the above method are performed, the storage medium includes a medium capable of storing program code, such as mobile storage, ROM, RAM, magnetic memory, or optical disc.

[0109] Alternatively, if the integrated portion described above in the present invention is implemented in the form of a software function module and sold or used as an independent product, it can be stored on a single computer-readable storage medium. Based on this understanding, the essential portion of the technical solution of the present invention, i.e., the portion that contributes to the prior art, can be embodied in the form of a software product, which is stored on a single storage medium and contains several instructions for causing a single computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage mediums include various media capable of storing program code, such as mobile storage, ROM, RAM, magnetic disks, or optical disks.

[0110] The above description represents only specific embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that are readily conceivable to those skilled in the art within the technical scope disclosed herein should be included within the scope of protection of the present invention. Accordingly, the scope of protection of the present invention shall be subject to the scope of protection of the claims. [Industrial applicability]

[0111] Embodiments of the present invention provide a road boundary detection method and apparatus, electronic equipment, and storage medium. The method includes identifying a road image collected by an image acquisition device installed on a vehicle, determining a plurality of road boundaries within the road image, and selecting a road boundary from the plurality of road boundaries that the vehicle can enter. By employing the technical solution of the embodiment of the present invention, it is possible to determine a road boundary that a vehicle can enter based on the identified road boundary, and in particular, in scenarios of road boundaries that are not visible, determining a road boundary that a vehicle can enter can provide sufficient grounds for deciding whether to change direction at an intersection.

Claims

1. A road boundary detection method performed by electronic equipment, Identifying road images collected by image acquisition equipment installed on the vehicle, and determining multiple road boundaries within the road images, This includes selecting a road boundary from among the aforementioned multiple road boundaries that the vehicle can enter, Determining multiple road boundaries within the aforementioned road image is: This includes detecting multiple roadways within the aforementioned road image and determining the boundaries of the multiple roads by connecting the ends of each roadway, Road boundary detection method.

2. Selecting a road boundary from among the aforementioned multiple road boundaries that the vehicle can enter is, Based on the aforementioned road image, the vehicle's own roadway is determined, This includes determining, from among the plurality of road boundaries, a road boundary into which the vehicle can enter, based on the roadway in which the vehicle is located. The road boundary detection method according to claim 1.

3. Based on the aforementioned road image, determining the roadway on which the vehicle is located is: Identifying traffic signs within the aforementioned road image, and determining the roadway on which the vehicle is located based on the traffic signs, or This includes identifying the direction of travel of other vehicles in the road image and determining the roadway on which the vehicle is located based on the direction of travel of the other vehicles. The road boundary detection method according to claim 2.

4. Determining the roadway in which the vehicle is located based on the aforementioned traffic signs is: The traffic sign indicates that the roadway in which the vehicle is located is not a one-way roadway, and in response to the traffic sign including a designated road marking, the vehicle determines the roadway in which it is located based on the designated road marking. The road boundary detection method according to claim 3.

5. Determining the roadway in which the vehicle is located based on the direction of travel of the other vehicle is, In response to the other vehicle's direction of travel being opposite to the vehicle's direction of travel, the system includes determining the vehicle's own roadway based on the roadway the other vehicle is located on. The road boundary detection method according to claim 3.

6. Determining which road boundary the vehicle can enter from among the multiple road boundaries, based on the roadway in which the vehicle is located, This includes determining, from among the multiple road boundaries, which the vehicle can enter, based on the aforementioned traffic sign and the roadway on which the vehicle is located. The road boundary detection method according to claim 3.

7. Determining which road boundary the vehicle can enter from among the multiple road boundaries, based on the roadway in which the vehicle is located, The process includes acquiring location information of the vehicle, determining map subdata related to the location information from previously acquired map data, and determining the road boundary into which the vehicle can enter from among the multiple road boundaries based on the map subdata, wherein the map data includes at least road data, road sign data, and traffic sign data. The road boundary detection method according to claim 2.

8. The aforementioned road boundary detection method is: Based on the road boundary into which the vehicle can enter, the vehicle's travel path is determined, and the travel time Further includes controlling the vehicle to travel along a road, A road boundary detection method according to any one of claims 1 to 7.

9. The aforementioned road boundary detection method is: Based on the road boundary into which the vehicle can enter, a first region of interest is established, and according to the first resolution... The method further includes acquiring an image corresponding to the first region of interest, wherein the road image is acquired according to a second resolution, the second resolution being smaller than the first resolution. A road boundary detection method according to any one of claims 1 to 7.

10. The aforementioned road boundary detection method is: The process further includes setting a second region of interest based on the road boundary into which the vehicle can enter, and acquiring an image corresponding to the second region of interest according to a first frame rate, wherein the road image is acquired according to a second frame rate, the second frame rate being smaller than the first frame rate. A road boundary detection method according to any one of claims 1 to 7.

11. A road boundary detection device comprising a detection unit and a selection unit, The detection unit is configured to identify road images collected by image acquisition equipment installed on the vehicle and to determine multiple road boundaries within the road images. The selection unit is configured to select a road boundary from among the plurality of road boundaries that the vehicle can enter. Determining multiple road boundaries within the aforementioned road image is: This includes detecting multiple roadways within the aforementioned road image and determining the boundaries of the multiple roads by connecting the ends of each roadway, Road boundary detection device.

12. A computer-readable storage medium having a processor that stores a computer program for causing the processor to perform the steps of the road boundary detection method described in any one of claims 1 to 7.

13. Electronic device comprising memory, a processor, and a computer program stored in memory and executable by the processor, wherein the processor executes the program to realize the steps of the road boundary detection method described in any one of claims 1 to 7.

14. A computer program that causes a computer to perform the steps of the road boundary detection method described in any one of claims 1 to 7.

Images