Methods and apparatus for identifying forked intersections

Abstract

This disclosure presents a method and apparatus for identifying road junctions. One specific implementation of the method includes: acquiring the position of a vehicle and the position of the road junction; determining whether the vehicle is within the detection range of the road junction based on the vehicle's position and the road junction's position; if within the detection range, inputting a real-time image of the area in front of the vehicle into a pre-trained detection model to obtain candidate guiding regions; inputting the candidate guiding regions into a pre-trained post-processing model to obtain a classification result and key points of the candidate guiding regions; if the classification result is a guiding region, fitting lane lines based on the key points of the candidate guiding regions. This implementation can optimize the performance of lane line recognition and navigation algorithms at road junctions, improving the user experience of the navigation system.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more particularly to methods and apparatus for identifying forked intersections. Background Technology

[0002] Intersections, as key points in map navigation, often require AR navigation algorithms to accurately draw lane lines and provide correct guidance.

[0003] Existing lane line recognition technology is based on traditional image techniques such as edge detection and parallel line detection.

[0004] The existing technical solutions have the following main shortcomings:

[0005] 1) Based on basic features such as edge information of images and human-defined rules, it cannot adapt well to the complex environment in real driving scenarios and is easily interfered with, resulting in inaccurate lane line recognition at intersections.

[0006] 2) No optimization was made for the fork in the road, making it impossible to determine whether the course has deviated, resulting in insufficient guidance. Summary of the Invention

[0007] Embodiments of this disclosure present methods and apparatus for identifying forked intersections.

[0008] In a first aspect, embodiments of this disclosure provide a method for identifying a diverging intersection, comprising: acquiring the position of a vehicle and the position of the diverging intersection, and determining whether the vehicle is within the detection range of the diverging intersection based on the position of the vehicle and the position of the diverging intersection; if within the detection range of the diverging intersection, inputting a real-time image of the front of the vehicle into a pre-trained detection model to obtain a candidate guiding region; inputting the candidate guiding region into a pre-trained post-processing model to obtain a classification result and key points of the candidate guiding region; if the classification result is a guiding region, fitting a lane line based on the key points of the candidate guiding region.

[0009] In some embodiments, the method further includes: acquiring navigation information; determining the relative position of the vehicle with respect to the lane lines based on the vehicle's position and the lane lines' position; determining whether the vehicle is deviating from its lane based on the relative position and the navigation information; if it is deviating, outputting a lane change reminder message, otherwise outputting a straight-ahead reminder message.

[0010] In some embodiments, the real-time acquired image of the front of the vehicle is input into the detection model to obtain a candidate guiding region, including: inputting the real-time acquired image of the front of the vehicle into the detection model to detect the candidate region and road signs; if the distance between the vehicle's position and the position of the intersection is greater than a predetermined value, the position of the road sign is used to determine whether the candidate region is within the tolerance range; if it is within the tolerance range, the candidate region is determined as a candidate guiding region.

[0011] In some embodiments, the real-time acquired image in front of the vehicle is input into the detection model to obtain a candidate guiding region, including: inputting the real-time acquired image in front of the vehicle into the detection model at the start frame of a predetermined period to obtain a candidate guiding region; and inputting the candidate guiding region into the post-processing model to obtain a classification result and key points of the candidate guiding region, including: if the classification result is a guiding region, recording the coordinates of 3 key points on each of the left and right lanes of the guiding region as state variables.

[0012] In some embodiments, inputting candidate flow guide regions into a post-processing model to obtain classification results and key points of candidate flow guide regions includes: performing the following tracking steps starting from the next frame after the starting frame in each predetermined period: taking the state variables recorded in the previous frame and calculating their bounding rectangle, expanding the bounding rectangle as the search area of ​​the current frame; inputting the search area of ​​the current frame into the post-processing model to obtain the classification results of the current frame and key points of candidate flow guide regions of the current frame; if the classification result of the current frame is a flow guide region, then updating the state variables with the key points of candidate flow guide regions of the current frame; otherwise, stopping tracking.

[0013] In some embodiments, the detection model adopts the YOLOv3 architecture, and the backbone network adopts ShuffleNet_v2.

[0014] In some embodiments, the post-processing model uses the shufflenet_v2 network and includes two branches: classification and regression.

[0015] Secondly, embodiments of this disclosure provide an apparatus for identifying road junctions, comprising: an acquisition unit configured to acquire the position of a vehicle and the position of the road junction, and determine whether the vehicle is within the detection range of the road junction based on the vehicle's position and the position of the road junction; a detection unit configured to, if the vehicle is within the detection range of the road junction, input a real-time acquired image of the front of the vehicle into a pre-trained detection model to obtain a candidate guiding region; a processing unit configured to input the candidate guiding region into a pre-trained post-processing model to obtain a classification result and key points of the candidate guiding region; and a fitting unit configured to, if the classification result is a guiding region, fit a lane line based on the key points of the candidate guiding region.

[0016] In some embodiments, the device further includes a navigation unit configured to: acquire navigation information; determine the relative position of the vehicle with respect to the lane lines based on the vehicle's position and the lane lines' position; determine whether the vehicle is veering off course based on the relative position and the navigation information; if veering off course, output lane change reminder information; otherwise, output straight-ahead reminder information.

[0017] In some embodiments, the detection unit is further configured to: input a real-time image of the front of the vehicle into the detection model to detect candidate regions and road signs; if the distance between the vehicle's position and the location of the intersection is greater than a predetermined value, determine whether the candidate region is within the tolerance range by using the location of the road sign; if it is within the tolerance range, determine the candidate region as a candidate diversion region.

[0018] In some embodiments, the detection unit is further configured to: input the image of the front of the vehicle acquired in real time into the detection model at the start frame of a predetermined period to obtain a candidate guiding area; and the processing unit is further configured to: if the classification result is a guiding area, record the coordinates of 3 key points on each of the left and right lanes of the guiding area as state variables.

[0019] In some embodiments, the processing unit is further configured to perform the following tracking steps starting from the next frame after the starting frame in each predetermined period: take the state variables recorded in the previous frame, calculate its bounding rectangle, expand the bounding rectangle and use it as the search area of ​​the current frame; input the search area of ​​the current frame into the post-processing model to obtain the classification result of the current frame and the key points of the candidate guide area of ​​the current frame; if the classification result of the current frame is a guide area, update the state variables with the key points of the candidate guide area of ​​the current frame; otherwise, stop tracking.

[0020] In some embodiments, the detection model adopts the YOLOv3 architecture, and the backbone network adopts ShuffleNet_v2.

[0021] In some embodiments, the post-processing model uses the shufflenet_v2 network and includes two branches: classification and regression.

[0022] Thirdly, embodiments of this disclosure provide an electronic device for identifying branching intersections, comprising: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in the first aspect.

[0023] Fourthly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements any of the methods described in the first aspect.

[0024] The embodiments of this disclosure provide a method and apparatus for identifying road junctions, proposing a deep learning-based method for lane line localization and fitting at road junctions that can run in real-time in a vehicle-mounted environment. By locating the relative positions of key targets (road signs and guide strips) at the road junction with the current vehicle, it can make corresponding judgments and provide appropriate guidance to determine whether the current vehicle is deviating from its course. By fitting the left and right lane lines at the road junction, it can provide more accurate coordinate shapes for AR navigation lane line drawing. Simultaneously, during driving, it combines GPS information control algorithms, calling the algorithm only within the effective range of the road junction segment, avoiding waste of computational resources. The embodiments of this disclosure can adapt to the complex environment of real-world driving scenarios, bringing a better navigation experience at road junctions. Attached Figure Description

[0025] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0026] Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied;

[0027] Figure 2 This is a flowchart of one embodiment of the method for identifying forked intersections according to the present disclosure;

[0028] Figure 3 This is a schematic diagram of an application scenario of the method for identifying forking intersections according to this disclosure;

[0029] Figure 4 This is a flowchart of yet another embodiment of the method for identifying forking intersections according to the present disclosure;

[0030] Figure 5 This is a schematic diagram of one embodiment of the device for identifying forked intersections according to the present disclosure;

[0031] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0032] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the method or apparatus for identifying crossroads of this application may be applied.

[0035] like Figure 1 As shown, system architecture 100 may include vehicle 101 and traffic sign 102.

[0036] Vehicle 101 can be a regular motor vehicle or an autonomous vehicle. Vehicle 101 may be equipped with a controller 1011, a network 1012, and sensors 1013. Network 1012 serves as the medium for providing a communication link between controller 1011 and sensors 1013. Network 1012 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0037] The controller (also known as the vehicle's brain) 1011 is responsible for the intelligent control of the vehicle 101. The controller 1011 can be a standalone controller, such as a programmable logic controller (PLC), a microcontroller, or an industrial control computer; it can also be a device composed of other electronic components with input / output ports and computational control functions; or it can be a computer device equipped with vehicle driving control applications. The controller is equipped with pre-trained detection and post-processing models.

[0038] Sensor 1013 can be various types of sensors, such as cameras, gravity sensors, wheel speed sensors, temperature sensors, humidity sensors, lidar, millimeter-wave radar, etc. In some cases, vehicle 101 may also be equipped with GNSS (Global Navigation Satellite System) equipment and SINS (Strap-down Inertial Navigation System), etc.

[0039] Vehicle 101 photographs traffic sign 102 while in motion. Traffic sign 102 may include road signs, traffic dividers, etc.

[0040] Vehicle 101 sends the captured raw images, including traffic signs, to the controller for identification, determining the guidance area and fitting lane lines. Combined with navigation information, it determines whether the vehicle has deviated from the course. If it has, the controller prompts the vehicle to change lanes; otherwise, it continues straight.

[0041] It should be noted that the method for identifying branch intersections provided in this application embodiment is generally executed by the controller 1011, and correspondingly, the device for identifying branch intersections is generally disposed in the controller 1011.

[0042] It should be understood that Figure 1 The number of controllers, networks, and sensors shown is merely illustrative. Any number of controllers, networks, and sensors can be used depending on implementation requirements.

[0043] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of a method for identifying a fork in a road according to the present disclosure. The method for identifying a fork in a road includes the following steps:

[0044] Step 201: Obtain the vehicle's location and the location of the intersection, and determine whether the vehicle is within the detection range of the intersection based on the vehicle's location and the location of the intersection.

[0045] In this embodiment, the execution entity of the method for identifying fork in the road (e.g.) Figure 1 The controller shown can acquire images of the road ahead via an onboard camera and determine the guiding zone of a road junction through image recognition. Because road junctions only appear during certain time periods during driving, and the vehicle's computing resources are very valuable, the algorithm only runs during these periods, and does not consume computing resources at other times. Specifically, it acquires the vehicle's current GPS location information in real time during driving. When the vehicle's position is less than a predetermined first distance (e.g., 90 meters) from a pre-stored road junction location in the navigation system, it sends a signal to the main program to start calling the algorithm and process the images from the onboard camera in real time. When the vehicle's position is less than a predetermined second distance (e.g., 5 meters) from the road junction location, lane changing is no longer possible, the algorithm stops calling, and the road junction segment is processed. All road junctions during driving are processed according to the above logic. Optionally, a positioning system other than GPS, such as the BeiDou system, can be used to obtain the vehicle's location information.

[0046] Step 202: If the vehicle is within the detection range of the intersection, the real-time image of the front of the vehicle is input into the pre-trained detection model to obtain the candidate diversion area.

[0047] In this embodiment, if the vehicle is currently within the detection range of the intersection, the detection algorithm shown in steps 202-204 can be activated. A lightweight detection network can be used to locate key elements (diversion zones, road signs) at the intersection in the image. The detection model is a neural network used to identify diversion zones and road signs. First, in terms of model structure, to reduce the consumption of vehicle computing resources, a lightweight backbone network, shufflenet_v2, is used for the detection model, and the network is further pruned to reduce computational load. To adapt to changes in the shape and scale of the diversion zone and road signs, a YOLOv3 detection network is used, and feature fusion is performed at more levels to improve model performance. Second, in terms of the calling strategy, when the distance to the intersection is far (i.e., greater than a predetermined value, such as greater than 60 meters), the diversion zone is a small target in the image, and the output of the detection model is uncertain. The position of the road sign is needed to assist in locating the position of the diversion zone. When the positional relationship between the two is within the tolerance range (e.g., the vertical line of the center of the outer rectangle of the road sign passes through the outer rectangle of the diversion zone), the position of the diversion zone can be considered reliable. As the driving distance gradually decreases, the target in the diversion area gradually increases in size, at which point the location of road signs is no longer needed to assist in localization. The training samples for the detection model need to be labeled with both the diversion area and road signs.

[0048] The detection model is trained in a supervised manner. Training samples need to be labeled with traffic diversion areas and road signs.

[0049] Step 203: Input the candidate diversion region into the pre-trained post-processing model to obtain the classification result and the key points of the candidate diversion region.

[0050] In this embodiment, a lightweight post-processing model is used to fit key points of the detected left and right lane lines in the guide area and filter false detections. Simultaneously, the post-processing model is used to track the guide area. The post-processing model is a neural network that can further determine whether the candidate guide area output by the detection model is a genuine guide area, and can also identify key points of the left and right lane lines within the guide area.

[0051] First, the post-processing model's structure is designed based on ShuffleNet_v2. The model has two branches: classification and regression. The classification branch (a binary classifier that determines whether the candidate guiding regions output by the detection model are genuine guiding regions) further distinguishes the detected guiding regions, mainly used to filter out false detections. The regression branch (a regression model) calculates three key points for each of the left and right lane lines of the guiding region to assist in lane line drawing. The post-processing model requires labeling the guiding regions and key points. The post-processing model is obtained through supervised training. When creating training samples for the post-processing model, at least three key points are selected on each side of the guiding region, including the start point, end point, and midpoint. The midpoint is generally chosen as the position between the start and end points.

[0052] Secondly, due to the large computational load of the detection model, to further reduce the consumption of computing resources, detection is only performed on keyframes (the starting frame of each cycle, which can be one second), and tracking is performed on the remaining frames within the cycle. Using conventional template-matching-based tracking methods would introduce additional computational load. Considering our actual application scenario, we propose a tracking method based on the aforementioned post-processing model, which can complete tracking without introducing additional computational load. Specifically, the implementation is as follows: 1) In the keyframe, the detection network is invoked, and the detected guide regions are fed into the post-processing model for classification and regression. For guide regions with positive classification results, the coordinates of their six key points are recorded as state variables, and tracking of the target begins from the next frame. 2) The bounding rectangle of the state variables recorded in the previous frame is calculated. This bounding rectangle is appropriately expanded as the search area for the current frame. The image area of ​​the current frame is fed into the post-processing model. The classification branch is used to determine whether the area contains a guide region, and the regression branch is used to obtain the accurate location of the key points of the guide region within the area. If the area is determined to contain a guide region, the state variables are updated using the key points of the guide region obtained in the current frame; if it is determined not to contain a guide region, tracking stops. 3) Repeat step 2) to complete the tracking of the target.

[0053] If tracking is interrupted, detection can be started in the next cycle, or the search area can be expanded within the current cycle and tracking can be restarted using the most recent state variables.

[0054] Step 204: If the classification result is a guide zone, then fit the lane line based on the key points of the candidate guide zone.

[0055] In this embodiment, the lane lines at the intersection are drawn by fitting a Bézier curve using the previously obtained lane line key points. For example... Figure 3 As shown, three key points on each of the left and right sides of the diversion area can be used to fit a lane line.

[0056] See also Figure 3 , Figure 3This is a schematic diagram illustrating an application scenario of the method for identifying forked intersections according to this embodiment. Figure 3 In the application scenario, the specific operation process on the vehicle system is as follows:

[0057] 1. During normal driving, the distance between the vehicle and the intersection is measured to determine whether the vehicle is currently within the detection range of the intersection. If it is, the detection algorithm start condition is met, and the process proceeds to step 2; otherwise, step 1 is repeated.

[0058] 2. Under the condition that the detection algorithm is activated, the onboard camera acquires real-time image information of the area in front of the vehicle, capturing multiple frames per second, but only calling the detection model once per second (e.g., ...). Figure 3 As shown, the image input detection model is only acquired in the initial frames of periods T, 2T, 3T… The detected image regions are then fed into a post-processing model for classification and keypoint regression. If a region is determined to be non-flowing, the detection algorithm is called again in the next second; if it is determined to be a flowing region, the keypoint information of the current frame is recorded as a state variable. For the remainder of that second, the post-processing model is used for tracking until the detection algorithm is called again in the next second. This achieves inter-second detection and intra-second tracking.

[0059] 3. Obtain the location of the diversion area and the coordinates of key points obtained in the previous step in real time, draw lane lines, and provide appropriate navigation prompts.

[0060] 4. If the conditions for starting the detection algorithm are not met, stop calling the detection model and start the detection algorithm again when the conditions for starting the detection algorithm are met next time.

[0061] Further reference Figure 4 This illustrates a flow 400 of yet another embodiment of a method for identifying intersections. The flow 400 of the method for identifying intersections includes the following steps:

[0062] Step 401: Obtain the vehicle's location and the location of the intersection, and determine whether the vehicle is within the detection range of the intersection based on the vehicle's location and the location of the intersection.

[0063] Step 402: If the vehicle is within the detection range of the intersection, the real-time image of the front of the vehicle is input into the pre-trained detection model to obtain the candidate diversion area.

[0064] Step 403: Input the candidate diversion region into the pre-trained post-processing model to obtain the classification result and the key points of the candidate diversion region.

[0065] Step 404: If the classification result is a guide zone, then fit the lane line based on the key points of the candidate guide zone.

[0066] Steps 401-404 are basically the same as steps 201-204, so they will not be described again.

[0067] Step 405: Obtain navigation information and determine the vehicle's relative position to the lane lines based on the vehicle's position and the lane lines' positions.

[0068] In this embodiment, the navigation information includes a driving route, indicating which lane to take at the intersection. The vehicle's position is obtained in real time via GPS, and then the coordinates of the lane lines fitted in step 404 are mapped to real space to determine the vehicle's position relative to the lane lines. For example, the vehicle is located in the left lane of the left lane line in the guide zone.

[0069] Step 406: Determine whether the vehicle is off course based on its relative position and navigation information.

[0070] In this embodiment, the navigation information indicates which lane the vehicle should take at a fork in the road. If the relative position matches the lane indicated by the navigation information, there is no deviation; otherwise, the vehicle has deviated from the navigation route.

[0071] Step 407: If the vehicle veers off course, output a lane change reminder message; otherwise, output a straight-ahead reminder message.

[0072] In this embodiment, if the vehicle veers off course, a warning message is output to prompt the driver to change lanes. Otherwise, the driver is prompted to continue straight.

[0073] from Figure 4 It can be seen from this that, with Figure 2 Compared to the corresponding embodiments, the process 400 of the method for identifying intersections in this embodiment embodies the step of using fitted lane lines for navigation. Therefore, the solution described in this embodiment can use the guide zone and the relative position of the current vehicle combined with the navigation route to guide straight-ahead or lane-changing traffic. This optimizes the performance of lane line recognition and navigation algorithms at intersections, improving the user experience of the navigation system.

[0074] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a device for identifying forked intersections, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0075] like Figure 5As shown, the device 500 for identifying intersections in this embodiment includes: an acquisition unit 501, a detection unit 502, a processing unit 503, and a fitting unit 504. The acquisition unit 501 is configured to acquire the position of the vehicle and the position of the intersection, and determine whether the vehicle is within the intersection detection range based on the vehicle's position and the intersection's position. The detection unit 502 is configured to, if the vehicle is within the intersection detection range, input a real-time acquired image of the area in front of the vehicle into a pre-trained detection model to obtain a candidate guiding region. The processing unit 503 is configured to input the candidate guiding region into a pre-trained post-processing model to obtain a classification result and key points of the candidate guiding region. The fitting unit 504 is configured to, if the classification result is a guiding region, fit lane lines based on the key points of the candidate guiding region.

[0076] In this embodiment, the specific processing of the acquisition unit 501, detection unit 502, processing unit 503, and fitting unit 504 of the device 500 for identifying forked intersections can be referred to... Figure 2 The corresponding steps are 201, 202, 203, and 204 in the embodiment.

[0077] In some optional implementations of this embodiment, the device 500 further includes a navigation unit (not shown in the figures), configured to: acquire navigation information; determine the relative position of the vehicle with respect to the lane lines based on the vehicle's position and the lane lines' position; determine whether the vehicle is deviating from its lane based on the relative position and the navigation information; if it is deviating, output lane change reminder information; otherwise, output straight-ahead reminder information.

[0078] In some optional implementations of this embodiment, the detection unit 502 is further configured to: input the real-time acquired image in front of the vehicle into the detection model to detect candidate areas and road signs; if the distance between the vehicle's position and the location of the intersection is greater than a predetermined value, then determine whether the candidate area is within the tolerance range by using the location of the road sign; if it is within the tolerance range, then determine the candidate area as a candidate diversion area.

[0079] In some optional implementations of this embodiment, the detection unit 502 is further configured to: input the image of the front of the vehicle acquired in real time into the detection model at the start frame of a predetermined period to obtain candidate guiding areas; and the processing unit is further configured to: if the classification result is a guiding area, record the coordinates of 3 key points on each of the left and right lanes of the guiding area as state variables.

[0080] In some optional implementations of this embodiment, the processing unit 503 is further configured to perform the following tracking steps starting from the next frame after the starting frame in each predetermined period: take the state variables recorded in the previous frame, calculate its bounding rectangle, expand the bounding rectangle and use it as the search area of ​​the current frame; input the search area of ​​the current frame into the post-processing model to obtain the classification result of the current frame and the key points of the candidate guide area of ​​the current frame; if the classification result of the current frame is a guide area, then update the state variables with the key points of the candidate guide area of ​​the current frame; otherwise, stop tracking.

[0081] In some optional implementations of this embodiment, the detection model adopts the YOLOv3 architecture, and the backbone network adopts ShuffleNet_v2.

[0082] In some optional implementations of this embodiment, the post-processing model uses the shufflenet_v2 network, which includes two branches: classification and regression.

[0083] The following is for reference. Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 1 A schematic diagram of the controller (600) in the system. Figure 6 The controller shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this disclosure.

[0084] like Figure 6 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0085] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0086] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0087] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently without being assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire the position of the vehicle and the position of the intersection, and determine whether the vehicle is within the intersection detection range based on the vehicle's position and the intersection's position; if within the intersection detection range, input the real-time acquired image of the area in front of the vehicle into a pre-trained detection model to obtain a candidate guiding region; input the candidate guiding region into a pre-trained post-processing model to obtain a classification result and key points of the candidate guiding region; if the classification result is a guiding region, fit a lane line based on the key points of the candidate guiding region.

[0088] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0090] The units described in the embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a detection unit, a processing unit, and a fitting unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit may also be described as "a unit that acquires the position of a vehicle and the position of a branch intersection, and determines whether the vehicle is within the detection range of the branch intersection based on the vehicle's position and the position of the branch intersection."

[0091] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for identifying forking intersections, comprising: Real-time location of vehicles and intersections; If the vehicle's position is less than a predetermined first distance from the intersection, the detection algorithm is invoked, and the real-time image of the front of the vehicle is input into the pre-trained detection model to obtain candidate guidance areas. The candidate diversion region is input into a pre-trained post-processing model to obtain the classification result and the key points of the candidate diversion region; If the classification result is a diversion area, then the lane line is fitted based on the key points of the candidate diversion area; If the vehicle's position is less than a predetermined second distance from the intersection, the detection algorithm is stopped, wherein the second distance is less than the first distance.

2. The method according to claim 1, wherein, The method further includes: Get navigation information; The relative position of the vehicle with respect to the lane line is determined based on the position of the vehicle and the position of the lane line. Based on the relative position and the navigation information, determine whether the vehicle is off course; If the vehicle veers off course, output a lane change reminder message; otherwise, output a straight-ahead reminder message.

3. The method according to claim 1, wherein, The step of inputting real-time acquired images of the area in front of the vehicle into the detection model to obtain candidate guidance regions includes: The real-time image of the front of the vehicle is input into the detection model to detect candidate regions and road signs; If the distance between the vehicle's position and the intersection's position is greater than a predetermined value, then the position of the road sign is used to determine whether the candidate area is within the tolerance range; If it is within the tolerance range, the candidate region is determined as the candidate diversion region.

4. The method according to claim 1, wherein, The step of inputting real-time acquired images of the area in front of the vehicle into the detection model to obtain candidate guidance regions includes: At the start frame of a predetermined period, images of the area in front of the vehicle, acquired in real time, are input into the detection model to obtain candidate guidance regions; and The step of inputting the candidate diversion region into the post-processing model to obtain the classification result and the key points of the candidate diversion region includes: If the classification result is a diversion area, then record the coordinates of 3 key points on each of the left and right lanes of the diversion area as state variables.

5. The method according to claim 4, wherein, The step of inputting the candidate diversion region into the post-processing model to obtain the classification result and the key points of the candidate diversion region includes: Within each predetermined period, starting from the frame following the initial frame, the following tracking steps are performed: Calculate the bounding rectangle of the state variables recorded in the previous frame, and expand the bounding rectangle to serve as the search area for the current frame. Input the search region of this frame into the post-processing model to obtain the classification result of this frame and the key points of the candidate diversion region of this frame; If the classification result of this frame is a guide region, then update the state variables with the key points of the candidate guide regions in this frame; otherwise, stop tracking.

6. The method according to any one of claims 1-5, wherein, The detection model adopts the YOLOv3 architecture, and the backbone network adopts ShuffleNet_v2.

7. The method according to claim 6, wherein, The post-processing model uses the shufflenet_v2 network and includes two branches: classification and regression.

8. A device for identifying forked intersections, comprising: The acquisition unit is configured to acquire the location of the vehicle and the location of the fork in the road in real time. The detection unit is configured to start calling the detection algorithm if the distance between the vehicle's position and the intersection is less than a predetermined first distance, inputting the real-time acquired image in front of the vehicle into a pre-trained detection model to obtain candidate traffic guidance areas; and to stop calling the detection algorithm if the distance between the vehicle's position and the intersection is less than a predetermined second distance, wherein the second distance is less than the first distance. The processing unit is configured to input the candidate diversion region into a pre-trained post-processing model to obtain the classification result and the key points of the candidate diversion region; The fitting unit is configured to fit a lane line based on the key points of the candidate guide area if the classification result is a guide area.

9. The apparatus according to claim 8, wherein, The device also includes a navigation unit configured to: Get navigation information; The relative position of the vehicle with respect to the lane line is determined based on the position of the vehicle and the position of the lane line. Based on the relative position and the navigation information, determine whether the vehicle is off course; If the vehicle veers off course, output a lane change reminder message; otherwise, output a straight-ahead reminder message.

10. The apparatus according to claim 8, wherein, The detection unit is further configured to: The real-time image of the front of the vehicle is input into the detection model to detect candidate regions and road signs; If the distance between the vehicle's position and the intersection's position is greater than a predetermined value, then the position of the road sign is used to determine whether the candidate area is within the tolerance range; If it is within the tolerance range, the candidate region is determined as the candidate diversion region.

11. The apparatus according to claim 8, wherein, The detection unit is further configured to: At the start frame of a predetermined period, images of the area in front of the vehicle, acquired in real time, are input into the detection model to obtain candidate guidance regions; and The processing unit is further configured to: If the classification result is a diversion area, then record the coordinates of 3 key points on each of the left and right lanes of the diversion area as state variables.

12. The apparatus according to claim 11, wherein, The processing unit is further configured to: Within each predetermined period, starting from the frame following the initial frame, the following tracking steps are performed: Calculate the bounding rectangle of the state variables recorded in the previous frame, and expand the bounding rectangle to serve as the search area for the current frame. Input the search region of this frame into the post-processing model to obtain the classification result of this frame and the key points of the candidate diversion region of this frame; If the classification result of this frame is a guide region, then update the state variables with the key points of the candidate guide regions in this frame; otherwise, stop tracking.

13. The apparatus according to any one of claims 8-12, wherein, The detection model adopts the YOLOv3 architecture, and the backbone network adopts ShuffleNet_v2.

14. The apparatus according to claim 13, wherein, The post-processing model uses the shufflenet_v2 network and includes two branches: classification and regression.

15. An electronic device for identifying forked intersections, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

16. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images