Lane confluence detection method, device, terminal, storage medium and product

Abstract

The present application provides a method, device, terminal, storage medium, and product for detecting the confluence of a lane, belonging to the field of autonomous driving technology. The method includes: acquiring a first image, where the first image is captured based on a camera device mounted on a vehicle, and the first image is captured with the camera device facing the direction of travel of the vehicle; identifying at least one first lane line and at least one second lane line from the first image, where the first lane line is the lane line of the target lane currently located by the vehicle, and the second lane line is the lane line adjacent to the first lane line; and determining detection information of the confluence of the target lane based on the at least one first lane line and the at least one second lane line. Since the first image is captured based on a camera device mounted on the vehicle, the first lane line and the second lane line in the first image are consistent with the lane lines on the actual road, thereby improving the accuracy of the detection information of the confluence of the target lane.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method, device, terminal, storage medium, and product for determining lane entrance information. Background Art

[0002] In order to facilitate the merging or exiting of vehicles, merging entrances are often set at certain locations in the lane. However, other vehicles merging in or out of the lane through the merging entrance may pose a safety hazard to the current vehicle. Therefore, during the autonomous driving process, the vehicle needs to detect the merging entrance of the lane where the vehicle is located, and then adjust the vehicle's driving behavior according to the lane merging entrance.

[0003] In related technologies, by locating the current position of the vehicle and based on the map data stored in the vehicle, the detection information of the confluence entrance of the lane where the vehicle is currently located is determined. The detection information is used to indicate whether there is a confluence entrance at that position of the lane or whether there is a confluence entrance within a preset distance after that position.

[0004] However, since map data is not updated in real time, some newly built confluence points cannot be updated to the map data in a timely manner, resulting in the confluence points detected by the above method being inconsistent with the actual confluence points. Therefore, the confluence points detected by the above method have poor accuracy. Summary of the Invention

[0005] The present invention provides a method, device, terminal, storage medium, and product for detecting lane confluences, which can improve the accuracy of lane confluence detection. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, a method for detecting a lane confluence is provided, the method comprising:

[0007] Acquire a first image, where the first image is captured based on a camera installed on a vehicle, and the first image is captured by the camera facing the direction of travel of the vehicle;

[0008] Identifying at least one first lane line and at least one second lane line from the first image, where the first lane line is a lane line of a target lane where the vehicle is currently located, and the second lane line is a lane line adjacent to the first lane line;

[0009] Based on the at least one first lane marking and the at least one second lane marking, detection information of a confluence entrance of the target lane is determined.

[0010] In one possible implementation, determining the detection information of the confluence entrance of the target lane based on the at least one first lane marking and the at least one second lane marking includes:

[0011] For each adjacent pair of a first lane line and a second lane line, determining a positional relationship between the first lane line and the second lane line;

[0012] Based on the positional relationship between each pair of first lane lines and second lane lines, detection information of the confluence entrance of the target lane is determined.

[0013] In another possible implementation, the positional relationship includes intersection and non-intersection; and determining the positional relationship between the first lane line and the second lane line includes:

[0014] determining a lateral distance difference between the first lane marking and the second lane marking;

[0015] If the lateral distance difference is not greater than a preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is an intersection;

[0016] If the lateral distance difference is greater than the preset lateral distance difference threshold, it is determined that the positional relationship between the first lane line and the second lane line is non-intersecting.

[0017] In another possible implementation, determining the positional relationship between the first lane marking and the second lane marking includes:

[0018] respectively determining first position information and second position information of the first lane line and the second lane line in a first coordinate system corresponding to the camera device;

[0019] Performing coordinate conversion on the first position information and the second position information to obtain third position information and fourth position information of the first lane line and the second lane line in a second coordinate system corresponding to the inertial measurement sensor of the vehicle;

[0020] Based on the third position information and the fourth position information, a positional relationship between the first lane line and the second lane line is determined.

[0021] In another possible implementation, determining the detection information of the confluence entrance of the target lane based on the positional relationship between each pair of first lane markings and second lane markings includes:

[0022] If there are a first lane line and a second lane line that are in a positional relationship of intersection, determining that the detection information indicates that the target lane includes a confluence entrance;

[0023] If there is no first lane line and second lane line that are in a positional relationship of intersection, it is determined that the detection information indicates that the target lane does not include a merging entrance.

[0024] In another possible implementation, the method further includes:

[0025] If the detection information indicates that the target lane includes a confluence, determining lane line types of the multiple lane line segments included in the at least one first lane line;

[0026] For each lane line segment, determining an intersection point between the lane line segment and the second lane line to obtain at least one intersection point;

[0027] Based on the lane line types of the multiple lane line segments and the at least one intersection point, a starting point and an ending point of the confluence entrance are determined.

[0028] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and the at least one intersection point is multiple;

[0029] The determining the starting point and the ending point of the confluence entrance based on the lane line types of the plurality of lane line segments and the at least one intersection point includes:

[0030] Based on the lane line types of the multiple lane line segments, the intersection point between the solid lane line and the second lane line is determined from the at least one intersection point as the starting point of the confluence entrance, and the intersection point between the dashed lane line and the second lane line is determined as the ending point of the confluence entrance.

[0031] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and the at least one intersection is one;

[0032] The determining the starting point and the ending point of the confluence entrance based on the lane line types of the plurality of lane line segments and the at least one intersection point includes:

[0033] Based on the lane line types of the multiple lane line segments, the starting point of the dotted lane line is determined as the starting point of the confluence entrance, and the intersection between the dotted lane line and the second lane line is determined as the end point of the confluence entrance.

[0034] In another possible implementation, the method further includes:

[0035] When the vehicle is traveling on the target lane, if the second image acquired next time does not include the starting point of the confluence entrance, the detection information is maintained as indicating that the target lane includes the confluence entrance.

[0036] In another possible implementation, the method further includes:

[0037] If the second image acquired next time includes the end point of the confluence entrance, the detection information is adjusted to indicate that the target lane does not include the confluence entrance; or

[0038] If it is determined that the target lane is the rightmost lane of the road, the detection information is adjusted so that the target lane does not include a merging entrance; or

[0039] If the vehicle changes lanes from the target lane to another lane, the detection information is adjusted so that the target lane does not include a merging entrance.

[0040] In another possible implementation, the first image is captured by the camera device during the automatic driving of the vehicle; and the method further includes:

[0041] determining driving behavior information of the vehicle based on the detection information;

[0042] Based on the driving behavior information, the vehicle is controlled to continue automatic driving.

[0043] According to another aspect of an embodiment of the present application, a device for detecting a lane confluence is provided, the device comprising:

[0044] an acquisition module, configured to acquire a first image, wherein the first image is acquired based on an imaging device installed on a vehicle, and the first image is acquired by the imaging device facing a traveling direction of the vehicle;

[0045] a recognition module, configured to recognize at least one first lane line and at least one second lane line from the first image, wherein the first lane line is a lane line of a target lane currently located by the vehicle, and the second lane line is a lane line adjacent to the first lane line;

[0046] The first determination module is configured to determine detection information of a confluence entrance of the target lane based on the at least one first lane line and the at least one second lane line.

[0047] In a possible implementation, the first determining module includes:

[0048] a first determining unit, configured to determine, for each adjacent pair of a first lane line and a second lane line, a positional relationship between the first lane line and the second lane line;

[0049] The second determining unit is configured to determine detection information of the confluence entrance of the target lane based on a positional relationship between each pair of the first lane markings and the second lane markings.

[0050] In another possible implementation, the positional relationship includes intersection and non-intersection; the first determination unit is used to determine the lateral distance difference between the first lane line and the second lane line; if the lateral distance difference is not greater than a preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be intersection; if the lateral distance difference is greater than the preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be non-intersection.

[0051] In another possible implementation, the first determination unit is used to respectively determine the first position information and the second position information of the first lane line and the second lane line in the first coordinate system corresponding to the camera device; perform coordinate conversion on the first position information and the second position information to obtain third position information and fourth position information of the first lane line and the second lane line in the second coordinate system corresponding to the inertial measurement sensor of the vehicle; and determine the positional relationship between the first lane line and the second lane line based on the third position information and the fourth position information.

[0052] In another possible implementation, the second determination unit is configured to determine that the detection information indicates that the target lane includes a confluence if there is a first lane line and a second lane line that intersect in a positional relationship; and to determine that the detection information indicates that the target lane does not include a confluence if there is no first lane line and a second lane line that intersect in a positional relationship.

[0053] In another possible implementation, the apparatus further includes:

[0054] The second determination module is used to determine the lane line type of multiple lane line segments included in the at least one first lane line if the detection information indicates that the target lane includes a confluence; for each lane line segment, determine the intersection between the lane line segment and the second lane line to obtain at least one intersection; and determine the starting point and end point of the confluence based on the lane line types of the multiple lane line segments and the at least one intersection.

[0055] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and there are multiple at least one intersection point; the second determination module is used to determine, based on the lane line types of the multiple lane line segments, the intersection point between the solid lane line and the second lane line from the at least one intersection point as the starting point of the confluence entrance, and determine the intersection point between the dashed lane line and the second lane line as the end point of the confluence entrance.

[0056] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and the at least one intersection is one; the second determination module is used to determine the starting point of the dashed lane line as the starting point of the confluence based on the lane line types of the multiple lane line segments, and determine the intersection between the dashed lane line and the second lane line as the end point of the confluence.

[0057] In another possible implementation, the first determination module is further configured to maintain the detection information that the target lane includes the confluence entrance if the second image acquired next time does not include the starting point of the confluence entrance during the process of the vehicle traveling on the target lane.

[0058] In another possible implementation, the apparatus further includes:

[0059] An adjustment module is configured to adjust the detection information to indicate that the target lane does not include the merging entrance if the end point of the merging entrance is included in the second image acquired next time; or, an adjustment module is configured to adjust the detection information to indicate that the target lane does not include the merging entrance if the target lane is determined to be the rightmost lane of the road; or, an adjustment module is configured to adjust the detection information to indicate that the target lane does not include the merging entrance if the vehicle changes lanes from the target lane to another lane.

[0060] In another possible implementation, the first image is captured by the camera during the automatic driving of the vehicle; and the apparatus further includes:

[0061] a third determining module, configured to determine driving behavior information of the vehicle based on the detection information;

[0062] A control module is used to control the vehicle to continue automatic driving based on the driving behavior information.

[0063] According to another aspect of an embodiment of the present application, a terminal is provided, comprising: a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the method for detecting the confluence entrance of a lane as described in any of the above-mentioned implementation methods.

[0064] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded by a processor and has a method for detecting the confluence entrance of a lane described in any of the above-mentioned implementation methods.

[0065] According to another aspect of an embodiment of the present application, a computer program product is provided, which includes at least one program code, and the at least one program code is loaded and executed by a processor to implement the method for detecting the confluence entrance of a lane as described in any of the above implementation methods.

[0066] The beneficial effects of the technical solutions provided by the embodiments of the present application include at least:

[0067] An embodiment of the present application provides a method for detecting the confluence of a lane. In this method, detection information of the confluence of a target lane is determined based on a first lane line and a second lane line in a first image. The first image is captured based on a camera installed on a vehicle. Therefore, the first lane line and the second lane line in the first image are consistent with the lane lines in the actual road, thereby improving the accuracy of the detection information of the confluence of the target lane. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0070] Figure 2 This is a flow chart of a method for detecting a lane confluence provided by an embodiment of the present application;

[0071] Figure 3 This is a flow chart of a method for detecting a lane confluence provided by an embodiment of the present application;

[0072] Figure 4 This is a schematic diagram of a lane confluence entrance provided by an embodiment of the present application;

[0073] Figure 5 is a schematic diagram of another lane confluence entrance provided in an embodiment of the present application;

[0074] Figure 6 1 is a schematic structural diagram of a detection device for a lane confluence provided in an embodiment of the present application;

[0075] Figure 7 1 is a schematic structural diagram of another lane confluence detection device provided in an embodiment of the present application;

[0076] Figure 8 This is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0078] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 The implementation environment includes a vehicle 101, a terminal 102, and a camera 103. Terminal 102 is located inside the vehicle, and camera 103 is located on top of the vehicle. Terminal 102 and camera 103 can be connected via a wired or wireless network. Camera 103 can transmit captured images or videos to terminal 102. Terminal 102 is configured to identify at least one lane in the image or video, and determine the confluence point of the vehicle's lane based on the at least one lane in the image or video.

[0079] It should be noted that the number of camera devices 103 can be one or more. In one possible implementation, the number of camera devices 103 is three, including: a left-view camera device, a front-view camera device, and a right-view camera device. Optionally, the three camera devices 103 are set on the front windshield of the vehicle 101; wherein the left-view camera device is set on the left side of the front windshield, the front-view camera device is set in the middle of the front windshield, and the right-view camera device is set on the right side of the front windshield.

[0080] Optional, see Figure 1 The implementation environment further includes an inertial measurement sensor 104 ; the inertial measurement sensor 104 is disposed inside the vehicle body. The terminal 102 and the inertial measurement sensor 104 may be connected via a wired or wireless network; the inertial measurement sensor 104 may transmit the vehicle's location information to the terminal 102 .

[0081] The lane confluence detection method in the embodiment of the present application can be widely used in various practical application scenarios:

[0082] For example, in the detection scenario of the overpass confluence, the camera device 103 captures images or videos near the overpass confluence and transmits the captured images or videos to the terminal 102; the terminal 102 is used to determine at least one lane in the image or video, and determine the overpass confluence of the lane where the vehicle is located through at least one lane in the image or video.

[0083] For another example, in a confluence detection scenario when the number of lanes is reduced, the camera device 103 captures images or videos near the confluence and transmits the captured images or videos to the terminal 102; the terminal 102 is used to determine at least one lane in the image or video, and determine the confluence of the lane where the vehicle is located through at least one lane in the image or video.

[0084] Figure 2 This is a flow chart of a method for detecting a lane confluence provided by an embodiment of the present application. Figure 2 , the method comprises the following steps:

[0085] 201. Acquire a first image, where the first image is captured by a camera installed on a vehicle, and the first image is captured by the camera facing a traveling direction of the vehicle.

[0086] 202. Identify at least one first lane line and at least one second lane line from the first image, where the first lane line is a lane line of a target lane where the vehicle is currently located, and the second lane line is a lane line adjacent to the first lane line.

[0087] 203. Determine detection information of a confluence entrance of a target lane based on at least one first lane marking and at least one second lane marking.

[0088] In one possible implementation, determining the detection information of the confluence entrance of the target lane based on the at least one first lane marking and the at least one second lane marking includes:

[0089] For each adjacent pair of first lane lines and second lane lines, determining a positional relationship between the first lane line and the second lane line;

[0090] Based on the positional relationship between each pair of first lane markings and second lane markings, detection information of a confluence entrance of the target lane is determined.

[0091] In another possible implementation, the positional relationship includes intersection and non-intersection; and determining the positional relationship between the first lane line and the second lane line includes:

[0092] determining a lateral distance difference between the first lane marking and the second lane marking;

[0093] If the lateral distance difference is not greater than a preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is an intersection;

[0094] If the lateral distance difference is greater than a preset lateral distance difference threshold, it is determined that the positional relationship between the first lane line and the second lane line is non-intersecting.

[0095] In another possible implementation, determining the positional relationship between the first lane marking and the second lane marking includes:

[0096] respectively determining first position information and second position information of the first lane line and the second lane line in a first coordinate system corresponding to the camera device;

[0097] Performing coordinate conversion on the first position information and the second position information to obtain third position information and fourth position information of the first lane line and the second lane line in a second coordinate system corresponding to the inertial measurement sensor of the vehicle;

[0098] Based on the third position information and the fourth position information, a positional relationship between the first lane line and the second lane line is determined.

[0099] In another possible implementation, determining detection information of a confluence entrance of a target lane based on a positional relationship between each pair of first lane markings and second lane markings includes:

[0100] If there are a first lane line and a second lane line that are in a positional relationship of intersection, determining that the detection information is the target lane including the confluence entrance;

[0101] If there is no first lane line and second lane line that are in a positional relationship of intersection, it is determined that the detection information indicates that the target lane does not include a merging entrance.

[0102] In another possible implementation, the method further includes:

[0103] If the detection information indicates that the target lane includes a confluence, determining lane line types of a plurality of lane line segments included in at least one first lane line;

[0104] For each lane segment, determining an intersection point between the lane segment and a second lane line to obtain at least one intersection point;

[0105] A starting point and an ending point of a confluence entrance are determined based on lane line types of the plurality of lane line segments and at least one intersection point.

[0106] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and at least one intersection point is multiple;

[0107] Determining a start point and an end point of a confluence entrance based on lane line types of multiple lane line segments and at least one intersection point includes:

[0108] Based on the lane line types of multiple lane line segments, the intersection point between the solid lane line and the second lane line is determined from at least one intersection point as the starting point of the confluence entrance, and the intersection point between the dashed lane line and the second lane line is determined as the ending point of the confluence entrance.

[0109] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and at least one intersection point is one;

[0110] Determining a start point and an end point of a confluence entrance based on lane line types of multiple lane line segments and at least one intersection point includes:

[0111] Based on the lane line types of the multiple lane line segments, a starting point of the dashed lane line is determined as a starting point of the confluence entrance, and an intersection point between the dashed lane line and the second lane line is determined as an end point of the confluence entrance.

[0112] In another possible implementation, the method further includes:

[0113] When the vehicle is traveling in the target lane, if the second image acquired next time does not include the starting point of the confluence entrance, the detection information is maintained as the target lane including the confluence entrance.

[0114] In another possible implementation, the method further includes:

[0115] If the second image acquired next time includes the end point of the confluence entrance, the detection information is adjusted to the target lane not including the confluence entrance; or

[0116] If the target lane is determined to be the rightmost lane of the road, the detection information is adjusted so that the target lane does not include the merging entrance; or

[0117] If the vehicle changes lanes from the target lane to another lane, the detection information is adjusted so that the target lane does not include the merging entrance.

[0118] In another possible implementation, the first image is captured by a camera device during the automatic driving of the vehicle; and the method further includes:

[0119] Determining driving behavior information of the vehicle based on the detection information;

[0120] Based on the driving behavior information, the vehicle is controlled to continue autonomous driving.

[0121] Figure 3 This is a flow chart of a method for detecting a lane confluence provided by an embodiment of the present application, which is executed by a terminal. Figure 3 , the method comprises the following steps:

[0122] 301. A terminal obtains a first image, where the first image is captured by a camera installed on a vehicle, and the first image is captured by the camera facing a traveling direction of the vehicle.

[0123] In this step, the camera device can be installed at any position of the vehicle. For example, the camera device can be installed on the top of the vehicle.

[0124] In one possible implementation, a camera captures video in real time while the vehicle is traveling. Accordingly, this step involves: the camera captures video in real time and transmits the captured video to a terminal; the terminal receives the video and determines, from the received video, the video frame image corresponding to the current moment as the first image.

[0125] In an embodiment of the present application, since the first image is a frame image extracted from a video, and the video is acquired in real time by a camera device, it is ensured that the first image can be updated in a timely manner to be consistent with the actual scene, thereby improving the accuracy of the acquired first image.

[0126] In another possible implementation, while the vehicle is traveling, the camera device captures images according to a preset period. Accordingly, this step is as follows: the camera device captures images according to a preset period and transmits the captured images to the terminal; the terminal receives the image and determines that the most recently received image is the first image. In the embodiment of the present application, the duration of the preset period is not specifically limited and can be set and modified as needed. Optionally, the duration of the preset period is any value between 0.1s and 1s, for example, the duration of the preset period is 0.1s, 0.2s, 0.3s, etc.

[0127] In the embodiment of the present application, since the terminal directly receives the image captured by the camera device and the image occupies a small amount of memory space, the operating speed of the terminal is improved.

[0128] It should be noted that the vehicle driving process includes any moment between the time the camera is turned on and the time the camera is turned off. In one possible implementation, the terminal turns on the camera when the vehicle starts and turns off the camera when the vehicle is turned off. In another possible implementation, the terminal turns on the camera when the vehicle departs from the route starting point along a preset route and turns off the camera when the vehicle reaches the route end.

[0129] 302. The terminal identifies at least one first lane line and at least one second lane line from the first image, where the first lane line is a lane line of a target lane where the vehicle is currently located, and the second lane line is a lane line adjacent to the first lane line.

[0130] In one possible implementation, the terminal clusters lane lines in the first image to identify at least one first lane line and at least one second lane line. Accordingly, this step involves the terminal acquiring a region of interest in the first image and clustering the lane lines in the region of interest to obtain at least one first lane line and at least one second lane line.

[0131] It should be noted that the first lane line is the lane line of the target lane where the vehicle is currently located. The lane lines of the target lane include the left lane line and the right lane line; at least one first lane line includes at least one of the left lane line and the right lane line. When the first lane line is the left lane line, the second lane line is the adjacent lane line of the left lane line; when the first lane line is the right lane line, the second lane line is the adjacent lane line of the right lane line. When the first lane line is the left lane line and the right lane line, the second lane line includes the adjacent lane lines of the left lane line and the adjacent lane lines of the right lane line. For example, see Figure 4 , when the first lane line is the right lane line, the second lane line is the adjacent lane line of the right lane line.

[0132] 303. The terminal determines, for each adjacent pair of the first lane line and the second lane line, a positional relationship between the first lane line and the second lane line.

[0133] In one possible implementation, the terminal determines the positional relationship between the first lane line and the second lane line based on the lateral distance difference between the first lane line and the second lane line. Accordingly, this step is: the terminal determines the lateral distance difference between the first lane line and the second lane line; if the lateral distance difference is not greater than a preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be intersecting; if the lateral distance difference is greater than the preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be non-intersecting. In the embodiment of the present application, the size of the preset lateral distance difference threshold is not specifically limited and can be set and modified as needed. Optionally, the preset lateral distance difference threshold is any value between 0m and 2.5m, for example, the preset lateral distance difference threshold is 1m, 1.5m, 2m, etc.

[0134] In an embodiment of the present application, the positional relationship between the first lane line and the second lane line is determined by the lateral distance difference between the two lane lines. When there is a confluence at the target lane, the lateral distance difference between the first lane line and the second lane line will be less than a preset lateral distance difference threshold, thereby improving the accuracy of the determined positional relationship.

[0135] In another possible implementation, the terminal determines the positional relationship between the first lane line and the second lane line based on the position information of the first lane line and the second lane line in the second coordinate system corresponding to the inertial measurement sensor. Accordingly, this step includes: the terminal respectively determines first position information and second position information of the first lane line and the second lane line in the first coordinate system corresponding to the camera device; performs coordinate conversion on the first position information and the second position information to obtain third position information and fourth position information of the first lane line and the second lane line in the second coordinate system corresponding to the inertial measurement sensor of the vehicle; and determines the positional relationship between the first lane line and the second lane line based on the third position information and the fourth position information.

[0136] Optionally, the first coordinate system corresponding to the camera device is an image coordinate system; the second coordinate system corresponding to the inertial measurement sensor is an inertial measurement sensor coordinate system, that is, an IMU (Inertial Measurement Unit) coordinate system. The first position information includes multiple coordinates corresponding to the first lane line in the image coordinate system, and the second position information includes multiple coordinates corresponding to the second lane line in the image coordinate system. It should be noted that the inertial measurement sensor can be installed at any position of the vehicle, for example, at the front end of the vehicle; wherein the inertial measurement sensor can detect the vehicle's position information in real time and will not be affected by wireless signals.

[0137] In one possible implementation, the terminal determines the positional relationship between the first lane line and the second lane line based on the third position information and the fourth position information as follows: the terminal determines the lateral distance difference between the first lane line and the second lane line based on the third position information and the fourth position information; if the lateral distance difference is not greater than a preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be intersecting; if the lateral distance difference is greater than the preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be non-intersecting. Optionally, the third position information includes multiple coordinates corresponding to the first lane line in the second coordinate system, and the fourth position information includes multiple coordinates corresponding to the second lane line in the second coordinate system.

[0138] In an embodiment of the present application, the positional relationship between the first lane line and the second lane line is determined based on the position information of the first lane line and the second lane line in the second coordinate system corresponding to the inertial measurement sensor, and the inertial measurement sensor can detect the position information of the vehicle in real time and is not affected by wireless signals, thereby ensuring the validity of the converted position information, and thus improving the stability of determining the positional relationship between the first lane line and the second lane line.

[0139] In another possible implementation, the terminal determines the positional relationship between the first lane line and the second lane line based on a changing trend of a lateral distance difference between the first lane line and the second lane line. Accordingly, in this step, the terminal determines a changing trend of the lateral distance difference between the first lane line and the second lane line, and if the changing trend of the lateral distance difference is gradually decreasing.

[0140] Optionally, the terminal determines a first position point and a second position point in front of the vehicle, wherein the interval between the first position point and the second position point is a second preset distance, and the first position point is a position point closer to the vehicle, and the second position point is a position point farther from the vehicle; the terminal determines the difference between the lateral distance difference corresponding to the first position point and the lateral distance difference corresponding to the second position point; if the difference is greater than the preset difference, the trend of the lateral distance difference is determined to be gradually decreasing, and the positional relationship between the first lane line and the second lane line is determined to be an intersection. In this embodiment of the present application, the size of the second preset distance and the preset difference is not specifically limited and can be set and modified as needed. Optionally, the second preset distance is any value between 5m and 50m, for example, the second preset distance is 10, 15m, 20m, etc. Optionally, the preset difference is any value between 0.5m and 2m, for example, the preset difference is 0.5m, 1m, 1.5m, etc.

[0141] In an embodiment of the present application, the positional relationship between the first lane line and the second lane line is determined based on the changing trend of the lateral distance difference between the first lane line and the second lane line, and the changing trend of the lateral distance difference can reflect the directions of the first lane line and the second lane line, so the accuracy of the determined positional relationship is improved.

[0142] 304. The terminal determines detection information of a confluence entrance of the target lane based on a positional relationship between each pair of first lane lines and second lane lines.

[0143] In this step, the merging entrance is used to indicate the entrance of other vehicles in the merging lane from the merging lane to the target lane. The merging entrance includes the merging starting point, the merging ending point, and the lane line between the merging starting point and the merging ending point, that is, the lane line of the merging lane.

[0144] In one possible implementation, the detection information includes whether the target lane includes a confluence entrance and whether the target lane does not include a confluence entrance. Accordingly, in this step, if the first lane line and the second lane line intersect, the terminal determines that the detection information indicates that the target lane includes a confluence entrance; if the first lane line and the second lane line do not intersect, the terminal determines that the detection information indicates that the target lane does not include a confluence entrance.

[0145] In the embodiment of the present application, whether the target lane includes a confluence entrance is determined by the positional relationship between the first lane line and the second lane line, and the positional relationship between the first lane line and the second lane line can reflect the detection information of the actual confluence entrance, thereby improving the accuracy of the detection information of the determined confluence entrance.

[0146] In another possible implementation, the terminal determines the detection information of the confluence of the target lane based on multiple detection information. Accordingly, this step is: the terminal obtains multiple first images captured within a preset time length, and if for each first image, there is a first lane line and a second lane line with an intersecting positional relationship, it is determined that the detection information is that the target lane includes the confluence. In the embodiment of the present application, the preset time length is not specifically limited and can be set and modified as needed. Optionally, the preset time length is any value between 0.1s and 2s, for example, the preset time length is 0.1s, 0.5s, 1s, etc.

[0147] In an embodiment of the present application, since the terminal determines the detection information of the confluence entrance of the target lane based on the detection information of multiple confluence entrances, it avoids the situation where a single determined detection result has errors and is inaccurate, thereby improving the accuracy of the determined confluence entrance detection information.

[0148] It should be noted that if the terminal determines that the target lane includes a confluence, the terminal can also detect the starting and ending points of the confluence. Accordingly, the terminal detects the starting and ending points of the confluence as follows: if the terminal detects that the target lane includes a confluence, the terminal determines the lane line type of multiple lane line segments included in at least one first lane line; for each lane line segment, the terminal determines the intersection between the lane line segment and the second lane line to obtain at least one intersection point; and based on the lane line types of the multiple lane line segments and the at least one intersection point, the terminal determines the starting and ending points of the confluence. Optionally, the lane line type includes a dashed lane line and a solid lane line.

[0149] In one possible implementation, the lane segment intersects with the second lane line, and the intersection point between the lane segment and the second lane line is determined to be the intersection point. In another possible implementation, the lane segment does not intersect with the second lane line, but the distance between the endpoint of the lane segment and the endpoint of the second lane line is less than a third preset distance, and the intersection point between the extension line of the lane segment and the extension line of the second lane line is determined to be the intersection point. In the embodiment of the present application, the size of the third preset distance is not specifically limited and can be set and modified as needed. Optionally, the third preset distance is any value between 0m and 0.5m, for example, the third preset distance is 0.1m, 0.2m, 0.3m, etc.

[0150] In an embodiment of the present application, since the terminal determines that the detection information shows that the target lane includes a confluence entrance, the terminal can also detect the starting point and end point of the confluence entrance, so that the detection information includes the starting point and end point information of the confluence entrance, thereby increasing the amount of information contained in the detection information, and thus improving the accuracy of the determined detection information.

[0151] In one possible implementation, a merging entrance corresponds to a merging lane. Before the target lane intersects the merging lane, the target lane and the merging lane are non-parallel, and the first lane line includes multiple lane segments with multiple intersection points with the second lane line. Accordingly, the terminal determines the starting point and ending point of the merging entrance based on the lane line types of the multiple lane line segments and at least one intersection point. The terminal determines, based on the lane line types of the multiple lane line segments, the intersection point between the solid lane line and the second lane line from the at least one intersection point as the starting point of the merging entrance, and determines the intersection point between the dashed lane line and the second lane line as the ending point of the merging entrance.

[0152] One thing to note is that, continue to refer to Figure 4 , the second lane line is the adjacent lane line of the first lane line, and the merging lane corresponding to the merging entrance includes the left lane line and the right lane line. Before the target lane intersects the merging lane, the second lane line is the left lane line of the merging lane, and the third lane line is the right lane line of the merging lane; that is, the intersection point between the solid lane line and the second lane line is the intersection point between the solid lane line and the left lane line of the merging lane. After the target lane intersects the merging lane, the left lane line of the merging lane coincides with the first lane line. At this time, the adjacent lane line of the first lane line is the right lane line of the merging lane, and the third lane line coincides with the second lane line. That is, the intersection point between the dashed lane line and the second lane line is the intersection point between the dashed lane line and the right lane line of the merging lane.

[0153] In the embodiment of the present application, since the first lane line is a dotted lane line in the confluence area and a solid lane line outside the confluence area, the intersection point between the solid lane line and the second lane line is consistent with the actual starting point of the confluence, and the intersection point between the dotted lane line and the second lane line is consistent with the actual ending point of the confluence, thereby improving the accuracy of the determined starting point and ending point of the confluence.

[0154] In another possible implementation, see Figure 5Before the target lane intersects the merging lane, the target lane and the merging lane are parallel, the first lane line is a dashed lane line in the merging entrance area, and the first lane line has a single intersection point with the second lane line. Accordingly, the terminal determines the starting point and ending point of the merging entrance based on the lane line types and at least one intersection point of the multiple lane line segments: the terminal determines, based on the lane line types of the multiple lane line segments, the starting point of the dashed lane line as the starting point of the merging entrance, and determines the intersection point between the dashed lane line and the second lane line as the ending point of the merging entrance.

[0155] In the embodiment of the present application, since the starting point of the dotted lane line is determined as the starting point of the confluence, and the intersection between the dotted lane line and the second lane line is determined as the end point of the confluence, when the number of lanes becomes smaller, the terminal can also determine the starting point and end point of the confluence of the target lane, thereby improving the accuracy of the determined detection information.

[0156] Another point that needs to be explained is that when the vehicle is traveling on the target lane, if the terminal determines in the detection information that the target lane includes the confluence entrance, the terminal can also track and detect the confluence entrance and update the detection information in real time; after the vehicle passes the confluence entrance, that is, when the target lane does not include the confluence entrance, the terminal adjusts the detection information to indicate that the target lane does not include the confluence entrance. Among them, one confluence entrance corresponds to one merging lane, and the process of the terminal tracking and detecting the confluence entrance is also the process of tracking and detecting the merging lane corresponding to the merging entrance. In an embodiment of the present application, the terminal tracks the merging lane by detecting the starting point and the end point of the confluence entrance.

[0157] In one possible implementation, the merging lane is long, and the distance between the merging entrance's starting point and its ending point is relatively far. As the vehicle travels from the merging entrance's starting point to the merging entrance's ending point, the acquired image may not include the merging entrance's starting point. Accordingly, the specific steps for tracking the merging entrance are as follows: while the vehicle is traveling in the target lane, if the next acquired second image does not include the merging entrance's starting point, maintain the detection information that the target lane includes the merging entrance.

[0158] In an embodiment of the present application, since when the distance to the merging lane is long, even if the image does not include the starting point of the merging entrance, the detection information can continue to be maintained as the target lane including the merging entrance, so that the determined detection information can be consistent with the actual situation, thereby improving the accuracy of the determined detection information.

[0159] After the vehicle passes through the confluence, the terminal can adjust the detection information to indicate that the target lane does not include the confluence. In one possible implementation, when the vehicle travels from the confluence's starting point to the confluence's ending point, if the image includes the confluence's ending point, the vehicle is about to pass through the confluence. Accordingly, the terminal adjusts the detection information as follows: if the next acquired second image includes the confluence's ending point, the terminal adjusts the detection information to indicate that the target lane does not include the confluence.

[0160] In an embodiment of the present application, since the vehicle is about to pass through the confluence when the end point of the confluence is included in the second image, the terminal adjusts the detection information so that the target lane does not include the confluence, thereby realizing real-time updating of the detection information, thereby improving the accuracy of the determined detection information.

[0161] In another possible implementation, when traffic volume is high, the end point of the confluence may be obscured by moving vehicles, resulting in a vehicle passing through the confluence without detecting the end point. In this case, the terminal adjusts the detection information based on the location of the target lane. Optionally, the terminal adjusts the detection information as follows: if the target lane is determined to be the rightmost lane of the road, the terminal adjusts the detection information so that the target lane does not include the confluence.

[0162] In one possible implementation, the terminal determines whether the target lane is the rightmost lane of the road based on the lateral distance between the target lane and the curb. Accordingly, the specific steps are: the terminal determines the lateral distance between the first lane line of the target lane and the curb; if the lateral distance between the first lane line and the curb is not greater than the fourth preset distance, the target lane is determined to be the rightmost lane of the road; if the lateral distance between the first lane line and the curb is greater than the fourth preset distance, the target lane is determined not to be the rightmost lane of the road. In this embodiment of the present application, the size of the fourth preset distance is not specifically limited and can be set and modified as needed. Optionally, the fourth preset distance is any value between 1m and 3m, for example, the fourth preset distance is 1m, 1.5m, 2.5m, etc.

[0163] In an embodiment of the present application, since when the end point of the confluence entrance is blocked by a moving vehicle, the terminal can adjust the detection information according to the position of the target lane, thereby realizing real-time updating of the detection information, thereby improving the accuracy of the determined detection information.

[0164] In another possible implementation, if the target lane includes a merging entrance, the vehicle may change lanes from the target lane to another lane without a merging entrance to avoid the impact of merging vehicles. Accordingly, the terminal adjusts the detection information as follows: if the vehicle changes lanes from the target lane to another lane, the terminal adjusts the detection information to indicate that the target lane does not include a merging entrance.

[0165] In an embodiment of the present application, when a vehicle changes lanes from a target lane to another lane without a merging entrance, the detection information is adjusted so that the target lane does not include a merging entrance, thereby achieving real-time updating of the detection information, thereby improving the accuracy of the determined detection information.

[0166] 305. The terminal determines the driving behavior information of the vehicle based on the detection information; and controls the vehicle to continue automatic driving based on the driving behavior information.

[0167] In one possible implementation, the driving behavior information includes vehicle speed information. Accordingly, the terminal determines the vehicle's driving behavior information based on the detection information by: if the detection information indicates that the target lane does not include a merge entrance, determining the vehicle's speed information to be a first preset speed; if the detection information indicates that the target lane includes a merge entrance, determining the vehicle's speed information to be a second preset speed. In this embodiment of the present application, the values of the first preset speed and the second preset speed are not specifically limited and can be set and modified as needed.

[0168] In one possible implementation, the driving behavior information includes vehicle speed information. Accordingly, the terminal controls the vehicle to continue autonomous driving based on the driving behavior information by adjusting the vehicle's speed to the first preset speed if the vehicle's speed information is a first preset speed; and adjusting the vehicle's speed to the second preset speed if the vehicle's speed information is a second preset speed.

[0169] An embodiment of the present application provides a method for detecting the confluence of a lane. In this method, detection information of the confluence of a target lane is determined based on a first lane line and a second lane line in a first image. The first image is captured based on a camera installed on a vehicle. Therefore, the first lane line and the second lane line in the first image are consistent with the lane lines in the actual road, thereby improving the accuracy of the detection information of the confluence of the target lane.

[0170] Figure 6 This is a schematic diagram of the structure of a detection device for a lane confluence provided by an embodiment of the present application. Figure 6 , the device comprises:

[0171] An acquisition module 601 acquires a first image, where the first image is acquired by a camera installed on a vehicle and the camera is facing the direction of travel of the vehicle.

[0172] A recognition module 602 is configured to recognize at least one first lane line and at least one second lane line from the first image, wherein the first lane line is a lane line of a target lane currently located by the vehicle, and the second lane line is a lane line adjacent to the first lane line;

[0173] The first determination module 603 is configured to determine detection information of a confluence entrance of a target lane based on at least one first lane marking and at least one second lane marking.

[0174] In one possible implementation, see Figure 7 The first determining module 603 includes:

[0175] A first determining unit 6031 is configured to determine, for each adjacent pair of first lane lines and second lane lines, a positional relationship between the first lane line and the second lane line;

[0176] The second determining unit 6032 is configured to determine detection information of a confluence entrance of the target lane based on a positional relationship between each pair of first lane markings and second lane markings.

[0177] In another possible implementation, the positional relationship includes intersection and non-intersection; the first determination unit 6031 is used to determine the lateral distance difference between the first lane line and the second lane line; if the lateral distance difference is not greater than a preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be intersection; if the lateral distance difference is greater than the preset lateral distance difference threshold, the positional relationship between the first lane line and the second lane line is determined to be non-intersection.

[0178] In another possible implementation, the first determination unit 6031 is used to respectively determine the first position information and the second position information of the first lane line and the second lane line in the first coordinate system corresponding to the camera device; perform coordinate conversion on the first position information and the second position information to obtain the third position information and the fourth position information of the first lane line and the second lane line in the second coordinate system corresponding to the inertial measurement sensor of the vehicle; and determine the positional relationship between the first lane line and the second lane line based on the third position information and the fourth position information.

[0179] In another possible implementation, the second determination unit 6032 is used to determine that the detection information is that the target lane includes the confluence if there is a first lane line and a second lane line whose positional relationship is such that they intersect; and to determine that the detection information is that the target lane does not include the confluence if there is no first lane line and a second lane line whose positional relationship is such that they intersect.

[0180] In another possible implementation, see Figure 7 , the device further comprises:

[0181] The second determination module 604 is used to determine the lane line type of multiple lane line segments included in at least one first lane line if the detection information indicates that the target lane includes a confluence; for each lane line segment, determine the intersection between the lane line segment and the second lane line to obtain at least one intersection point; and determine the starting point and ending point of the confluence based on the lane line type of the multiple lane line segments and the at least one intersection point.

[0182] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and at least one intersection is multiple; the second determination module 604 is used to determine, based on the lane line types of multiple lane line segments, the intersection between the solid lane line and the second lane line from at least one intersection as the starting point of the confluence, and determine the intersection between the dashed lane line and the second lane line as the end point of the confluence.

[0183] In another possible implementation, the lane line types include dashed lane lines and solid lane lines, and at least one intersection is one; the second determination module 604 is used to determine the starting point of the dashed lane line as the starting point of the confluence based on the lane line types of multiple lane line segments, and determine the intersection between the dashed lane line and the second lane line as the end point of the confluence.

[0184] In another possible implementation, the first determination module 603 is further configured to maintain the detection information that the target lane includes the confluence entrance if the next acquired second image does not include the starting point of the confluence entrance during the vehicle's driving on the target lane.

[0185] In another possible implementation, see Figure 7 , the device further comprises:

[0186] The adjustment module 605 is used to adjust the detection information to indicate that the target lane does not include the merging entrance if the second image acquired next time includes the end point of the merging entrance; or, the adjustment module is used to adjust the detection information to indicate that the target lane does not include the merging entrance if the target lane is determined to be the rightmost lane of the road; or, the adjustment module is used to adjust the detection information to indicate that the target lane does not include the merging entrance if the vehicle changes lanes from the target lane to another lane.

[0187] In another possible implementation, the adjustment module 605 is also used to adjust the detection information to indicate that the target lane does not include the confluence entrance if the second lane line is obscured and the end point of the confluence entrance cannot be determined and if the starting point of the confluence entrance is not included in the second image acquired next time.

[0188] In another possible implementation, the first image is taken by a camera device during the automatic driving process of the vehicle; see Figure 7 , the device further comprises:

[0189] A third determining module 606 is configured to determine driving behavior information of the vehicle based on the detection information;

[0190] The control module 607 is used to control the vehicle to continue automatic driving based on the driving behavior information.

[0191] An embodiment of the present application provides a lane confluence detection device, which determines detection information of the confluence of a target lane based on a first lane line and a second lane line in a first image, and the first image is captured based on a camera device installed on a vehicle. In this way, the first lane line and the second lane line in the first image are consistent with the lane lines in the actual road, thereby improving the accuracy of the detection information of the confluence of the target lane.

[0192] It should be noted that the lane confluence detection device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the detection of the lane confluence. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the server can be divided into different functional modules to complete all or part of the functions described above. In addition, the lane confluence detection device provided in the above embodiment and the lane confluence detection method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0193] Figure 8 The following is a block diagram illustrating the structure of a terminal 800 according to an exemplary embodiment of the present invention. Terminal 800 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0194] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0195] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0196] Memory 802 may include one or more computer-readable storage media, which may be non-transitory. Memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 802 is used to store at least one instruction, which is executed by processor 801 to implement the lane confluence detection method provided in the method embodiment of the present application.

[0197] In some embodiments, terminal 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 803 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 804, a display screen 805, a camera 806, an audio circuit 807, a positioning component 808, and a power supply 809.

[0198] The peripheral device interface 803 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0199] The RF circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 804 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 804 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0200] Display screen 805 is used to display a user interface (UI). This UI can include graphics, text, icons, videos, or any combination thereof. When display screen 805 is a touch screen display, it can also capture touch signals on or above the surface of display screen 805. These touch signals can be input as control signals to processor 801 for processing. Display screen 805 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be a single display screen 805, located on the front panel of terminal 800. In other embodiments, there can be at least two display screens 805, located on different surfaces of terminal 800 or in a foldable design. In still other embodiments, display screen 805 can be a flexible display, located on a curved or foldable surface of terminal 800. Display screen 805 can also be configured as a non-rectangular, irregular shape, i.e., a special-shaped screen. Display screen 805 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0201] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0202] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 801 for processing, or input into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0203] Positioning component 808 is used to locate the current geographic location of terminal 800 to implement navigation or LBS (Location Based Service). Positioning component 808 can be based on the US GPS (Global Positioning System), China's Beidou system, Russia's Greninja system, or the European Union's Galileo system.

[0204] Power supply 809 is used to power various components in terminal 800. Power supply 809 can be AC power, DC power, disposable batteries, or rechargeable batteries. When power supply 809 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0205] In some embodiments, the terminal 800 further includes one or more sensors 810 , including but not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , a fingerprint sensor 814 , an optical sensor 815 , and a proximity sensor 816 .

[0206] The accelerometer 811 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 800. For example, the accelerometer 811 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 811. The accelerometer 811 can also be used to collect game or user motion data.

[0207] The gyroscope sensor 812 can detect the orientation and rotation angle of the terminal 800. It can work in conjunction with the accelerometer 811 to collect the user's 3D movements of the terminal 800. Based on the data collected by the gyroscope sensor 812, the processor 801 can implement the following functions: motion sensing (for example, changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0208] The pressure sensor 813 can be set on the side frame of the terminal 800 and / or the lower layer of the display screen 805. When the pressure sensor 813 is set on the side frame of the terminal 800, it can detect the user's grip signal of the terminal 800, and the processor 801 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 813. When the pressure sensor 813 is set on the lower layer of the display screen 805, the processor 801 controls the operational controls on the UI interface based on the user's pressure operation on the display screen 805. The operational controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0209] The fingerprint sensor 814 is used to collect the user's fingerprint. The processor 801 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the user's identity based on the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 814 can be set on the front, back, or side of the terminal 800. When a physical button or manufacturer logo is provided on the terminal 800, the fingerprint sensor 814 can be integrated with the physical button or manufacturer logo.

[0210] The optical sensor 815 is used to detect ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 based on the ambient light intensity detected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is decreased. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 based on the ambient light intensity detected by the optical sensor 815.

[0211] Proximity sensor 816, also known as a distance sensor, is typically located on the front panel of terminal 800. Proximity sensor 816 is used to detect the distance between the user and the front of terminal 800. In one embodiment, when proximity sensor 816 detects that the distance between the user and the front of terminal 800 is gradually decreasing, processor 801 controls display screen 805 to switch from the screen-on state to the screen-off state. When proximity sensor 816 detects that the distance between the user and the front of terminal 800 is gradually increasing, processor 801 controls display screen 805 to switch from the screen-off state to the screen-on state.

[0212] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0213] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one program code, and the at least one program code is loaded and executed by a processor to implement a method for detecting a lane confluence entrance as in any of the above implementations.

[0214] An embodiment of the present application further provides a computer program product, which includes at least one program code, and the at least one program code is loaded and executed by a processor to implement a method for detecting a lane confluence entrance as in any of the above implementations.

[0215] In some embodiments, the computer program involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network may constitute a blockchain system.

[0216] The above description is merely an optional embodiment of the embodiments of the present application and is not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting a lane confluence, characterized in that: The method comprises: Acquire a first image, where the first image is captured based on a camera installed on a vehicle, and the first image is captured by the camera facing the traveling direction of the vehicle; Identifying at least one first lane line and at least one second lane line from the first image, where the first lane line is a lane line of a target lane where the vehicle is currently located, and the second lane line is a lane line adjacent to the first lane line; respectively determining first position information and second position information of the first lane line and the second lane line in a first coordinate system corresponding to the camera device; Performing coordinate conversion on the first position information and the second position information to obtain third position information and fourth position information of the first lane line and the second lane line in a second coordinate system corresponding to the inertial measurement sensor of the vehicle; determining, based on the third position information and the fourth position information, a lateral distance difference between the first lane line and the second lane line; if the lateral distance difference is not greater than a preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is intersecting; and if the lateral distance difference is greater than the preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is non-intersecting; If there are a first lane line and a second lane line that intersect in position, determining the detection information of the confluence entrance of the target lane as the target lane including the confluence entrance; if there are no first lane line and the second lane line that intersect in position, determining the detection information as the target lane not including the confluence entrance; If the detection information indicates that the target lane does not include a merging entrance, the vehicle's driving speed is adjusted to a first preset speed; if the detection information indicates that the target lane includes a merging entrance, the vehicle's driving speed is adjusted to a second preset speed.

2. The method according to claim 1, characterized in that The method further comprises: For each adjacent pair of a first lane line and a second lane line, determining a positional relationship between the first lane line and the second lane line; Based on the positional relationship between each pair of first lane lines and second lane lines, detection information of the confluence entrance of the target lane is determined.

3. The method according to claim 2, characterized in that The determining a positional relationship between the first lane line and the second lane line includes: determining a lateral distance difference between the first lane marking and the second lane marking; If the lateral distance difference is not greater than a preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is an intersection; If the lateral distance difference is greater than the preset lateral distance difference threshold, it is determined that the positional relationship between the first lane line and the second lane line is non-intersecting.

4. The method according to claim 1, wherein The method further comprises: If the detection information indicates that the target lane includes a confluence, determining lane line types of the multiple lane line segments included in the at least one first lane line; For each lane line segment, determining an intersection point between the lane line segment and the second lane line to obtain at least one intersection point; Based on the lane line types of the multiple lane line segments and the at least one intersection point, a starting point and an ending point of the confluence entrance are determined.

5. The method according to claim 4, characterized in that The lane line types include dashed lane lines and solid lane lines, and the at least one intersection point is multiple; The determining the starting point and the ending point of the confluence entrance based on the lane line types of the plurality of lane line segments and the at least one intersection point includes: Based on the lane line types of the multiple lane line segments, the intersection point between the solid lane line and the second lane line is determined from the at least one intersection point as the starting point of the confluence entrance, and the intersection point between the dashed lane line and the second lane line is determined as the ending point of the confluence entrance.

6. The method according to claim 4, characterized in that The lane line types include dashed lane lines and solid lane lines, and the at least one intersection point is one; The determining the starting point and the ending point of the confluence entrance based on the lane line types of the plurality of lane line segments and the at least one intersection point includes: Based on the lane line types of the multiple lane line segments, the starting point of the dotted lane line is determined as the starting point of the confluence entrance, and the intersection between the dotted lane line and the second lane line is determined as the end point of the confluence entrance.

7. The method according to any one of claims 4 to 6, characterized in that The method further comprises: When the vehicle is traveling on the target lane, if the second image acquired next time does not include the starting point of the confluence entrance, the detection information is maintained as indicating that the target lane includes the confluence entrance.

8. The method according to claim 7, characterized in that The method further comprises: If the second image acquired next time includes the end point of the confluence entrance, the detection information is adjusted to indicate that the target lane does not include the confluence entrance; or If it is determined that the target lane is the rightmost lane of the road, the detection information is adjusted so that the target lane does not include a merging entrance; or If the vehicle changes lanes from the target lane to another lane, the detection information is adjusted so that the target lane does not include a merging entrance.

9. The method according to claim 1, characterized in that The first image is captured by the camera device during the automatic driving of the vehicle; the method further includes: determining driving behavior information of the vehicle based on the detection information; Based on the driving behavior information, the vehicle is controlled to continue automatic driving.

10. A detection device for a lane confluence, characterized in that: The device comprises: an acquisition module, configured to acquire a first image, wherein the first image is acquired based on an imaging device installed on a vehicle, and the first image is acquired by the imaging device facing the traveling direction of the vehicle; a recognition module, configured to recognize at least one first lane line and at least one second lane line from the first image, wherein the first lane line is a lane line of a target lane currently located by the vehicle, and the second lane line is a lane line adjacent to the first lane line; a determination module for respectively determining first position information and second position information of the first lane line and the second lane line in a first coordinate system corresponding to the camera device; performing coordinate conversion on the first position information and the second position information to obtain third position information and fourth position information of the first lane line and the second lane line in a second coordinate system corresponding to the inertial measurement sensor of the vehicle; determining a lateral distance difference between the first lane line and the second lane line based on the third position information and the fourth position information; if the lateral distance difference is not greater than a preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is intersecting; if the lateral distance difference is greater than the preset lateral distance difference threshold, determining that the positional relationship between the first lane line and the second lane line is non-intersecting; if there are first lane lines and second lane lines with an intersecting positional relationship, determining that the detection information of the confluence entrance of the target lane is that the target lane includes the confluence entrance; if there are no first lane lines and second lane lines with an intersecting positional relationship, determining that the detection information is that the target lane does not include the confluence entrance; A module for performing the following steps: if the detection information indicates that the target lane does not include a merging entrance, adjusting the vehicle's driving speed to a first preset speed; if the detection information indicates that the target lane includes a merging entrance, adjusting the vehicle's driving speed to a second preset speed.

11. A terminal, characterized in that: The terminal includes: A processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method for detecting the confluence entrance of a lane as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method for detecting a lane confluence according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product includes at least one program code, and the at least one program code is loaded and executed by a processor to implement the method for detecting a lane confluence according to any one of claims 1 to 9.

Images