Method, device, electronic device and storage medium for determining observation lane line

Abstract

The disclosed embodiments disclose a method, device, electronic device and storage medium for determining an observed lane line, wherein the method comprises: obtaining sensed lane line information; determining a target observation mode based on the sensed lane line information; and determining the observed lane line based on the target observation mode. The disclosed embodiments can determine different observation modes as target observation modes based on different situations of sensed lane line information, and then determine the observed lane line based on the target observation mode, thereby achieving matching of the observed lane line determined by different observation modes with the map lane line in different scenarios, and effectively improving the accuracy of positioning.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and in particular to a method, device, electronic device and storage medium for determining an observed lane line. Background Art

[0002] In the positioning of mobile devices such as vehicles, it is usually necessary to process the collected original image information through perception algorithms to obtain segmentation information such as lane lines and curbs as observation information, and match it with map element information to obtain the precise position and posture of the vehicle in the high-precision map. However, in some special road conditions, the accuracy of vehicle positioning using the existing single observation mode is low. Summary of the invention

[0003] In order to solve the technical problems such as low positioning accuracy of the above-mentioned single observation mode, the present disclosure is proposed. The embodiments of the present disclosure provide a method, device, electronic device and storage medium for determining an observation lane line.

[0004] According to one aspect of an embodiment of the present disclosure, a method for determining an observed lane line is provided, comprising: acquiring perceived lane line information; determining a target observation mode based on the perceived lane line information; and determining an observed lane line based on the target observation mode.

[0005] According to another aspect of an embodiment of the present disclosure, a device for determining an observed lane line is provided, comprising: a first acquisition module for acquiring perceived lane line information; a first processing module for determining a target observation mode based on the perceived lane line information; and a second processing module for determining an observed lane line based on the target observation mode.

[0006] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method for determining the observation lane line described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for determining the observation lane line described in any of the above embodiments of the present disclosure.

[0008] Based on the method, device, electronic device and storage medium for determining the observation lane line provided by the above-mentioned embodiments of the present disclosure, different observation modes can be determined as target observation modes based on different situations of perceived lane line information, and then the observation lane line is determined based on the target observation mode, so that the observation lane lines determined by different observation modes in different scenarios are matched with the map lane lines, thereby effectively improving the accuracy of positioning.

[0009] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the method for determining the observed lane line provided by the present disclosure;

[0012] Figure 2 is a flowchart of a method for determining an observation lane line provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of step 202 provided by an exemplary embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of step 2024 provided by an exemplary embodiment of the present disclosure;

[0015] Figure 5 is a flow chart of a method for determining an observation lane line provided by another exemplary embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram of the structure of a device for determining an observation lane line provided by an exemplary embodiment of the present disclosure;

[0017] Figure 7 is a structural schematic diagram of a first processing module 502 provided by an exemplary embodiment of the present disclosure;

[0018] Figure 8 is a schematic structural diagram of a second determining unit 5024 provided by an exemplary embodiment of the present disclosure;

[0019] Fig. 9 is a structural schematic diagram of a device for determining an observation lane line provided by another exemplary embodiment of the present disclosure;

[0020] Fig.10 It is a structural schematic diagram of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0022] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0023] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0024] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0025] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0026] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0027] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0028] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0030] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0031] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0032] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0033] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0034] SUMMARY OF THE DISCLOSURE

[0035] In the process of realizing the present disclosure, the inventors found that in the positioning of mobile devices such as vehicles, it is usually necessary to process the collected original image information through a perception algorithm to obtain segmentation information such as lane lines and curbs as observation information, and match it with map element information to obtain the precise position and posture of the vehicle in the high-precision map. However, in some special road conditions, the accuracy of positioning the vehicle using the existing single observation mode is low.

[0036] Exemplary Overview

[0037] Figure 1 This is an exemplary application scenario of the method for determining the observed lane line provided in the present disclosure.

[0038] In the autonomous driving scenario, the vehicle needs to collect the surrounding environment image information in real time during driving, obtain the perceived lane line through a certain perception algorithm, and match the perceived lane line with the map lane line to determine the precise position of the vehicle in the high-precision map. In the figure, x0y represents the vehicle coordinate system, and the origin 0 is the center of the rear axle of the vehicle. Using the method for determining the observed lane line provided by the present disclosure, different observation modes can be adopted according to the different road scenes in which the vehicle is located, and then the observed lane line can be determined based on different observation modes. The precise position of the vehicle is determined based on the matching of the observed lane line with the map lane line, rather than using a single observation mode such as the perceived lane line to locate the vehicle. For example, three observation modes can be provided, namely, a first mode (also called a parsing mode), a second mode (also called a raw mode) and a third mode (also called a road model mode), wherein the first mode determines the observed lane line based on the semantic segmentation result of the lane line of the environment image, the second mode determines the observed lane line based on the curve equation of the fitted lane line, and the third mode determines the observed lane line based on the lane line equation after compensating the pitch angle. In this way, when the road scene where the vehicle is located cannot fit the lane line curve equation through the environment image, the first mode can be selected, and the semantic segmentation result can be directly used to determine the observed lane line. When the lane line curve equation can be fitted, the second mode or the third mode can be selected according to the actual situation. For example, the third mode can be selected when the map lane lines are parallel, and the second mode can be selected when the map lane lines are not parallel, or the second mode can be selected in combination with the results obtained by the perception algorithm to further determine the second mode or the third mode, thereby achieving the use of a relatively optimal observation mode in different road scenes, and effectively improving the accuracy of vehicle positioning.

[0039] Exemplary Methods

[0040] Figure 2 is a flow chart of a method for determining an observed lane line provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as a vehicle-mounted computing platform, such as Figure 2 As shown, the following steps are included:

[0041] Step 201, obtaining sensed lane line information.

[0042] The perceived lane line information is the lane line information obtained by perceiving and processing the collected environment image data through a certain perception algorithm. The perceived lane line information may include at least one of the following information: the lane line semantic segmentation result, the lane line curve equation obtained by lane line curve fitting, the parallel state of the lane line, etc., which can be set according to actual needs.

[0043] Step 202: Determine a target observation mode based on the sensed lane line information.

[0044] Among them, the target observation mode is an observation mode suitable for the current road scene determined based on the perceived lane line information. The observation mode can be set according to actual needs, and the observation mode can include at least two modes, such as the first mode, the second mode and the third mode, wherein the first mode determines the observed lane line based on the semantic segmentation result of the lane line of the environment image, the second mode determines the observed lane line based on the curve equation of the fitted lane line, and the third mode determines the observed lane line based on the lane line equation after compensating the pitch angle. When the road scene where the vehicle is located cannot fit the lane line curve equation through the environment image, the first mode can be selected, and the semantic segmentation result is directly used to determine the observed lane line. When the lane line curve equation can be fitted, the second mode or the third mode can be selected according to the actual situation, such as when the map lane lines are parallel, the third mode is selected, and when the map lane lines are not parallel, the second mode can be selected or the second mode or the third mode can be further determined in combination with the results obtained by the perception algorithm, so as to achieve the use of the relatively optimal observation mode in different road scenes, and effectively improve the accuracy of vehicle positioning.

[0045] Step 203: Determine the observed lane line based on the target observation mode.

[0046] After the target observation mode is determined, the observation lane line can be determined based on the target observation mode, and then the movable device can be positioned based on the observation lane line to obtain the precise position and posture of the movable device in the high-precision map.

[0047] Among them, the observed lane line obtained is the lane line information used to match with the map lane line elements to determine the precise position of the mobile device in the high-precision map, and the perceived lane line information is the lane line semantic segmentation result or further curve fitting result obtained in the perception stage based on the perception algorithm. Compared with the perceived lane line, the observed lane line refers to processing or not processing the perceived lane line in accordance with the scene in different road scenes. Among them, not processing, such as the road scene that needs to adopt the first mode, that is, directly using the lane line semantic segmentation result of the perceived lane line as the observed lane line, processing, such as for the road scene applicable to the third mode, pitch angle compensation, etc., to reduce the impact of changes in road slope, vehicle bumps, rapid acceleration and deceleration of the vehicle, etc. on the accuracy of the perception results, and determine more accurate lane line information for the corresponding scene, so that the observed lane line can be better matched with the map lane line, and the positioning accuracy is improved.

[0048] The method for determining the observation lane line provided in this embodiment can determine the target observation mode that works best for the current road scene based on the perceived lane line information of different road scenes, and then determine the observation lane line based on the target observation mode, thereby achieving matching of the observation lane lines determined by different observation modes in different scenes with the map lane lines, effectively improving the positioning accuracy, and solving the problem of low accuracy in vehicle positioning using a single observation mode in the prior art.

[0049] Figure 3 It is a flowchart of step 202 provided by an exemplary embodiment of the present disclosure.

[0050] In an optional example, the sensed lane line information includes the sensed lane line type; step 202 may specifically include the following steps:

[0051] Step 2021: Determine the number of first lane lines of the first type based on the sensed lane line information.

[0052] The lane line type may include at least two types, such as a curve equation type and a semantic segmentation result type, which may be set according to actual needs. The lane line type is a type preliminarily determined based on the perception algorithm in the perception stage. The first type may be a curve equation type, that is, the number of first lane lines that can fit the curve equation in the perception stage.

[0053] Step 2022a: In response to the number of the first lane lines meeting a preset number threshold, determining that the target observation mode is the first mode.

[0054] Among them, the first mode is to determine the observed lane line based on the lane line semantic segmentation result of the environment image. The preset number threshold can be set according to actual needs, for example, it is set to 0. When the first number of lane lines is 0, it means that the perception stage cannot fit the lane line curve equation based on the lane line semantic segmentation result. Therefore, the target observation mode can only be the first mode, and the observed lane line is determined based on the lane line semantic segmentation result. The lane line semantic segmentation result includes the regional coordinates of the lane line obtained by semantic segmentation or the coordinate point set belonging to the lane line. The specific representation method of the lane line semantic segmentation result can be set according to actual needs, and this disclosure does not limit it.

[0055] The present invention discloses that in a road scene where the lane curve equation cannot be fitted, the lane semantic segmentation result is used to determine the observed lane to position the movable device, thereby ensuring the positioning accuracy.

[0056] In an optional example, the sensed lane line information further includes lane line attribute information; after determining the number of first lane lines of the first type based on the sensed lane line information in step 2021, step 202 further includes:

[0057] Step 2022b, in response to the first lane line quantity not meeting a preset quantity threshold, based on the current position of the vehicle and the sensed lane line information, determine whether the main lane line information and / or other lane line information of the vehicle meets preset conditions.

[0058] Among them, the vehicle refers to any movable device, such as an autonomous driving vehicle, a robot, etc. Lane line attribute information may include parameters of the lane line curve equation, lane line position and other lane line related information, which can be set according to actual needs. The current position of the vehicle is the position component in the posture obtained by positioning the vehicle based on the target observation mode determined at the previous moment. The target observation mode determined at the current moment is to be used as the target observation mode of the positioning process at the next moment, and so on, so that each positioning process can use the optimal observation mode for positioning. The main lane line information of the vehicle refers to the lane line information of the lane where the vehicle is currently located, such as the left lane line information and the right lane line information of the lane where the vehicle is currently located. Other lane line information refers to the lane line information other than the main lane line on the road, such as the perception that the current road includes two other lane lines in addition to the left lane line and the right lane line of the lane where the vehicle is currently located. The preset conditions can be set according to actual needs. For example, if the left and right lane lines of the main lane line are incomplete, such as only the left lane line or only the right lane line, it is determined that the main lane line information does not meet the preset conditions. For example, if there are too many other lane lines within a certain distance, it is determined that the other lane line information does not meet the preset conditions, and so on.

[0059] Step 2023a, in response to the main lane line information and / or other lane line information not satisfying a preset condition, determining that the target observation mode is the second mode.

[0060] Among them, the second mode refers to the observation mode of determining the observed lane line based on the curve equation of the fitted lane line. Since the curve equation of the lane line can be fitted, the use of the lane line curve equation to determine the observed lane line can have a better positioning effect than the semantic segmentation result, so it can be determined that the second mode can be used. However, it is determined that the main lane line information and / or other lane line information do not meet the preset conditions, indicating that the parallel state cannot be determined, and thus the pitch angle compensation cannot be performed. Therefore, the lane line equation after the pitch angle compensation cannot be used to determine the third mode of the observed lane line. Therefore, the target observation mode is determined to be the second mode, ensuring that the best observation mode that can currently achieve the effect is used in different road scenarios. For example, if there are other lane lines in addition to the main lane line within one meter from the vehicle, the distance between the lane lines is too small, and there may be new lane lines and old lane lines at the same time. The new lane line refers to the newly set lane line markings on the road, and the old lane line refers to the original lane line markings corresponding to the new lane line markings. For example, the lane lines of a road are re-marked because the old lane lines are severely worn and cannot be recognized in some sections or because of re-planning, and the new lane lines do not cover the original lane lines, but are separated by a certain distance (for example, within one meter). This may cause some sections to perceive the new lane lines and the old lane lines at the same time. In this case, the target observation mode is directly determined to be the second mode.

[0061] In an optional example, after determining whether the main lane line information and / or other lane line information of the vehicle meets a preset condition based on the current position of the vehicle and the sensed lane line information in step 2022b, step 202 further includes:

[0062] Step 2023b, in response to the main lane line information and other lane line information satisfying preset conditions, obtaining a first map lane line corresponding to the main lane line of the vehicle based on the current position of the vehicle.

[0063] Among them, the first map lane line is the map lane line element corresponding to the main lane line where the vehicle is located in the map. The first map lane line can be a map lane line with a certain range determined in the map according to the current position of the vehicle as the first map lane line. The specific range can be set according to actual needs. For example, the map lane line with a certain length range along the direction of vehicle travel with the current position of the vehicle as the starting point is determined as the first map lane line. When it is determined that the main lane line information and other lane line information meet the preset conditions, it means that the parallelism between the lane lines can be determined, and then the corresponding first map lane line can be combined to determine which mode the target observation mode should adopt.

[0064] Step 2024, determining a first parallel state of the first map lane line based on a preset rule.

[0065] The preset rules may be set according to actual needs, as long as the parallel state of the lane line of the first map can be determined. For example, the first parallel state may be determined based on the difference between the starting point distance and the ending point distance of the left and right lane lines in the lane line of the first map.

[0066] Step 2025a, in response to the first parallel state being non-parallel and the parallel state of the main lane line included in the main lane line information being parallel, determining the second parallel state of the main lane line based on the lane line attribute information in the main lane line information.

[0067] Among them, the lane line attribute information includes the parameters of the lane line curve equation, the lane line position and other lane line related information. The parallel state of the main lane line included in the main lane line information is the parallel state of the lane line output in the perception stage. For example, the perception stage outputs it according to the target observation mode determined at the previous moment. For example, the target observation mode determined at the previous moment is the third mode, that is, the current position of the vehicle is located based on the observed lane line determined by the third mode, and the third mode is considered to be parallel. Therefore, the parallel state of the lane line is output as parallel in the perception stage at the current moment. In fact, as the vehicle moves, the parallel state of the lane line may change from parallel to non-parallel, resulting in the parallel state output in the perception stage being inaccurate. The perception stage can also determine the parallel state of the lane line in other ways, such as determining the parallel state of the lane line based on the lane line information of the perception result. The specific setting can be based on actual needs and is not limited in this disclosure. When the first parallel state of the lane line in the first map is determined to be non-parallel, and the parallel state of the main lane line determined in the perception stage is parallel, it means that the two results are inconsistent, and it is necessary to further determine its parallelism based on the lane line attribute information of the main lane line, that is, determine the second parallel state of the main lane line. The specific determination method can be determined according to the actual situation of the lane line attribute information. For example, the second parallel state can be determined based on the difference between the starting point distance and the ending point distance of the left and right lane lines of the main lane line. When the difference between the starting point distance and the ending point distance is greater than the distance threshold, it is determined that the left and right lane lines are not parallel, that is, the second parallel state of the main lane line is parallel, otherwise it is determined that the left and right lane lines are parallel.

[0068] Step 2026a: In response to the second parallel state being non-parallel, determining that the target observation mode is the second mode or the third mode.

[0069] Among them, the third mode is an observation mode for determining the observed lane line based on the lane line equation after compensating the pitch angle, wherein the compensation for the pitch angle can be achieved based on the condition of the lane line being parallel, or it can be compensated based on the historical pitch angle, which can be set specifically according to actual needs. Compensation based on the parallelism of the lane lines means that when compensating the pitch angle, the lane lines are considered to be parallel, and compensation is performed with the parallelism of the lane lines as a constraint. In this step, since the first parallel state of the lane line in the first map and the second parallel state of the main lane line are not parallel, in this case, the compensation for the pitch angle when the third mode is used can be based on the historical pitch angle. When the lane line in the first map and the perceived main lane line are not parallel, compensating for the pitch angle based on the historical pitch angle can improve the observation effect to a certain extent. Of course, the target observation mode can also be directly determined to be the second mode, and whether the second mode or the third mode is used can be set according to actual needs.

[0070] Step 2026b: In response to the second parallel state being parallel, determining the target observation mode to be the second mode.

[0071] When the first parallel state of the lane line of the first map is not parallel, and the second parallel state of the perceived main lane line is parallel, it means that the parallel states of the two are inconsistent.

[0072] In an optional example, after determining the first parallel state of the first map lane line based on a preset rule in step 2024, the method further includes:

[0073] Step 2025b: In response to the first parallel state being parallel, determining the target observation mode to be the third mode.

[0074] When the first parallel state of the lane line in the first map is parallel, it means that the main lane line of the lane where the vehicle is currently located is actually parallel. Therefore, the pitch angle compensation can be performed on the perceived lane line based on the lane line parallel constraint, and the observation lane line is determined based on the lane line equation after the pitch angle compensation, so as to reduce the influence of the road slope change, the vehicle bump, the rapid acceleration and deceleration of the vehicle, etc. on the determined observation lane line, that is, the target observation mode is determined to be the third mode, and the observation lane line is determined based on the third mode, and then the vehicle is positioned to improve the precision and accuracy of the positioning result. The specific pitch angle compensation method can be set according to actual needs, and the present disclosure does not limit it. For example, when the pitch angle (pitch) is too large, the left and right lane lines of the main lane line form an "inner eight" state, and when the pitch angle is too small, the left and right lane lines of the main lane line form an "outer eight" state. The principle of pitch angle compensation is that the left and right lane lines of the main lane line after compensation are parallel.

[0075] The present invention reduces the influence of road slope change, vehicle bump, sudden acceleration and deceleration of the vehicle, etc. on the observed lane line by determining the observed lane line based on the lane line equation after compensating for the pitch angle when the lane lines on the map are parallel, so that the observed lane line can be better matched with the map lane line, and the precise positioning of the vehicle is realized, thereby improving the positioning accuracy.

[0076] In an alternative example, Figure 4 2 is a flow chart of step 2024 provided by an exemplary embodiment of the present disclosure. In this example, step 2024 determines the first parallel state of the first map lane line based on a preset rule, including:

[0077] Step 20241, determining a first distance based on a first starting point of a left lane line and a second starting point of a right lane line of a first map lane line.

[0078] Among them, the first distance is the distance between the first starting point of the left lane line and the second starting point of the right lane line. The first distance can be calculated by the coordinates of the first starting point and the coordinates of the second starting point, and the specific calculation method is not repeated. For example, the first starting point and the second starting point can be points with the same ordinate in the vehicle coordinate system (a coordinate system with the center of the rear axle of the vehicle as the origin, the length direction of the vehicle as the ordinate axis (x-axis), and the width direction of the vehicle as the abscissa axis (y-axis)), then the first distance can be the absolute value of the difference between the abscissas. The specific starting point, end point and distance determination method can be set according to actual needs, and this disclosure does not limit it.

[0079] Step 20242, determining the second distance based on the first end point of the left lane line and the second end point of the right lane line.

[0080] The principle for determining the second distance is similar to that of the first distance, and will not be repeated here.

[0081] Step 20243, determine the distance difference between the first distance and the second distance.

[0082] The distance difference may be an absolute value, for example, if the first distance is 4 meters and the second distance is 4.3 meters, the determined distance difference is 0.3 meters.

[0083] Step 20244, in response to the distance difference being less than a preset threshold, determining that the first parallel state of the first map lane line is parallel.

[0084] Among them, the preset threshold can be set according to actual needs, for example, it can be set to 1 meter, 0.5 meter, etc. When the distance difference is less than the preset threshold, it means that the distance between the starting point of the left and right lane lines and the distance between the ending point of the left and right lane lines are small, and the two can be considered to be parallel.

[0085] Figure 5 It is a flowchart of a method for determining an observation lane line provided by another exemplary embodiment of the present disclosure.

[0086] In an optional example, determining the observed lane line based on the target observation mode in step 203 includes:

[0087] Step 2031a, in response to the target observation mode being the first mode, determining the observed lane line based on the lane line semantic segmentation result of the environment image.

[0088] Among them, the lane line semantic segmentation result includes the segmented area or coordinate point set belonging to the lane line. The observed lane line is determined based on the lane line semantic segmentation result. These areas or coordinate point sets belonging to the lane line are used as the observed lane lines and matched with the map lane lines to determine the precise position and posture of the vehicle.

[0089] Step 2031b, in response to the target observation mode being the second mode, determining the observation lane line based on the curve equation of the fitted lane line.

[0090] The basic premise for determining that the target observation mode is the second mode is that the curve equation of the lane line can be fitted, so the curve equation of the fitted lane line can be used as the observed lane line. The curve equation of the fitted lane line can be a parameterized cubic equation, and the specific representation method will not be repeated.

[0091] Step 2031c, in response to the target observation mode being the third mode, determining the observed lane line based on the lane line equation after compensating for the pitch angle, wherein the pitch angle is compensated according to the lane line parallel constraint.

[0092] Under the condition that the lane lines are parallel, compensating the pitch angle based on the lane line parallel constraint can reduce the impact of factors such as road slope changes, vehicle bumps, and rapid acceleration and deceleration of the vehicle on the accuracy of observed lane lines, improve the accuracy of observation results, and thus improve the precision and accuracy of positioning based on the observation results.

[0093] In an optional example, the step 201 of acquiring the sensed lane line information includes:

[0094] Step 2011, obtaining environmental image data.

[0095] The environmental image data may be acquired through an image sensor installed on the vehicle, such as a camera, and the specific environmental image data acquisition method may be set according to actual needs.

[0096] Step 2012, semantic segmentation is performed on the environment image data to obtain lane line semantic segmentation results.

[0097] Among them, semantic segmentation can be implemented using any implementable semantic segmentation model, such as a semantic segmentation model based on deeplabv3+ and its related series, a semantic segmentation model based on FCN (Fully Convolutional Networks) and its related series, a semantic segmentation model based on UNet and its related series, and so on. The specific setting can be based on actual needs and is not limited in this disclosure.

[0098] Step 2013: Perform curve fitting on the lane line semantic segmentation result to obtain the curve equation corresponding to each lane line.

[0099] The curve fitting may be implemented by any feasible fitting method, which is not limited in the present disclosure. For example, a parameterized cubic equation may be used for fitting.

[0100] Step 2014: determining a third parallel state between adjacent lane lines based on the curve equations corresponding to the lane lines.

[0101] After determining the curve equations corresponding to each lane line, the third parallel state between adjacent lane lines can be determined based on the parameters of the curve equation and the characteristics of the curve equation. The specific determination method can be any feasible method, which is not limited in this disclosure.

[0102] Step 2015: Determine the perceived lane line information based on at least one of the semantic segmentation result, the curve equations corresponding to each lane line, and the third parallel state between adjacent lane lines.

[0103] The specific perceived lane line information can be set according to actual needs to include at least one of the semantic segmentation results, the curve equations corresponding to each lane line, the third parallel state between adjacent lane lines and more other related information, which can be set according to actual needs.

[0104] Any of the methods for determining an observed lane line provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any of the methods for determining an observed lane line provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any of the methods for determining an observed lane line mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0105] Exemplary Devices

[0106] Figure 6 is a schematic diagram of the structure of a device for determining an observation lane line provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiment of the present disclosure, such as Figure 6The device shown includes: a first acquisition module 501 , a first processing module 502 , and a second processing module 503 .

[0107] The first acquisition module 501 is used to acquire the perceived lane line information; the first processing module 502 is used to determine the target observation mode based on the perceived lane line information acquired by the first acquisition module 501; the second processing module 503 is used to determine the observed lane line based on the target observation mode determined by the first processing module 502.

[0108] Figure 7 is a schematic diagram of the structure of a first processing module 502 provided by an exemplary embodiment of the present disclosure.

[0109] In an optional example, the sensed lane line information includes the sensed lane line type; the first processing module 502 includes: a first determination unit 5021 and a first processing unit 5022a.

[0110] The first determination unit 5021 is used to determine the number of first lane lines of the first type based on the perceived lane line information obtained by the first acquisition module 501; the first processing unit 5022a is used to determine that the target observation mode is the first mode in response to the number of the first lane lines determined by the first determination unit 5021 meeting a preset number threshold.

[0111] In an optional example, the sensed lane line information also includes lane line attribute information; the first processing module 502 also includes: a second processing unit 5022b and a third processing unit 5023a.

[0112] The second processing unit 5022b is used to determine whether the main lane line information and / or other lane line information of the vehicle meets the preset conditions based on the current position of the vehicle and the perceived lane line information in response to the first lane line number not meeting the preset number threshold; the third processing unit 5023a is used to determine that the target observation mode is the second mode in response to the second processing unit 5022b determining that the main lane line information and / or the other lane line information do not meet the preset conditions.

[0113] In an optional example, the first processing module 502 further includes: a fourth processing unit 5023b, a second determining unit 5024, a fifth processing unit 5025a, a sixth processing unit 5026a and a seventh processing unit 5026b.

[0114] The fourth processing unit 5023b is used to obtain the first map lane line corresponding to the main lane line of the vehicle based on the current position of the vehicle in response to the main lane line information and the other lane line information satisfying the preset conditions; the second determination unit 5024 is used to determine the first parallel state of the first map lane line based on a preset rule; the fifth processing unit 5025a is used to determine the second parallel state of the main lane line based on the lane line attribute information in the main lane line information in response to the second determination unit 5024 determining that the first parallel state is non-parallel and the parallel state of the main lane line included in the main lane line information is parallel; the sixth processing unit 5026a is used to determine that the target observation mode is the second mode or the third mode in response to the fifth processing unit 5025a determining that the second parallel state is non-parallel; the seventh processing unit 5026b is used to determine that the target observation mode is the second mode in response to the fifth processing unit 5025a determining that the second parallel state is parallel.

[0115] In an optional example, the first processing module 502 further includes: an eighth processing unit 5025b, configured to determine that the target observation mode is the third mode in response to the first parallel state being parallel.

[0116] In an alternative example, Figure 8 2 is a schematic diagram of the structure of a second determining unit 5024 provided by an exemplary embodiment of the present disclosure. In this example, the second determining unit 5024 includes: a first determining subunit 50241, a second determining subunit 50242, a third determining subunit 50243 and a first processing subunit 50244.

[0117] The first determination subunit 50241 is used to determine the first distance based on the first starting point of the left lane line and the second starting point of the right lane line of the first map lane line; the second determination subunit 50242 is used to determine the second distance based on the first ending point of the left lane line and the second ending point of the right lane line; the third determination subunit 50243 is used to determine the distance difference between the first distance obtained by the first determination subunit 50241 and the second distance obtained by the second determination subunit 50242; the first processing subunit 50244 is used to determine that the first parallel state of the first map lane line is parallel in response to the distance difference obtained by the third determination subunit 50243 being less than a preset threshold.

[0118] Fig. 9 It is a structural schematic diagram of a device for determining an observation lane line provided by another exemplary embodiment of the present disclosure.

[0119] In an optional example, the second processing module 503 includes: a first response unit 5031 , a second response unit 5032 , and a third response unit 5033 .

[0120] The first response unit 5031 is used to determine the observed lane line based on the lane line semantic segmentation result of the environment image in response to the target observation mode being the first mode; the second response unit 5032 is used to determine the observed lane line based on the curve equation of the fitted lane line in response to the target observation mode being the second mode; the third response unit 5033 is used to determine the observed lane line based on the lane line equation after compensating for the pitch angle in response to the target observation mode being the third mode, wherein the pitch angle is compensated according to the lane line parallel constraint.

[0121] In an optional example, the first acquisition module 501 includes: an acquisition unit 5011 , a segmentation unit 5012 , a fitting unit 5013 , a third determination unit 5014 and a fourth determination unit 5015 .

[0122] An acquisition unit 5011 is used to acquire environmental image data; a segmentation unit 5012 is used to perform semantic segmentation on the environmental image data acquired by the acquisition unit 5011 to obtain a lane line semantic segmentation result; a fitting unit 5013 is used to perform curve fitting on the lane line semantic segmentation result obtained by the segmentation unit 5012 to obtain a curve equation corresponding to each lane line; a third determination unit 5014 is used to determine a third parallel state between adjacent lane lines based on the curve equations corresponding to each lane line obtained by the fitting unit 5013; a fourth determination unit 5015 is used to determine the perceived lane line information based on at least one of the semantic segmentation result, the curve equations corresponding to each lane line, and the third parallel state between adjacent lane lines.

[0123] Exemplary Electronic Devices

[0124] The embodiment of the present disclosure also provides an electronic device, including: a memory for storing a computer program;

[0125] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method for determining the observation lane line described in any of the above embodiments of the present disclosure is implemented.

[0126] Fig.10 FIG. 1 is a schematic diagram of a structure of an application embodiment of the electronic device disclosed in the present invention. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0127] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0128] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0129] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0130] For example, the input device 13 may be the microphone or microphone array mentioned above, which is used to capture the input signal of the sound source.

[0131] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0132] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0133] Of course, to simplify, Fig.10 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0134] Exemplary computer program products and computer-readable storage media

[0135] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0136] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0137] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0138] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0139] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0140] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0141] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0142] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0143] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0144] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0145] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for determining an observation lane line, comprising: Obtaining the perceived lane line information; The sensed lane line information includes at least one of a lane line semantic segmentation result, a lane line curve equation, and a lane line parallel state information; Determining a target observation mode based on the sensed lane line information; The target observation mode is one of the first mode, the second mode and the third mode; the first mode is a mode for determining the observed lane line based on the lane line semantic segmentation result; The second mode is a mode for determining the observed lane line based on the lane line curve equation; The third mode is a mode for determining the observed lane line based on the lane line equation after compensating the pitch angle; Based on the target observation mode, determining an observation lane line; Wherein, the acquiring of the sensed lane line information includes: Acquire environmental image data; Based on the environmental image data, the perceived lane line information is determined.

2. The method according to claim 1, wherein: The sensed lane line information includes a lane line type sensed; and determining a target observation mode based on the sensed lane line information includes: Based on the sensed lane line information, determining the number of first lane lines of a first type; In response to the number of the first lane lines meeting a preset number threshold, the target observation mode is determined to be the first mode.

3. The method according to claim 2, wherein: The sensed lane line information further includes lane line attribute information; after determining the number of first lane lines of the first type based on the sensed lane line information, the method further includes: In response to the first lane line quantity not meeting a preset quantity threshold, determining whether the main lane line information and / or other lane line information of the vehicle meets a preset condition based on the current position of the vehicle and the sensed lane line information; In response to the main lane line information and / or the other lane line information not satisfying the preset condition, determining that the target observation mode is the second mode.

4. The method according to claim 3, wherein: After determining whether the main lane line information and / or other lane line information of the vehicle meets a preset condition based on the current position of the vehicle and the sensed lane line information, the method further includes: In response to the main lane line information and the other lane line information satisfying the preset condition, acquiring a first map lane line corresponding to the main lane line of the vehicle based on the current position of the vehicle; Determine a first parallel state of the lane line of the first map based on a preset rule; In response to the first parallel state being non-parallel and the parallel state of the main lane line included in the main lane line information being parallel, determining a second parallel state of the main lane line based on lane line attribute information in the main lane line information; In response to the second parallel state being non-parallel, determining the target observation mode to be the second mode or the third mode; In response to the second parallel state being parallel, determining the target observation mode to be the second mode.

5. The method according to claim 4, wherein: After determining the first parallel state of the first map lane line based on the preset rule, the method further includes: In response to the first parallel state being parallel, the target observation mode is determined to be the third mode.

6. The method according to claim 4, wherein: The determining the first parallel state of the lane line of the first map based on a preset rule includes: Determine a first distance based on a first starting point of a left lane line and a second starting point of a right lane line of the first map lane line; Determining a second distance based on the first end point of the left lane line and the second end point of the right lane line; determining a distance difference between the first distance and the second distance; In response to the distance difference being less than a preset threshold, determining that the first parallel state of the first map lane line is parallel.

7. The method according to claim 1, wherein: The step of determining an observation lane line based on the target observation mode includes: In response to the target observation mode being the first mode, determining the observed lane line based on a lane line semantic segmentation result of the environment image; In response to the target observation mode being the second mode, determining the observation lane line based on a curve equation of the fitted lane line; In response to the target observation mode being the third mode, the observed lane line is determined based on a lane line equation after compensating for a pitch angle, wherein the pitch angle is compensated according to a lane line parallel constraint.

8. The method according to any one of claims 1 to 7, wherein: The acquiring of the sensed lane line information includes: Acquire environmental image data; Performing semantic segmentation on the environment image data to obtain a lane line semantic segmentation result; Performing curve fitting on the lane line semantic segmentation result to obtain curve equations corresponding to each lane line; Determining a third parallel state between adjacent lane lines based on the curve equations corresponding to the lane lines respectively; The perceived lane line information is determined based on at least one of the semantic segmentation result, the curve equations corresponding to each lane line, and the third parallel state between adjacent lane lines.

9. A device for determining an observation lane line, comprising: A first acquisition module is used to acquire the sensed lane line information; The sensed lane line information includes at least one of a lane line semantic segmentation result, a lane line curve equation, and a lane line parallel state information; A first processing module, configured to determine a target observation mode based on the sensed lane line information; The target observation mode is one of the first mode, the second mode and the third mode; the first mode is a mode for determining the observed lane line based on the lane line semantic segmentation result; The second mode is a mode for determining the observed lane line based on the lane line curve equation; The third mode is a mode for determining the observed lane line based on the lane line equation after compensating the pitch angle; A second processing module, used for determining an observation lane line based on the target observation mode; Wherein, the first acquisition module is specifically used for: Acquire environmental image data; Based on the environmental image data, the perceived lane line information is determined.

10. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method for determining an observation lane line according to any one of claims 1 to 8.

11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method for determining the observation lane line described in any one of claims 1-8 above.

Images