Vehicle positioning method and device, electronic device, and storage medium

Abstract

The application discloses a vehicle positioning method and device, an electronic device and a storage medium. The method comprises the following steps: acquiring image information collected at a road end; inputting the image information collected at the road end into a lane line detection model trained in advance to obtain a target lane line; inputting the image information collected at the road end into a vehicle detection model trained in advance to obtain a target vehicle, wherein the target vehicle comprises a target vehicle positioning result; and performing constraint on the positioning result of the target vehicle according to the target lane line to obtain a final positioning result of the target vehicle. The positioning accuracy of the target vehicle can be improved by the application.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a vehicle positioning method and device, electronic device, and storage medium. Background Technology

[0002] As intelligent transportation roadside monitoring systems become more mature, the requirements for the positioning accuracy of vehicles, the main traffic participants on the road, are becoming increasingly higher. However, most of the vehicle detection algorithms currently in use rely on 2D detection.

[0003] The commonly used vehicle detection bounding box is usually located by using the center point of the bottom edge as the vehicle's image location, and then converting it to latitude and longitude information. However, this often results in a large error, which affects the vehicle localization result. Summary of the Invention

[0004] This application provides a vehicle positioning method and apparatus, electronic device, and storage medium to improve positioning accuracy and reduce vehicle positioning deviation.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a vehicle positioning method, wherein the method includes:

[0007] Acquire image information collected at the roadside;

[0008] The image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane line;

[0009] The image information collected at the roadside is input into a pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle localization result;

[0010] The positioning result of the target vehicle is constrained based on the target lane line to obtain the final positioning result of the target vehicle.

[0011] In some embodiments, constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle includes:

[0012] Based on the 2D detection box in the target vehicle positioning result, determine the coordinates of the bottom center point of the 2D detection box, and use the coordinates of the bottom center point as the image pixel coordinate positioning result of the target vehicle.

[0013] When the target vehicle is in the first driving state, the image pixel coordinate positioning result of the target vehicle is adjusted according to the pixel coordinate position of the target lane line.

[0014] In some embodiments, constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle further includes:

[0015] Based on the target lane line, determine the pixel coordinate position of the target lane line and calculate the slope parameter of the target lane line according to different types of lane lines;

[0016] The driving state of the target vehicle is determined based on the heading angle of the target vehicle and the slope parameter of the target lane line, wherein the heading angle of the target vehicle is calculated by connecting the vehicle positioning points of the target vehicle positioning results in adjacent image frames.

[0017] In some embodiments, constraining the positioning result of the target vehicle based on the target lane line includes:

[0018] Using the center position of the lane line as the prior position, and based on the pixel coordinate position of the target lane line as the standard, the image pixel coordinate positioning result of the target vehicle is adjusted to the pixel coordinate position of the target lane line within the 2D detection frame of the target vehicle.

[0019] In some embodiments, the first state of the target vehicle's driving state includes: the target vehicle exhibiting a tilt angle exceeding a preset threshold in a curve.

[0020] In one embodiment of this application, the image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane line, including: performing image segmentation on the lane line detection results to obtain each lane line instance as the pixel segmentation result of the lane line; fitting the segmentation results of multiple pixel unit widths of each lane line to obtain a line with a one pixel unit width of the lane line as the target lane line.

[0021] In some embodiments, the pre-trained vehicle detection model includes:

[0022] Based on the object detection network, the original road image data and corresponding labels are used as training data to train a 2D vehicle detection model.

[0023] In some embodiments, the pre-trained lane detection model includes:

[0024] Machine learning is used to train the system using multiple sets of data. Each set of data includes: an image of vertical lane lines and lane line type labels for solid lines, dashed lines, double solid lines, and double dashed lines in the image.

[0025] In some embodiments, acquiring the image information collected at the roadside includes:

[0026] Acquire image information from the roadside perspective, wherein the image information includes at least one of the following: image data of vehicles traveling on the road, and image data of lane lines on the road.

[0027] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0028] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0029] The at least one technical solution adopted in this application can achieve the following beneficial effects: a target lane line is detected using a lane line detection model, and a target vehicle is detected using a vehicle detection model. The positioning result of the target vehicle can be further constrained based on the target lane line to obtain the final positioning result of the target vehicle. After constraint, the positioning accuracy of the target vehicle can be improved. For example, a vehicle that might originally be located in a green belt can be positioned within the lane through constraint, or a vehicle that was originally located in an adjacent lane can be positioned within the actual lane, etc. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0031] Figure 1 This is a flowchart illustrating the vehicle positioning method in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the vehicle positioning device in the embodiments of this application;

[0033] Figure 3 This is a schematic diagram illustrating the effect of the vehicle positioning method in the embodiments of this application;

[0034] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] During their research, the inventors discovered that, typically, during a vehicle's turning process, the vehicle's 2D detection bounding box remains parallel to the top and bottom edges of the image, and its left and right edges are parallel to the left and right edges of the image. This results in a significant portion of the 2D detection bounding box being background, and the vehicle's side not being parallel to the bounding box, creating an oblique intersection. In this situation, the center point of the bottom edge of the detection bounding box will deviate considerably from the actual vehicle position.

[0037] To address the aforementioned shortcomings, the vehicle positioning method in this application embodiment, based on the constraint method of lane lines for 2D vehicle positioning, firstly detects the lane lines in the image by training a lane line monitoring model; then, it uses the lane lines as constraints on vehicle positioning points to reduce vehicle positioning deviation.

[0038] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0039] This application provides a vehicle positioning method, such as... Figure 1 The diagram shows a vehicle positioning method flowchart in an embodiment of this application. The method includes at least the following steps S110 to S140:

[0040] Step S110: Obtain image information collected at the roadside.

[0041] Image information is obtained from the perspective of the roadside. This image information is used as the training set during the training phase and as the image data to be detected during the detection phase.

[0042] The term "roadside" here typically refers to roadside units deployed at intersections or road sections. These units use cameras to capture image information from the roadside perspective. Intersections include right-turn intersections, left-turn intersections, etc. At right-turn intersections, the turning radius is smaller, resulting in a greater vehicle tilt. Conversely, at left-turn intersections, the turning radius is larger, resulting in a smaller vehicle tilt.

[0043] Image information includes images containing lane lines and images containing vehicles, typically divided into two different sets of images. It's important to note that, based on prior knowledge, vehicles usually travel within lane lines (lane center lines), so prior knowledge can be used to constrain the detected vehicle localization results.

[0044] At the same time, it's important to consider that roadside image information typically has a fixed background. For example, roadside green belts or flower beds. If image information from the same roadside source is used, the background information remains largely constant.

[0045] Furthermore, the significant tilt angle that vehicles exhibit when turning or making a U-turn should be considered. Using the same image information collected from the same roadside location will not yield accurate positioning results. For example, the original 2D vehicle detection bounding box might place the vehicle within a flower bed, resulting in positioning errors.

[0046] Step S120: Input the image information collected at the road end into the pre-trained lane line detection model to detect the target lane line.

[0047] The image information collected at the roadside is input into a pre-trained lane detection model, which can detect multiple lane lines and the types of these lane lines.

[0048] For example, lane line data at roadside angles can be collected during training: marking solid lines, dashed lines, double solid lines (including oncoming lanes during turns), double dashed lines, and other vertical lane lines, and processing the marked .json files as label data. Then, using the label data and the original training data as training samples, the built neural network model is trained and saved as a lane line detection model.

[0049] Preferably, the lane line detection model adds recognition results for different lane line types, which can identify lane lines in the image information collected at the road end and the type of lane line described therein.

[0050] Step S130: Input the image information collected at the roadside into the pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle positioning result.

[0051] The image information collected at the roadside is also input into the vehicle detection model to detect the location of the target vehicle. Of course, it can also include the type of vehicle, such as a truck, a car, or a bicycle.

[0052] For example, vehicle data is collected from roadside angles, vehicles are labeled, and the labeled .json files are processed to correspond to the original image labels. A YOLOv7 object detection network is built, and the original data and corresponding labels are used as training data to train a 2D vehicle detection model and output the vehicle detection model.

[0053] It is understood that other target detection networks can also be used. This application does not impose specific limitations on the embodiments, and those skilled in the art can make different choices according to the actual use scenario.

[0054] The image information input to the vehicle detection model and the lane line detection model can be the same frame, which can improve the accuracy of image recognition.

[0055] Step S140: Constrain the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle.

[0056] The positioning result of the target vehicle is constrained based on the target lane lines in the detection results of the two detection models. This can be understood as including, but not limited to, adjusting the vehicle's relative position coordinates laterally or longitudinally. After constraint and adjustment, the final positioning result of the target vehicle can be obtained.

[0057] Furthermore, whether or not constraints are needed also depends on whether the vehicle is currently turning left or right. Therefore, the vehicle's state needs to be determined by the vehicle's heading angle (YAW), and then constraints can be applied based on the conditions.

[0058] In one embodiment of this application, constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle includes: determining the coordinates of the bottom center point of the 2D detection frame based on the 2D detection frame of the target vehicle, and using the coordinates of the bottom center point as the image pixel coordinate positioning result of the target vehicle; determining the pixel coordinate position of the target lane line based on the target lane line and calculating the slope parameter of the target lane line according to different types of lane lines; determining the driving state of the target vehicle based on the heading angle of the target vehicle and the slope parameter of the target lane line, wherein the heading angle of the target vehicle is calculated by connecting the vehicle positioning points in adjacent image frames; and adjusting the image pixel coordinate positioning result of the target vehicle based on the pixel coordinate position of the target lane line when the driving state of the target vehicle is in a first state.

[0059] In practice, when identifying target lane lines and target vehicles, the coordinates of the center point of the bottom edge of the 2D detection frame of the target vehicle are determined, and these coordinates are used as the image pixel coordinates of the target vehicle. This process yields the image pixel coordinates of the target vehicle, as the center point of the bottom edge is typically used as the vehicle's location point.

[0060] Then, based on the target lane line, the pixel coordinate position of the target lane line is determined, and the slope parameter of the target lane line is calculated according to different types of lane lines. That is, lane lines are detected using lane lines, the pixel coordinate position of the lane lines is output, and the corresponding slope is calculated according to different lane lines (solid lines, dashed lines, double solid lines, double dashed lines, etc., vertical lane lines).

[0061] Finally, based on the heading angle of the target vehicle and the slope parameters of the target lane line, the driving status of the target vehicle is determined. This typically refers to whether the target vehicle is traveling straight or turning left or right.

[0062] When determining whether a vehicle is turning, the vehicle's heading angle is calculated by connecting the vehicle's positioning lines in previous and subsequent image frames. Then, the vehicle's heading angle is used to determine whether it is in a turning state by checking if it is parallel to the slope of the lane line.

[0063] If the target vehicle is in the first driving state, the image pixel coordinate positioning result of the target vehicle is adjusted according to the pixel coordinate position of the target lane line.

[0064] The first state refers to the right turn. In the embodiments of this application, there may also be a second state and a third state to represent states such as left turn and straight-ahead movement. The first state is only used to illustrate the vehicle's driving state in the embodiments of this application and is not intended to limit the specific state.

[0065] In one embodiment of this application, the image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane line, including: performing image segmentation on the lane line detection results to obtain each lane line instance as the pixel segmentation result of the lane line; fitting the segmentation results of multiple pixel unit widths of each lane line to obtain a line with a one pixel unit width of the lane line as the target lane line.

[0066] In specific implementation, the lane line detection model in this application embodiment can detect different types of lane lines. At the same time, it is also necessary to use a detection network with image segmentation capabilities to detect all the required lane line pixels in the image through the image segmentation network, which is used as the output of the lane line detection model, i.e., each lane line instance.

[0067] For example, the lane line detection results from left to right include: solid line 1, solid line 2, dashed line 1, dashed line 2, double solid line, double dashed line, etc. After image segmentation, the pixels of each lane line are separated. This allows them to be distinguished from other non-lane line pixels, and also creates distinct instance elements between different lane lines.

[0068] In the embodiments of this application, only the center line of the lane line (the center line is the lane line) is needed. That is, the lane line that originally had a width of n pixels is fitted into a lane line with a width of 1 pixel. Only then can a constraint line be obtained more accurately and intersected with the 2D vehicle detection box.

[0069] In one embodiment of this application, constraining the positioning result of the target vehicle based on the target lane line includes: taking the vehicle driving at the center position of the lane line as a priori, and using the pixel coordinate position of the target lane line as a standard, adjusting the image pixel coordinate positioning result of the target vehicle to the pixel coordinate position of the target lane line in the 2D detection frame of the target vehicle.

[0070] In practice, a vehicle detection model is used to detect images taken by cameras set up at the roadside, and outputs 2D detection boxes of the detected vehicles. The center point of the bottom edge of the detection box is processed and used as the image pixel coordinates of the vehicle. Lane lines are detected using lane lines, and the pixel coordinates of the lane lines are output. The corresponding slope is calculated according to different lane lines.

[0071] Furthermore, based on the slope, it is determined whether the vehicle is turning by whether the vehicle's heading angle is parallel to the lane line slope.

[0072] Finally, the image pixel coordinate positioning result of the target vehicle in the 2D detection frame of the target vehicle in the turning state is adjusted to the pixel coordinate position of the target lane line.

[0073] In one embodiment of this application, the first state of the target vehicle's driving state includes: the target vehicle exhibiting a tilt angle exceeding a preset threshold in a turning lane; if the target vehicle does not exhibit a tilt angle exceeding the preset threshold in a turning lane, then the target vehicle's driving state is in a second state.

[0074] In practice, lane lines are detected using lane line detection, and their pixel coordinates are output. The corresponding slope is calculated for different lane lines. Based on a preset threshold, the vehicle's heading angle is calculated by connecting vehicle locations from previous and subsequent frames. Whether the vehicle's heading angle is parallel to the lane line slope determines if the vehicle is turning. If the vehicle's heading angle is not parallel to the lane line slope, the vehicle is considered not to be turning.

[0075] Alternatively, the typical scenario of turning refers to turning or making a U-turn at an intersection, including but not limited to right-turn and left-turn intersections. At a right-turn intersection, the turning radius is smaller, and the vehicle's tilt is greater. Conversely, at a left-turn intersection, the turning radius is larger, and the vehicle's tilt is smaller. Therefore, the second and first states can also be the results of judgments based on different turning directions (left or right).

[0076] like Figure 3 As shown, the vehicle's left or right turn is determined based on the heading angle. The original positioning point is P1. For a vehicle in a right turn state, if the 2D bounding box intersects the lane line, the intersection point P(where P is the lane line and the 2D bounding box) is calculated. Figure 3 In the special case (equivalent to P1), the center point P2 of the left part of the bottom intersection point P is used as the positioning point, using the intersection point P(P1) and the pixel coordinates of the box.

[0077] It is evident that point P2 is more suitable for representing the actual vehicle positioning point P3 than point P1 in the detection frame. If a vehicle is significantly tilted to the right during a right turn, the original positioning error can reach at least 2-3 meters. Constraints can greatly reduce this error. Furthermore, if a large tilt angle occurs on a right turn, the original positioning might result in the vehicle being positioned in a flowerbed.

[0078] After adopting the positioning method of this application, the original positioning point P1 can be replaced by positioning point P2, thereby getting closer to the actual positioning point P3.

[0079] In one embodiment of this application, the pre-trained vehicle detection model includes: training a 2D vehicle detection model based on the YOLOv7 object detection network, using the original road image data and corresponding labels as training data.

[0080] In practical implementation, the YOLOv7 object detection network can effectively detect 2D vehicle models. Furthermore, unlike other detection networks in related technologies, it offers higher accuracy and faster speed, meeting the actual needs of roadside vehicle detection.

[0081] In one embodiment of this application, the pre-trained lane detection model includes: training using multiple sets of data through machine learning, wherein each set of data includes: an image containing vertical lane lines and lane line type labels corresponding to solid lines, dashed lines, double solid lines, and double dashed lines in the image.

[0082] In practice, when training the lane detection model on the image dataset, multiple sets of data are used for machine learning training. Each set of data includes an image of the vertical lane lines and corresponding lane line type labels (solid line, dashed line, double solid line, double dashed line). In other words, in addition to containing the vertical lane lines, the model also includes lane line type label information. The lane detection model outputs multiple pixels (lane line center points). Since lane lines have line widths, the detected multiple pixels are considered as a single line, and the center line position of the solid line, dashed line, double solid line, or double dashed line is calculated.

[0083] In addition, based on the intersecting lane lines, it can be determined that the target vehicle deviates at a greater angle during the right turn.

[0084] By incorporating multiple lane line type labels during the training phase, the lane line detection model can output lane line detection results, including the type of the detected lane line. This lane line type information can be used to subsequently determine the target vehicle's state by combining it with the vehicle's YAW heading angle.

[0085] In one embodiment of this application, acquiring image information collected at the roadside includes: acquiring image information from the roadside perspective, wherein the image information includes at least one of the following: image data of vehicles traveling on the road, and image data of lane lines on the road.

[0086] In practice, the image data of vehicles traveling on the road and the image data of lane lines on the road can be image information collected from the same roadside or image information collected from multiple roadside locations. For example, in some complex intersection scenarios, the roadside equipment needs to locate and track target vehicles.

[0087] This application embodiment also provides a vehicle positioning device 200, such as Figure 2 As shown, a structural schematic diagram of a vehicle positioning device in an embodiment of this application is provided. The vehicle positioning device 200 includes at least: an acquisition module 210, a first detection module 220, a second detection module 230, and a constraint module 240, wherein:

[0088] In one embodiment of this application, the acquisition module 210 is specifically used to: acquire image information collected at the roadside.

[0089] Image information is obtained from the perspective of the roadside. This image information is used as the training set during the training phase and as the image data to be detected during the detection phase.

[0090] The term "roadside" here typically refers to roadside units deployed at intersections or road sections. These units use cameras to capture image information from the roadside perspective. Intersections include, but are not limited to, right-turn and left-turn intersections. At right-turn intersections, the turning radius is smaller, resulting in greater vehicle tilt. Conversely, at left-turn intersections, the turning radius is larger, leading to less vehicle tilt.

[0091] Image information typically includes images containing lane lines and images containing vehicles, usually divided into two different sets of images. It is important to note that, based on prior knowledge, vehicles usually travel within lane lines (lane center lines), so the detected vehicle localization results can be constrained using prior knowledge.

[0092] At the same time, it's important to consider that roadside image information typically has a fixed background. For example, roadside green belts or flower beds. If image information from the same roadside source is used, the background information remains largely constant.

[0093] Furthermore, the significant tilt angle that vehicles exhibit when turning or making a U-turn should be considered. Using the same image information collected from the same roadside location will not yield accurate positioning results. For example, the original 2D vehicle detection bounding box might place the vehicle within a flower bed, resulting in positioning errors.

[0094] In one embodiment of this application, the first detection module 220 is specifically used to: input the image information collected at the road end into a pre-trained lane line detection model to detect the target lane line.

[0095] The image information collected at the roadside is input into a pre-trained lane detection model, which can detect multiple lane lines and the types of these lane lines.

[0096] For example, during training, roadside lane line data can be collected: marking solid lines, dashed lines, double solid lines (including oncoming lanes during turns), double dashed lines, and other vertical lane lines, and processing the marked .json files as label data. Then, using the label data and the original training data as training samples, the built neural network model is trained and saved as a lane line detection model.

[0097] Preferably, the lane line detection model adds recognition results for different lane line types, which can identify lane lines in the image information collected at the road end and the type of lane line described therein.

[0098] In one embodiment of this application, the second detection module 230 is specifically used to: input the image information collected at the roadside into a pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle positioning result.

[0099] The image information collected at the roadside is also input into the vehicle detection model to detect the location of the target vehicle. Of course, it can also include the type of vehicle, such as a truck, a car, or a bicycle.

[0100] For example, vehicle data is collected from roadside angles, vehicles are labeled, and the labeled .json files are processed to correspond to the original image labels. A YOLOv7 object detection network is built, and the original data and corresponding labels are used as training data to train a 2D vehicle detection model and output the vehicle detection model.

[0101] It is understood that other target detection networks can also be used. This application does not specifically limit the network in its embodiments, and those skilled in the art can make different choices based on the actual application scenario.

[0102] The image information input to the vehicle detection model and the lane detection model can be from the same frame, which can improve the accuracy of image recognition.

[0103] In one embodiment of this application, the constraint module 240 is specifically used to: constrain the positioning result of the target vehicle according to the target lane line to obtain the final positioning result of the target vehicle.

[0104] The positioning result of the target vehicle is constrained based on the target lane lines in the detection results of the two detection models. This can be understood as including, but not limited to, adjusting the vehicle's relative position coordinates laterally or longitudinally. After constraint and adjustment, the final positioning result of the target vehicle can be obtained.

[0105] Furthermore, whether or not constraints are needed also depends on whether the vehicle is currently turning left or right. Therefore, the vehicle's state needs to be determined by the vehicle's heading angle (YAW), and then constraints can be applied based on the conditions.

[0106] It is understood that the above-mentioned vehicle positioning device can realize each step of the vehicle positioning method provided in the foregoing embodiments. The relevant explanations of the vehicle positioning method are applicable to the vehicle positioning device and will not be repeated here.

[0107] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0108] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0109] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0110] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a vehicle positioning device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0111] Acquire image information collected at the roadside;

[0112] The image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane line;

[0113] The image information collected at the roadside is input into a pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle localization result;

[0114] The positioning result of the target vehicle is constrained based on the target lane line to obtain the final positioning result of the target vehicle.

[0115] The above is as stated in this application. Figure 1The method executed by the vehicle positioning device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0116] The electronic device can also perform Figure 1 The method for executing the vehicle positioning device, and realizing the vehicle positioning device in Figure 1 The functions of the embodiments shown are not described in detail here.

[0117] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the vehicle positioning device in the illustrated embodiment is specifically used to perform the following:

[0118] Acquire image information collected at the roadside;

[0119] The image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane line;

[0120] The image information collected at the roadside is input into a pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle localization result;

[0121] The positioning result of the target vehicle is constrained based on the target lane line to obtain the final positioning result of the target vehicle.

[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0128] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A vehicle positioning method, wherein, The method includes: Acquire image information collected at the roadside; The image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane line; The image information collected at the roadside is input into a pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle localization result; The positioning result of the target vehicle is constrained based on the target lane line to obtain the final positioning result of the target vehicle; The step of constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle further includes: Based on the target lane line, determine the pixel coordinate position of the target lane line and calculate the slope parameter of the target lane line according to the type of lane line; The driving state of the target vehicle is determined based on the heading angle of the target vehicle and the slope parameter of the target lane line, wherein the heading angle of the target vehicle is calculated by connecting the vehicle positioning points of the target vehicle positioning results in adjacent image frames. The step of constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle includes: Based on the 2D detection box in the target vehicle positioning result, determine the coordinates of the bottom center point of the 2D detection box, and use the coordinates of the bottom center point as the image pixel coordinate positioning result of the target vehicle. When the target vehicle is in the first driving state, the image pixel coordinate positioning result of the target vehicle is adjusted according to the pixel coordinate position of the target lane line; The step of constraining the positioning result of the target vehicle based on the target lane line includes: Using the center position of the lane line as the prior position, and based on the pixel coordinate position of the target lane line as the standard, the image pixel coordinate positioning result of the target vehicle is adjusted to the pixel coordinate position of the target lane line within the 2D detection frame of the target vehicle.

2. The method as described in claim 1, wherein, The image information collected at the roadside is input into a pre-trained lane detection model to detect the target lane lines, including: The detection results of lane lines are processed by image segmentation to obtain each lane line instance as the pixel segmentation result of the lane line; The segmentation results of each lane line with a width of multiple pixels are fitted to obtain a line with a width of one pixel for that lane line as the target lane line.

3. The method as described in claim 1, wherein, The pre-trained lane detection model includes: The lane detection model is trained using machine learning using multiple sets of data. Each set of data includes: an image of a vertical lane line and lane line type labels for solid lines, dashed lines, double solid lines, and double dashed lines in the image.

4. The method as described in claim 1, wherein, The acquisition of image information collected at the roadside includes: Acquire image information from the roadside perspective, wherein the image information includes at least one of the following: image data of vehicles traveling on the road, and image data of lane lines on the road.

5. A vehicle positioning device, wherein, The device includes: The acquisition module is used to acquire image information collected at the roadside. The first detection module is used to input the image information collected at the roadside into a pre-trained lane line detection model to detect the target lane line; The second detection module is used to input the image information collected at the roadside into a pre-trained vehicle detection model to detect the target vehicle, wherein the target vehicle includes the target vehicle positioning result; The constraint module is used to constrain the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle. The step of constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle further includes: Based on the target lane line, determine the pixel coordinate position of the target lane line and calculate the slope parameter of the target lane line according to the type of lane line; The driving state of the target vehicle is determined based on the heading angle of the target vehicle and the slope parameter of the target lane line, wherein the heading angle of the target vehicle is calculated by connecting the vehicle positioning points of the target vehicle positioning results in adjacent image frames. The step of constraining the positioning result of the target vehicle based on the target lane line to obtain the final positioning result of the target vehicle includes: Based on the 2D detection box in the target vehicle positioning result, determine the coordinates of the bottom center point of the 2D detection box, and use the coordinates of the bottom center point as the image pixel coordinate positioning result of the target vehicle. When the target vehicle is in the first driving state, the image pixel coordinate positioning result of the target vehicle is adjusted according to the pixel coordinate position of the target lane line; The step of constraining the positioning result of the target vehicle based on the target lane line includes: Using the center position of the lane line as the prior position, and based on the pixel coordinate position of the target lane line as the standard, the image pixel coordinate positioning result of the target vehicle is adjusted to the pixel coordinate position of the target lane line within the 2D detection frame of the target vehicle.

6. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 4.

7. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 4.

Images