Obstacle line pressing judgment method and device, electronic equipment and storage medium

Abstract

The invention discloses an obstacle line pressing judgment method and device, electronic equipment and a storage medium. The method comprises the steps that convex hull outer contour information of pixel points of a 2D image of a camera is acquired; in response to the obstacle tracking trajectory information, projecting external contour information of an obstacle tracking trajectory under 3D coordinates to a 2D image of the camera; and judging whether the outer contour information of the convex hull is matched with the external contour information in the 2D image of the camera or not, and taking a matching result as a judgment result of whether an obstacle presses a line or not. According to the invention, accurate judgment of line pressing of the obstacle is realized.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and particularly to an obstacle lane line crossing judgment method, device, electronic device, and storage medium. Background Art

[0002] In the field of autonomous driving, the planning module needs to obtain whether an obstacle has crossed the left lane line or the right lane line of the lane where the vehicle is located, or has not crossed any lane lines, so as to perform processing on whether to bypass the obstacle or change lanes.

[0003] When judging whether a vehicle has crossed a lane line, usually when the accuracy of the offline map is very low, the lane line crossing judgment is made by using the 3D perception information of the offline map and the obstacle tracking trajectory track, resulting in low accuracy. At the same time, if the result is obtained from the image perception of the camera, it will be relatively accurate.

[0004] If only the image perception or the method of matching the obstacle tracking trajectory track with the offline map is adopted, it is still not possible to achieve good obstacle lane line crossing judgment. Summary of the Invention

[0005] Embodiments of this application provide an obstacle lane line crossing judgment method, device, electronic device, and storage medium, so as to utilize the 2D image to detect the correctness of lane line crossing, and match the lane line crossing attribute of the obstacle in the 3D coordinates through the projection of the circumscribed contour on the 2D image as the basis for judging whether the vehicle has crossed the lane line.

[0006] Embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide an obstacle lane line crossing judgment method, where the judgment method includes:

[0008] Obtain the convex hull outer contour information of the camera 2D image pixel points;

[0009] In response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinates onto the camera 2D image;

[0010] Judge whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the judgment result of whether the vehicle has crossed the lane line.

[0011] In some embodiments, the step of judging whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and using the matching result as the judgment result of whether the vehicle has crossed the lane line includes:

[0012] Traverse each obstacle tracking trajectory to screen out whether there is an obstacle tracking trajectory within the sensing area according to the sensing area of the camera;

[0013] If so, align the time stamps of the obstacle tracking trajectory and the convex hull outer contour information;

[0014] If not, it is determined that it is uncertain whether the line is crossed.

[0015] In some embodiments, the aligning the time stamps of the obstacle tracking trajectory and the convex hull outer contour information includes:

[0016] Perform linear alignment on the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system;

[0017] Obtain the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through a transformation matrix;

[0018] According to the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, transfer the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, and delete the circumscribed contour points outside the pixel coordinate system.

[0019] In some embodiments, after transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further includes:

[0020] Determine whether the number of remaining circumscribed contour points is greater than or equal to 3 after transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system;

[0021] If so, calculate the similarity between the outer contour formed by the remaining outer contour points of the obstacle tracking trajectory in the pixel coordinate system and the convex hull outer contour information;

[0022] If the similarity calculation result meets the requirements, it is determined that the line is crossed.

[0023] In some embodiments, after transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further includes:

[0024] Determine whether the number of remaining circumscribed contour points is less than 3 when transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system;

[0025] If so, calculate the distance value from each point in the convex hull outer contour information to each side of the outer contour formed by the remaining circumscribed contour points;

[0026] If the distance value meets the requirements, it is determined that the line is crossed.

[0027] In some embodiments, the obtaining the convex hull outer contour information of the camera 2D image pixel points includes:

[0028] After the camera is initialized, a conversion matrix from the world coordinate system to the camera coordinate system, the pixel resolution of the camera, and a conversion function from the camera coordinate system to the pixel coordinate system are obtained;

[0029] Receive the lane line pixel points set collected by the camera, where the lane line pixel points set includes a left lane line pixel points set and a right lane line pixel points set;

[0030] After clustering according to the left lane line pixel points set and / or the right lane line pixel points set, a plurality of point clusters are obtained, and each point cluster is used as a pixel point on the line;

[0031] Generate convex hull data of the circumscribed contour according to the multiple point clusters and put it into the lane marking queue.

[0032] In some embodiments, the projecting the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate onto the camera 2D image in response to the obstacle tracking trajectory information includes:

[0033] When receiving the obstacle tracking trajectory information, find the convex hull data of the circumscribed contour of the frame closest to the obstacle tracking trajectory from the lane marking queue;

[0034] Generate a conversion matrix from the UTM coordinate system to the ego-vehicle coordinate system according to the positioning data of the convex hull data of the circumscribed contour of the closest frame, and convert the obstacle tracking trajectory data from the UTM coordinate system to the ego-vehicle coordinate system,

[0035] Wherein, when converting from the UTM coordinate system to the ego-vehicle coordinate system, the height of the circumscribed contour in the UTM coordinate system takes the ground height where the obstacle tracking trajectory is located.

[0036] In a second aspect, an embodiment of the present application further provides an obstacle pixel point on the line determination device, where the determination device includes:

[0037] An acquisition module, configured to acquire the convex hull outer contour information of the camera 2D image pixel points;

[0038] A response module, configured to project the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate onto the camera 2D image in response to the obstacle tracking trajectory information;

[0039] A matching module, configured to determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the determination result of whether the obstacle is on the line.

[0040] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, where the executable instructions, when executed, cause the processor to execute the above method.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, cause the electronic device to execute the above method.

[0042] The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects:

[0043] First, obtain the convex hull outer contour information of the camera 2D image pixels, and then, in response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image. Finally, determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the determination result of whether the obstacle presses the line. Through the joint processing and judgment of the line-pressing segmentation result of the 2D image and the 3D detection information of the obstacle tracking trajectory track, the above method realizes the judgment of whether the perceived obstacle presses the lane line. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0045] Figure 1 is a schematic flowchart of a method for judging whether an obstacle presses a line in an embodiment of the present application;

[0046] Figure 2 is a schematic structural diagram of an obstacle line-pressing judgment device in an embodiment of the present application;

[0047] Figure 3 is a schematic structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0049] The following will describe in detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0050] An embodiment of the present application provides a method for judging whether an obstacle presses a line, as Figure 1As shown in the figure, a schematic flowchart of the obstacle line - pressing judgment method in the embodiments of the present application is provided. The method at least includes the following steps S110 to S130:

[0051] Step S110: Obtain the outer contour information of the convex hull of the camera 2D image pixel points.

[0052] The camera 2D image pixel points are obtained based on the 2D image data collected by at least one camera, and the outer contour information of the convex hull is determined according to the camera 2D image pixel points. In the embodiments of the present application, the specific number and position of the cameras are not limited, and are subject to the position and number of sensors deployed on the autonomous vehicle in the actual scenario.

[0053] Step S120: In response to the obstacle tracking trajectory information, project the outer contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image.

[0054] Trigger with the obstacle tracking trajectory track data (using lidar as the main sensor). After receiving the obstacle tracking trajectory track data, project the outer contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image, and perform outer contour information judgment after projection processing.

[0055] Step S130: Judge whether the outer contour information of the convex hull matches the outer contour information in the camera 2D image, and use the matching result as the judgment result of whether the obstacle presses the line.

[0056] In the case of judging that the outer contour information of the convex hull matches the outer contour information in the camera 2D image, the matching result is used as the judgment result of whether the vehicle presses the line in different cases. Finally, give the attribute of whether the convex hull pressing the line is on the left lane or the right lane to the obstacle tracking trajectory track.

[0057] Through the above method, by performing relevant segmentation algorithms and post - processing on the 2D image, a pixel point set of the obstacle line - pressing position can be obtained; then project the polygon of the obstacle tracking trajectory track in 3D coordinates onto the 2D image and match it with the line - pressing pixel point set, and the line - pressing attribute of the obstacle tracking trajectory track can be obtained.

[0058] In an embodiment of the present application, the step of judging whether the outer contour information of the convex hull matches the outer contour information in the camera 2D image and using the matching result as the judgment result of whether the vehicle presses the line includes: traversing each obstacle tracking trajectory, screening out whether there is an obstacle tracking trajectory within the sensing area according to the sensing area of the camera; if so, align the time stamps of the obstacle tracking trajectory and the outer contour information of the convex hull; if not, judge that it is uncertain whether the line is pressed.

[0059] When determining whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, it is necessary to traverse each obstacle tracking trajectory track. Usually, it can be filtered first according to the camera sensing area. Those not within the sensing area are judged as uncertain whether they cross the line, and those within the sensing area are further judged. It can be understood that the camera sensing area refers to the angular area and the distance area. For example, the FOV field of view angle of the camera sensing area is 120 degrees.

[0060] In an embodiment of the present application, the time stamp alignment of the obstacle tracking trajectory and the convex hull outer contour information includes: linearly aligning the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system; obtaining the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through a transformation matrix; according to the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, and deleting the circumscribed contour points transferred outside the pixel coordinate system.

[0061] Since the time stamps of the line crossing data of the obstacle tracking trajectory track and the 2D image pixel points are different, time alignment is required:

[0062] (1) First, linearly align the polygon of the obstacle tracking trajectory track in the UTM coordinate system to complete time alignment; (2) Then, obtain the polygon (circumscribed contour) of the obstacle tracking trajectory track in the camera coordinate system through the transformation matrix world2camera, and then transfer the polygon (circumscribed contour) of the obstacle tracking trajectory track from the camera coordinate system to the pixel coordinate system through the function matrix (internal and external parameters) in the camera model, and delete the polygon (circumscribed contour) points transferred outside the pixel.

[0063] In an embodiment of the present application, after transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further includes: judging whether the number of remaining circumscribed contour points is greater than or equal to 3 after transferring the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system; if so, calculating the similarity between the circumscribed contour formed by the remaining external contour points of the obstacle tracking trajectory in the pixel coordinate system and the convex hull outer contour information; if the similarity calculation result meets the requirements, it is judged as crossing the line.

[0064] Calculate the similarity between the polygon (outer contour) with three or more points in the pixel coordinate system of the obstacle tracking trajectory track and the convex hull of the line-crossing. If the result is greater than 0, it indicates that the track crosses the line. For a polygon with less than three points, calculate the distance from each point of the convex hull of the line-crossing to each side of the polygon. If the distance is less than a certain threshold, it is also determined that the track crosses the line.

[0065] It should be noted that through the above calculations, the remaining polygon points in the pixel coordinate system may be greater than or equal to three, or may be less than three.

[0066] When converting from the UTM coordinate system to the ego-vehicle coordinate system, the height of the polygon in the UTM coordinate system is taken as the ground height where the obstacle tracking trajectory track is located, that is, only the pixel points projected on the vehicle bottom. Considering the situation of occlusion, usually, the height information of the center point of the obstacle tracking trajectory track in the UTM coordinate system is subtracted by half of the height dimension of the obstacle tracking trajectory track.

[0067] In an embodiment of the present application, after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further includes: judging whether the number of remaining circumscribed contour points is less than three when converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system; if so, calculate the distance value from each point in the convex hull circumscribed contour information to each side of the circumscribed contour formed by the remaining circumscribed contour points; if the distance value meets the requirements, it is judged that the track crosses the line.

[0068] In order to better obtain the judgment result, when it is judged that the number of remaining circumscribed contour points is less than three when converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, the distance value from each point in the convex hull circumscribed contour information to each side of the circumscribed contour can be calculated to judge whether the track crosses the line.

[0069] In an embodiment of the present application, the obtaining of the convex hull circumscribed contour information of the camera 2D image pixel points includes: obtaining the conversion matrix from the world coordinate system to the camera coordinate system, the pixel resolution of the camera, and the conversion function from the camera coordinate system to the pixel coordinate system after the camera is initialized; receiving the lane line crossing pixel point set collected by the camera, where the lane line crossing pixel point set includes the left lane line crossing pixel point set and the right lane line crossing pixel point set; clustering the left lane line crossing pixel point set and / or the right lane line crossing pixel point set to obtain a plurality of point clusters, and each point cluster is used as a line-crossing point; generating circumscribed contour convex hull data according to the plurality of point clusters and putting it into the lane marking queue.

[0070] At the beginning of initialization, by reading the external camera parameters, a transformation matrix world2camera from the world coordinate system to the camera coordinate system is generated. The camera model is obtained to acquire the pixel resolution of the camera and the transformation function from the camera coordinate system to the pixel coordinate system.

[0071] Then, the left lane line pressing pixel point set and the right lane line pressing pixel point set given by the same 2D camera are received and then converted into the required format for fusion, that is, the pixel point set contains many lane lines.

[0072] Furthermore, the left and right point sets are respectively clustered according to the Euclidean distance to obtain point clusters, and each point cluster represents a pressing point. Then, the convex hulls of the clustered point clusters are generated, that is, the convex circumscribed polygon of the point clusters is generated.

[0073] Finally, all the convex hulls generated in the current frame are put into the queue lane_markers_list. They are put into the lane_markers_list queue and triggered by track. The time frequencies are inconsistent, and operations such as denoising and time alignment are performed in the queue.

[0074] Furthermore, triggered by the track data, after the track data is received, the convex hull data of the lane pressing closest to the timestamp of the obstacle tracking trajectory track is found from the lane_markers_list and extracted. Then, the positioning data closest to the timestamp of the track is obtained, and a transformation matrix g2v from the UTM coordinate system to the ego-vehicle coordinate system (which needs to be transformed to the ego-vehicle coordinate system) of the obstacle tracking trajectory track data is generated.

[0075] In an embodiment of the present application, the projecting the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinates onto the camera 2D image in response to the obstacle tracking trajectory information includes: in the case of receiving the obstacle tracking trajectory information, finding the convex hull data of the circumscribed contour of the frame closest to the obstacle tracking trajectory from the lane marker queue; generating a transformation matrix from the UTM coordinate system to the ego-vehicle coordinate system according to the positioning data of the convex hull data of the circumscribed contour of the closest frame, and transforming the obstacle tracking trajectory data from the UTM coordinate system to the ego-vehicle coordinate system. When transforming from the UTM coordinate system to the ego-vehicle coordinate system, the height of the circumscribed contour in the UTM coordinate system takes the ground height where the obstacle tracking trajectory is located.

[0076] Triggered by the obstacle tracking trajectory track data, after the track data is received, the convex hull data of the lane pressing closest to the timestamp or distance of the obstacle tracking trajectory track is found from the above-mentioned lane_markers_list and extracted, and then the positioning data closest to the timestamp of the obstacle tracking trajectory track is obtained.

[0077] The embodiment of the present application also provides an obstacle line - pressing judgment device 200, as Figure 2 shown, which provides a structural schematic diagram of the obstacle line - pressing judgment device in the embodiment of the present application. The obstacle line - pressing judgment device 200 at least includes: an acquisition module 210, a response module 220, and a matching module 230, where:

[0078] In an embodiment of the present application, the acquisition module 210 is specifically configured to: acquire the convex hull outer contour information of the camera 2D image pixel points.

[0079] The camera 2D image pixel points are obtained according to the 2D image data collected by at least one camera, and the convex hull outer contour information is determined based on the camera 2D image pixel points. In the embodiment of the present application, the specific number and position of the cameras are not limited, and are subject to the position and number of sensors deployed on the autonomous vehicle in the actual scenario.

[0080] In an embodiment of the present application, the response module 220 is specifically configured to: in response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate onto the camera 2D image.

[0081] Trigger with the obstacle tracking trajectory track data (lidar as the main sensor). After receiving the obstacle tracking trajectory track data, project the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate onto the camera 2D image, and perform outer contour information judgment after projection processing.

[0082] In an embodiment of the present application, the matching module 230 is specifically configured to: judge whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the judgment result of whether the obstacle presses the line.

[0083] In the case of judging that the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, then divide into different cases and use the matching result as the judgment result of whether the vehicle presses the line. Finally, give the attribute of whether the convex hull pressing the line is the left lane or the right lane to the obstacle tracking trajectory track.

[0084] In an embodiment of the present application, the matching module 230 is further configured to:

[0085] Traverse each of the obstacle tracking trajectories, and screen out whether there is an obstacle tracking trajectory within the sensing area according to the sensing area of the camera;

[0086] If so, align the time stamps of the obstacle tracking trajectory and the convex hull outer contour information;

[0087] If not, it is determined that it is uncertain whether the line is crossed.

[0088] In an embodiment of the present application, the matching module 230 is further configured to:

[0089] Perform linear alignment on the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system;

[0090] Obtain the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through the transformation matrix;

[0091] According to the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, transfer the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, and delete the circumscribed contour points outside the converted pixel coordinate system.

[0092] In an embodiment of the present application, it further includes a first judgment module, which is used for:

[0093] Judge whether the number of remaining circumscribed contour points is greater than or equal to 3 after the circumscribed contour information of the obstacle tracking trajectory is transferred from the camera coordinate system to the pixel coordinate system;

[0094] If so, calculate the similarity between the circumscribed contour formed by the remaining external contour points of the obstacle tracking trajectory in the pixel coordinate system and the convex hull external contour information;

[0095] If the similarity calculation result meets the requirements, it is determined that the line is crossed.

[0096] In an embodiment of the present application, it further includes a second judgment module, which is used for:

[0097] Judge whether the number of remaining circumscribed contour points is less than 3 when the circumscribed contour information of the obstacle tracking trajectory is transferred from the camera coordinate system to the pixel coordinate system;

[0098] If so, calculate the distance value from each point in the convex hull external contour information to each side of the external contour formed by the remaining circumscribed contour points;

[0099] If the distance value meets the requirements, it is determined that the line is crossed.

[0100] In an embodiment of the present application, the acquisition module 310 is further configured to:

[0101] After the camera is initialized, obtain the transformation matrix from the world coordinate system to the camera coordinate system, the pixel resolution of the camera, and the transformation function from the camera coordinate system to the pixel coordinate system;

[0102] Receive the lane line crossing pixel point set collected by the camera, and the lane line crossing pixel point set includes a left lane line crossing pixel point set and a right lane line crossing pixel point set;

[0103] After clustering the left lane line pressing pixel point set and / or the right lane line pressing pixel point set, a plurality of point clusters are obtained, and each of the point clusters is used as a pressing point.

[0104] Generate the circumscribed contour convex hull data according to the multiple point clusters and put it into the lane marking queue.

[0105] In an embodiment of the present application, the response module 320 is further configured to:

[0106] When receiving the obstacle tracking trajectory information, find out the circumscribed contour convex hull data of the frame closest to the obstacle tracking trajectory from the lane marking queue;

[0107] Generate a conversion matrix for converting the UTM coordinate system to the vehicle coordinate system according to the positioning data of the circumscribed contour convex hull data of the closest frame, and convert the obstacle tracking trajectory data from the UTM coordinate system to the vehicle coordinate system.

[0108] Wherein, when converting from the UTM coordinate system to the vehicle coordinate system, the height of the circumscribed contour in the UTM coordinate system takes the ground height where the obstacle tracking trajectory is located.

[0109] It can be understood that the above-mentioned obstacle pressing line judgment device can implement each step of the obstacle pressing line judgment method provided in the foregoing embodiment. The relevant explanations about the obstacle pressing line judgment method are applicable to the obstacle pressing line judgment device, and will not be elaborated here.

[0110] Figure 3 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 3 . At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0111] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 only a bidirectional arrow is used in

[0112] Memory, which is used to store programs. Specifically, the program can include program code, and the program code includes computer operation instructions. The memory can include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0113] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an obstacle line-pressing judgment device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0114] Obtain the convex hull outer contour information of the camera 2D image pixel points;

[0115] In response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image;

[0116] Determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the judgment result of whether the vehicle presses the line.

[0117] The above as in this application Figure 1The method executed by the obstacle line-pressing determination device disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuit in hardware in the processor or by instructions in software form. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0118] The electronic device can also execute Figure 1 the method executed by the obstacle line-pressing determination device in Figure 1 the illustrated embodiment and implement the functions of the obstacle line-pressing determination device in

[0119] The embodiments of the present application also propose a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 1 the method executed by the obstacle line-pressing determination device in the illustrated embodiment, and specifically used to execute:

[0120] Obtain the convex hull outer contour information of the camera 2D image pixel points;

[0121] In response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image;

[0122] Determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the determination result of whether the vehicle crosses the line.

[0123] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0125] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0127] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0128] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0129] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0130] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0132] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for judging obstacle line pressing, wherein, The determination method includes: Obtaining the outer contour information of the convex hull of the pixel points of the camera 2D image; In response to the obstacle tracking trajectory information, projecting the outer contour information of the obstacle tracking trajectory in the 3D coordinate system onto the camera 2D image; Determining whether the outer contour information of the convex hull matches the outer contour information in the camera 2D image, and using the matching result as the determination result of whether the obstacle presses the line.

2. The method according to claim 1, wherein The step of determining whether the outer contour information of the convex hull matches the outer contour information in the camera 2D image, and using the matching result as the determination result of whether the obstacle presses the line includes: Traversing each obstacle tracking trajectory, and screening out whether there is an obstacle tracking trajectory within the sensing area according to the sensing area of the camera; If so, aligning the time stamps of the obstacle tracking trajectory and the outer contour information of the convex hull; If not, it is determined that it is uncertain whether the line is pressed.

3. The method according to claim 2, wherein The step of aligning the time stamps of the obstacle tracking trajectory and the outer contour information of the convex hull includes: Performing linear alignment on the outer contour of the obstacle tracking trajectory in the UTM coordinate system; Obtaining the outer contour information of the obstacle tracking trajectory in the camera coordinate system through a transformation matrix; According to the outer contour information of the obstacle tracking trajectory in the camera coordinate system, transforming the outer contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, and deleting the outer contour points outside the pixel coordinate system after transformation.

4. The method according to claim 3, after transforming the outer contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, further including: Determining whether the number of remaining outer contour points is greater than or equal to 3 after transforming the outer contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system; If so, calculating the similarity between the outer contour formed by the remaining outer contour points of the obstacle tracking trajectory in the pixel coordinate system and the outer contour information of the convex hull; If the similarity calculation result meets the requirements, it is determined that the line is pressed.

5. The method according to claim 3, wherein, After transforming the outer contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, further including: Determining whether the number of remaining outer contour points is less than 3 when transforming the outer contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system; If so, calculating the distance value from each point in the outer contour information of the convex hull to each side of the outer contour formed by the remaining outer contour points; If the distance value meets the requirements, it is determined that the line is pressed.

6. The method according to claim 1, wherein The step of obtaining the outer contour information of the pixel points of the camera 2D image includes: After the camera is initialized, obtaining the transformation matrix from the world coordinate system to the camera coordinate system, the pixel resolution of the camera, and the transformation function from the camera coordinate system to the pixel coordinate system; Receiving the lane line pressing pixel point set collected by the camera, where the lane line pressing pixel point set includes a left lane line pressing pixel point set and a right lane line pressing pixel point set; Clustering the left lane line pressing pixel point set and / or the right lane line pressing pixel point set to obtain a plurality of point clusters, and each point cluster is used as a line pressing point. Generate the convex hull data of the circumscribed contour based on multiple point clusters and put it into the lane marking queue.

7. The method according to claim 6, wherein The projection of the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image in response to the obstacle tracking trajectory information includes: When receiving the obstacle tracking trajectory information, find the convex hull data of the circumscribed contour of the frame closest to the obstacle tracking trajectory from the lane marking queue; Generate a transformation matrix from the UTM coordinate system to the ego-vehicle coordinate system based on the positioning data of the convex hull data of the closest frame, and transform the obstacle tracking trajectory data from the UTM coordinate system to the ego-vehicle coordinate system, wherein, when converting from the UTM coordinate system to the ego-vehicle coordinate system, the height of the circumscribed contour in the UTM coordinate system takes the ground height where the obstacle tracking trajectory is located.

8. An obstacle wire pressing judgment device, wherein, The judging device includes: An acquisition module for acquiring the convex hull outer contour information of the pixel points of the camera 2D image; A response module for projecting the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image in response to the obstacle tracking trajectory information; A matching module for judging whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and taking the matching result as the judging result of whether the obstacle presses the line.

9. An electronic device, comprising: A processor; And A memory arranged to store computer-executable instructions, which when executed cause the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which when executed by an electronic device including a plurality of application programs, cause the electronic device to execute the method according to any one of claims 1 to 7.

Images