Obstacle line-pressing judgment method, device, electronic device, and storage medium

Abstract

The application discloses an obstacle line pressing judgment method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring convex hull outer contour information of a camera 2D image pixel point; in response to obstacle tracking trajectory information, projecting the outer contour information of the obstacle tracking trajectory in a 3D coordinate on a camera 2D image; judging whether the convex hull outer contour information matches the outer contour information in the camera 2D image or not, and taking the matching result as a judgment result of whether the obstacle presses a line or not. The application realizes accurate judgment of obstacle line pressing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to an obstacle line pressing judgment method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In the field of automatic driving, a planning module needs to obtain whether an obstacle presses a left lane line or a right lane line of a lane where a vehicle is located, or neither, so as to perform processing of whether to go around the obstacle or change lanes.

[0003] When judging whether a vehicle presses a line, usually when the accuracy of an offline map is very low, line pressing judgment is performed by using the offline map and 3D perception information of an obstacle tracking trajectory track, which causes low accuracy. Meanwhile, the result of image perception from a camera is relatively accurate.

[0004] If only image perception or offline map matching of the obstacle tracking trajectory track is used, obstacle line pressing judgment cannot be better achieved. SUMMARY

[0005] Embodiments of the present application provide an obstacle line pressing judgment method and device, an electronic device, and a storage medium, to use 2D image detection to determine the correctness of line pressing, and to use projection of an obstacle's circumscribed contour in 3D coordinates on a 2D image to match the line pressing attribute of the obstacle as a basis for judging whether a vehicle presses a line.

[0006] Embodiments of the present application use the following technical solutions:

[0007] In a first aspect, embodiments of the present application provide an obstacle line pressing judgment method, wherein the judgment method comprises:

[0008] Obtaining convex hull contour information of a camera 2D image pixel point;

[0009] In response to obstacle tracking trajectory information, projecting circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto a camera 2D image;

[0010] Judging whether the convex hull contour information matches the circumscribed contour information in the camera 2D image, and taking the matching result as a judgment result of whether a vehicle presses a line.

[0011] In some embodiments, judging whether the convex hull contour information matches the circumscribed contour information in the camera 2D image, and taking the matching result as a judgment result of whether a vehicle presses a line, comprises:

[0012] Iterating through each of the obstacle tracking trajectories to filter out whether there is an obstacle tracking trajectory in the perception area according to the perception area of the camera;

[0013] If yes, timestamp align the obstacle tracking trajectory with the convex hull contour information;

[0014] If no, determine whether it is a line press or not.

[0015] In some embodiments, the timestamp aligning the obstacle tracking trajectory with the convex hull contour information comprises:

[0016] Linearly aligning the circumscribed contour of the obstacle tracking trajectory under the UTM coordinate system;

[0017] Obtaining the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through a conversion matrix;

[0018] According to the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, and deleting the circumscribed contour points converted to the pixel coordinate system.

[0019] In some embodiments, after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further comprises:

[0020] Determining whether the number of remaining circumscribed contour points after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system is greater than or equal to 3;

[0021] If yes, calculating the similarity between the remaining circumscribed contour of the obstacle tracking trajectory in the pixel coordinate system and the convex hull contour information;

[0022] If the similarity calculation result meets the requirements, it is determined to be a line press.

[0023] In some embodiments, after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further comprises:

[0024] Determining whether the number of remaining circumscribed contour points after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system is less than 3;

[0025] If yes, calculating the distance value of each point in the convex hull contour information to each edge of the remaining circumscribed contour;

[0026] If the distance value meets the requirements, it is determined to be a line press.

[0027] In some embodiments, the obtaining of the convex hull contour information of the camera 2D image pixel point comprises:

[0028] After the camera is initialized, a conversion matrix from a world coordinate system to a camera coordinate system, a pixel resolution of the camera, and a conversion function from the camera coordinate system to a pixel coordinate system are obtained;

[0029] Receiving a lane line pixel set collected by the camera, the lane line pixel set including a left lane line pixel set and a right lane line pixel set;

[0030] After clustering the left lane line pixel set and / or the right lane line pixel set, a plurality of point clusters are obtained, each of the point clusters being used as a lane line pixel;

[0031] Generating an outer contour convex hull data from the plurality of point clusters and putting the outer contour convex hull data into a lane marker queue.

[0032] In some embodiments, the projecting, in response to the obstacle tracking trajectory information, the outer contour information of the obstacle tracking trajectory in 3D coordinates onto a 2D image of the camera, includes:

[0033] In a case where the obstacle tracking trajectory information is received, finding, from the lane marker queue, outer contour convex hull data of a nearest frame in the obstacle tracking trajectory;

[0034] Generating a conversion matrix from a UTM coordinate system to a self-vehicle coordinate system according to positioning data of the outer contour convex hull data of the nearest frame, and converting the obstacle tracking trajectory data from the UTM coordinate system to the self-vehicle coordinate system,

[0035] In the conversion from the UTM coordinate system to the self-vehicle coordinate system, a height of the outer contour in the UTM coordinate system is taken as a ground height of the obstacle tracking trajectory.

[0036] In a second aspect, the embodiments of the present application further provide an obstacle lane line judgment device, wherein the judgment device includes:

[0037] An acquisition module, configured to acquire convex hull outer contour information of a 2D image pixel of a camera;

[0038] A response module, configured to project, in response to obstacle tracking trajectory information, outer contour information of the obstacle tracking trajectory in 3D coordinates onto a 2D image of the camera;

[0039] A matching module, configured to judge whether the convex hull outer contour information matches the outer contour information in the 2D image of the camera, and take a result of the matching as a judgment result of whether the obstacle is in a lane line.

[0040] In a third aspect, the embodiments of the present application further provide an electronic device, including a processor and a memory arranged to store computer executable instructions, the executable instructions, when executed, causing the processor to perform the above method.

[0041] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium storing one or more programs, which, when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method described above.

[0042] The above at least one technical scheme adopted by the embodiments of the present application can achieve the following beneficial effects:

[0043] First, the convex hull contour information of the camera 2D image pixel points is acquired, and then in response to the obstacle tracking trajectory information, the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinates is projected onto the camera 2D image. Finally, it is judged whether the convex hull contour information matches the circumscribed contour information in the camera 2D image, and the matching result is taken as the judgment result of whether the obstacle is pressed line. The above method realizes the judgment of whether the perceived obstacle is pressed lane line through the joint processing of the 2D image pressed line segmentation result and the 3D detection information of the obstacle tracking trajectory track. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 It is a flowchart of the obstacle pressed line judgment method in the embodiments of the present application;

[0046] Figure 2 It is a structural schematic diagram of the obstacle pressed line judgment device in the embodiments of the present application;

[0047] Figure 3 It is a structural schematic diagram of an electronic device in the embodiments of the present application. DETAILED DESCRIPTION

[0048] To make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely in combination with the embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0049] The technical scheme provided by the embodiments of the present application will be described in detail below in combination with the drawings.

[0050] The embodiments of the present application provide an obstacle pressed line judgment method, which comprises the following steps: Figure 1As shown, a flowchart of the obstacle line pressing judgment method in the embodiment of the present application is provided, and the method comprises at least the following steps S110 to S130.

[0051] In step S110, convex hull outer contour information of a camera 2D image pixel point is acquired.

[0052] The convex hull outer contour information is determined according to the camera 2D image pixel point acquired according to the 2D image data collected by at least one camera. In the embodiment of the present application, the specific number and position of the camera are not limited, and the position and number of the sensors deployed in the actual scene of the autonomous vehicle are used as the reference.

[0053] In step S120, in response to the obstacle tracking trajectory information, the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate is projected onto the camera 2D image.

[0054] The circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate is projected onto the camera 2D image in response to the obstacle tracking trajectory track data (laser radar as the main sensor). After the projection processing, the outer contour information is judged.

[0055] In step S130, it is judged whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and the matching result is taken as the judgment result of whether the obstacle presses the line.

[0056] In the case where the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, different cases are divided, and the matching result is taken as the judgment result of whether the vehicle presses the line. Finally, the line pressing convex hull is given to the obstacle tracking trajectory track, and the attribute of the left lane or the right lane is given to the obstacle tracking trajectory track.

[0057] Through the above method, the pixel point set of the obstacle line pressing position can be obtained by performing the related segmentation algorithm and post-processing on the 2D image. Then, the polygon of the obstacle tracking trajectory track in the 3D coordinate is projected onto the 2D image and matched with the line pressing pixel point set, so that the line pressing attribute of the obstacle tracking trajectory track can be obtained.

[0058] In an embodiment of the present application, the judgment of whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and the matching result is taken as the judgment result of whether the vehicle presses the line, comprises: traversing each of the obstacle tracking trajectories, and screening whether there is an obstacle tracking trajectory in the perception area according to the perception area of the camera; if yes, the obstacle tracking trajectory is time-stamped aligned with the convex hull outer contour information; and if no, it is determined that whether the line is pressed is uncertain.

[0059] In judging whether the convex hull contour information matches the circumscribed contour information in the camera 2D image, each obstacle tracking trajectory track needs to be traversed. Generally, it can be filtered according to the camera perception area first, and if it is not in the perception area, it is judged as uncertain whether to press the line, and if it is in the perception area, it is further judged. It can be understood that the camera perception area refers to the angle area and the distance area. For example, the camera perception area FOV field of view angle is 120 degrees.

[0060] In an embodiment of the present application, the timestamp alignment of the obstacle tracking trajectory and the convex hull contour information includes: linearly aligning the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system; obtaining the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through the conversion matrix; converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system according to the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, and deleting the circumscribed contour points converted to the pixel coordinate system.

[0061] Since the obstacle tracking trajectory track and the line pressing data timestamp of the 2D image pixel point are different, time alignment is needed:

[0062] (1) First, linearly align the polygon of the obstacle tracking trajectory track in the UTM coordinate system, complete the time alignment; (2) Then, through the conversion matrix world2camera, the polygon (circumscribed contour) of the obstacle tracking trajectory track in the camera coordinate system is obtained, and then through the function matrix (internal and external parameters) in the camera model, the polygon (circumscribed contour) of the obstacle tracking trajectory track is converted from the camera coordinate system to the pixel coordinate system, and the polygon (circumscribed contour) points converted to the pixel are deleted.

[0063] In an embodiment of the present application, after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, it further includes: judging whether the number of remaining circumscribed contour points after converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system is greater than or equal to 3; if yes, the circumscribed contour composed of the remaining circumscribed contour points of the obstacle tracking trajectory in the pixel coordinate system is calculated. Similarity calculation is performed on the convex hull contour information; if the similarity calculation result meets the requirements, it is judged as pressing the line.

[0064] The polygon (contour) of the obstacle tracking trajectory track with greater than or equal to 3 points in the pixel coordinate system is calculated with the pressure line convex hull, and if the result is greater than 0, it indicates that the track pressure line, for the polygon with less than 3 points, if the distance between each point of the pressure line convex hull and each edge of the polygon is less than a certain threshold, it is also determined as a pressure line.

[0065] It should be noted that the above calculation may result in the remaining polygon points in the pixel coordinate system being greater than or equal to 3, or less than 3.

[0066] In the conversion from the UTM coordinate system to the ego coordinate system, the polygon height of the UTM coordinate system is the ground height of the obstacle tracking trajectory track, that is, only the pixel point of the projected vehicle bottom is considered, and the situation of occlusion is considered. Usually, the center point height information of the obstacle tracking trajectory track in the UTM coordinate system is subtracted by half of the obstacle tracking trajectory track height dimension.

[0067] In an embodiment of the present application, after the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system, it further includes: judging whether the number of remaining circumscribed contour points when the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system is less than 3; if so, calculating the distance value of each point in the convex hull contour information to each edge of the outer contour formed by the remaining circumscribed contour points; if the distance value meets the requirements, it is determined as a pressure line.

[0068] In order to better obtain the judgment result, when the number of remaining circumscribed contour points when the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system is less than 3, it can be determined whether it is a pressure line by calculating the distance value of each point in the convex hull contour information to each edge of the circumscribed contour.

[0069] In an embodiment of the present application, the convex hull contour information of the camera 2D image pixel points includes: a conversion matrix from the world coordinate system to the camera coordinate system, a pixel resolution of the camera, and a conversion function from the camera coordinate system to the pixel coordinate system after the camera is initialized; receiving a lane line pressure line pixel point set collected by the camera, the lane line pressure line pixel point set including a left lane line pressure line pixel point set and a right lane line pressure line pixel point set; obtaining a plurality of point clusters after clustering the left lane line pressure line pixel point set and / or the right lane line pressure line pixel point set, each point cluster being used as a pressure line point; generating an outer contour convex hull data according to the plurality of point clusters, and putting it into a lane marking queue.

[0070] Firstly, when initializing, the conversion matrix world2camera from the world coordinate system to the camera coordinate system is generated by reading the camera extrinsic data. The camera model is obtained to obtain the pixel resolution of the camera and the conversion function from the camera coordinate system to the pixel coordinate system.

[0071] Then, the left lane line pressure line pixel point set and the right lane line pressure line pixel point set given by the same 2D camera are received, and then converted into the required format for fusion, that is, the pixel point set contains many lane lines.

[0072] Further, the left and right point sets are clustered according to the Euclidean distance to obtain point clusters, and each point cluster represents a pressure line point. Then, the clustered point clusters are convex hull generated, that is, the convex hull polygon of the point cluster is generated.

[0073] Finally, all the convex hulls generated in the current frame are put into the queue lane_markers_list. Put into the lane_markers_list queue, and trigger with track, the time frequency is inconsistent, and the queue is denoised, time aligned and the like.

[0074] Further, the track data is triggered, the closest frame pressure line convex hull data to the track timestamp is found from the lane_markers_list after the track data is received, and is extracted. Then, the closest positioning data to the track timestamp is obtained, and the conversion matrix g2v from the UTM coordinate system to the vehicle coordinate system is generated.

[0075] In an embodiment of the present application, the response to the obstacle tracking trajectory information, the projection of the obstacle tracking trajectory in the 3D coordinate system onto the camera 2D image includes: in the case where the obstacle tracking trajectory information is received, the closest convex hull data of the obstacle tracking trajectory in the frame is found from the lane marker queue. According to the positioning data of the closest convex hull data of the frame, the conversion matrix from the UTM coordinate system to the vehicle coordinate system is generated, and the obstacle tracking trajectory data is converted from the UTM coordinate system to the vehicle coordinate system, wherein when the UTM coordinate system is converted to the vehicle coordinate system, the height of the convex hull in the UTM coordinate system is the ground height of the obstacle tracking trajectory.

[0076] The track data is triggered by the obstacle tracking trajectory track data, the closest pressure line convex hull data to the obstacle tracking trajectory track timestamp or distance is found from the lane_markers_list above after the track data is received, and is extracted, and then the closest positioning data to the obstacle tracking trajectory track timestamp is obtained.

[0077] The embodiment of the present application also provides an obstacle pressure line judgment device 200, such as Figure 2 As shown, a schematic diagram of the structure of the obstacle line pressing judgment device in an embodiment of the present application is provided. The obstacle line pressing judgment device 200 includes at least: an acquisition module 210, a response module 220, and a matching module 230, wherein:

[0078] In one embodiment of the present application, the acquisition module 210 is specifically configured to acquire convex hull outer contour information of pixel points of a camera 2D image.

[0079] The camera 2D image pixels are obtained based on 2D image data collected by at least one camera, and the convex hull outer contour information is determined based on the camera 2D image pixels. The specific number and position of the cameras are not limited in the embodiments of this application, and are based on the position and number of sensors deployed by the autonomous vehicle in the actual scenario.

[0080] In one embodiment of the present application, the response module 220 is specifically configured to: project the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image in response to the obstacle tracking trajectory information.

[0081] The system is triggered by obstacle tracking track data (LiDAR is the main sensor). After receiving the obstacle tracking track data, the external contour information of the obstacle tracking track in 3D coordinates is projected onto the camera's 2D image. After projection processing, the external contour information is determined.

[0082] In one embodiment of the present application, the matching module 230 is specifically used to determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the determination result of whether the obstacle is on the line.

[0083] If the convex hull's outer contour matches the circumscribed contour in the camera's 2D image, the matching result is used as the vehicle's lane-crossing determination. Finally, the left or right lane attribute of the convex hull is assigned to the obstacle tracking track.

[0084] In one embodiment of the present application, the matching module 230 is further configured to:

[0085] Traversing each obstacle tracking track, and screening whether there is an obstacle tracking track within the perception area according to the camera's perception area;

[0086] If yes, aligning the obstacle tracking trajectory with the convex hull outer contour information by time stamp;

[0087] If no, it is determined that whether the lane compression is uncertain.

[0088] In an embodiment of the present application, the matching module 230 is further configured to:

[0089] Linearly align the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system;

[0090] Obtain the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through the conversion matrix;

[0091] Convert the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system, and delete the circumscribed contour points converted to the pixel coordinate system.

[0092] In an embodiment of the present application, it further comprises a first judging module configured to:

[0093] Judge whether the number of remaining circumscribed contour points after the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system is greater than or equal to 3;

[0094] If yes, perform similarity calculation on the outer contour composed of the remaining outer contour points of the obstacle tracking trajectory in the pixel coordinate system and the convex hull outer contour information;

[0095] If the similarity calculation result meets the requirement, it is determined that the lane compression.

[0096] In an embodiment of the present application, it further comprises a second judging module configured to:

[0097] Judge whether the number of remaining circumscribed contour points when the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system is less than 3;

[0098] If yes, calculate the distance value of each point in the convex hull outer contour information to each edge of the outer contour composed of the remaining circumscribed contour points;

[0099] If the distance value meets the requirement, it is determined that the lane compression.

[0100] In an embodiment of the present application, the obtaining module 310 is further configured to:

[0101] Obtain the conversion matrix from the world coordinate system to the camera coordinate system, the pixel resolution of the camera, and the conversion function from the camera coordinate system to the pixel coordinate system after the camera is initialized;

[0102] Receive the lane compression pixel point set collected by the camera, wherein the lane compression pixel point set comprises a left lane compression pixel point set and a right lane compression pixel point set;

[0103] According to the left lane line pixel point set and / or the right lane line pixel point set, a plurality of point clusters are obtained after clustering, and each point cluster is used as a compression line point;

[0104] According to the plurality of point clusters, an external contour convex hull data is generated and put into a lane marking queue.

[0105] In an embodiment of the present application, the response module 320 is further configured to:

[0106] In a case where the obstacle tracking trajectory information is received, the external contour convex hull data of the most recent frame in the obstacle tracking trajectory is found from the lane marking queue;

[0107] According to the positioning data of the external contour convex hull data of the most recent frame, a conversion matrix from the UTM coordinate system to the ego vehicle coordinate system is generated, and the obstacle tracking trajectory data is converted from the UTM coordinate system to the ego vehicle coordinate system,

[0108] In the conversion from the UTM coordinate system to the ego vehicle coordinate system, the height of the external contour in the UTM coordinate system is the ground height of the obstacle tracking trajectory.

[0109] It can be understood that the above-mentioned obstacle compression line judgment device can realize each step of the obstacle compression line judgment method provided in the foregoing embodiments, and the related explanations of the obstacle compression line judgment method are all applicable to the obstacle compression line judgment device, which will not be repeated here.

[0110] Figure 3 is a structural schematic diagram of an electronic device of an embodiment of the present application. Please refer to Figure 3 At the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The memory can include a memory such as a high-speed random access memory (RAM), and can also include a non-volatile memory such as at least one disk memory. Of course, the electronic device can also include other hardware required by the business.

[0111] The processor, the network interface and the memory can be connected with each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 3 Only one bidirectional arrow is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0112] The memory is used to store programs. Specifically, the program can include program code including computer operation instructions. The memory can include an internal memory and a non-volatile memory, and provide instructions and data for the processor.

[0113] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs, and forms the obstacle line pressing judgment device at a logical level. The processor executes the program stored in the memory, and is specifically used for executing the following operations:

[0114] Obtain the convex hull contour information of the camera 2D image pixel points;

[0115] In response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinates onto the camera 2D image;

[0116] Determine whether the convex hull contour information matches the circumscribed contour information in the camera 2D image, and take the matching result as the judgment result of whether the vehicle is pressing the line.

[0117] The above as described in the present application Figure 1The method performed by the obstacle pressure line judgment device disclosed in the embodiment can be applied in a processor or implemented by the processor. The processor can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The above processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiment of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0118] The electronic device can also execute Figure 1 The method performed by the obstacle pressure line judgment device, and implement the functions of the obstacle pressure line judgment device in Figure 1 the embodiment. The embodiment of the present application will not be repeated here.

[0119] The embodiment of the present application also proposes a computer readable storage medium, which stores one or more programs, the one or more programs include instructions, when the instructions are executed by an electronic device including a plurality of application programs, the electronic device can execute Figure 1 The method performed by the obstacle pressure line judgment device in the embodiment, and specifically for executing

[0120] Obtain the convex hull contour information of the camera 2D image pixel point;

[0121] In response to the obstacle tracking trajectory information, project the circumscribed contour information of the obstacle tracking trajectory in the 3D coordinate on the camera 2D image;

[0122] determining whether the convex hull contour information matches the circumscribed contour information in the camera 2D image, and using the result of the determination as the determination result of whether the vehicle is on the lane.

[0123] Those skilled in the art will appreciate that embodiments of the application can be supplied as methods, systems, or computer program products. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be embodied in the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer usable program code embodied therein.

[0124] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It should be understood that each flow and / or block in the flowchart and / or block diagrams, and combinations of flows and / or blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by the flow or flows and / or block or blocks.

[0125] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagrams flow or flows and / or block or blocks. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by the flow or flows and / or block or blocks.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagrams flow or flows and / or block or blocks. Figure 1 one or more flows and / or blocks ​ means for performing the function specified by the flow or flows and / or block or blocks.

[0127] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0128] Memory can include non-persistent memory, such as volatile random access memory (RAM) and / or non-volatile memory, such as read only memory (ROM), electrically erasable read only memory (EEPROM), flash memory, or the like. Memory is an example of computer readable media.

[0129] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0130] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the identified element.

[0131] Those skilled in the art will understand that embodiments of the present application can be provided as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer readable program code.

[0132] The embodiments of the present application described above are merely given as examples of the application, but are not intended to limit the present application. Various modifications and changes can be made thereto by those skilled in the art which fall within the scope of the present application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for determining if an obstacle is pressing a line, wherein: The judgment method includes: Get the convex hull outer contour information of the camera's 2D image pixel points; In response to the obstacle tracking trajectory information, projecting the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image; Determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the determination result of whether the obstacle is pressing the line; The determining whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and using the matching result as the determination result of whether the obstacle is on the line, includes: Traversing each obstacle tracking track, and screening whether there is an obstacle tracking track within the perception area according to the camera's perception area; If yes, aligning the obstacle tracking trajectory with the convex hull outer contour information by time stamp; aligning the obstacle tracking trajectory with the convex hull outer contour information by time stamp includes: Linearly aligning the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system; obtaining circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through a transformation matrix; converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system based on the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, and deleting circumscribed contour points converted outside the pixel coordinate system; Determine whether the number of remaining circumscribed contour points of the obstacle tracking trajectory after the circumscribed contour information is converted from the camera coordinate system to the pixel coordinate system is greater than or equal to 3; if so, perform similarity calculation between the convex hull circumscribed contour information and the remaining circumscribed contour points of the obstacle tracking trajectory in the pixel coordinate system; if the similarity calculation result meets the requirements, determine that the obstacle is pressed; Determine whether the number of remaining circumscribed contour points is less than 3 when the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system; if so, calculate the distance value from each point in the convex hull outer contour information to each edge of the outer contour composed of the remaining circumscribed contour points; if the distance value meets the requirements, it is determined to be a line press.

2. The method according to claim 1, wherein: The step of obtaining the convex hull outer contour information of the camera 2D image pixel points includes: After the camera is initialized, the transformation matrix from the world coordinate system to the camera coordinate system, the pixel resolution of the camera, and the transformation function from the camera coordinate system to the pixel coordinate system are obtained; Receive a lane line pressure pixel point set acquired by a camera, wherein the lane line pressure pixel point set includes a left lane line pressure pixel point set and a right lane line pressure pixel point set; A plurality of point clusters are obtained by clustering the left lane line pressing pixel point set and / or the right lane line pressing pixel point set, each of the point clusters being used as a pressing point; Generate convex hull data of the outer contour based on multiple point clusters and put them into the lane marking queue.

3. The method according to claim 2, wherein: The step of projecting the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image in response to the obstacle tracking trajectory information includes: Upon receiving the obstacle tracking trajectory information, finding the convex hull data of the circumscribed contour of the closest frame to the obstacle tracking trajectory from the lane marking queue; Generate a transformation matrix from the UTM coordinate system to the vehicle coordinate system based on the positioning data of the convex hull data of the most recent frame, and transform the obstacle tracking trajectory information from the UTM coordinate system to the vehicle coordinate system. When the UTM coordinate system is converted to the vehicle coordinate system, the height of the circumscribed contour in the UTM coordinate system is the ground height where the obstacle tracking track is located.

4. An obstacle line-pressing judgment device, wherein: The judging device comprises: An acquisition module is used to obtain the convex hull outer contour information of the camera's 2D image pixel points; A response module, configured to project the circumscribed contour information of the obstacle tracking trajectory in 3D coordinates onto the camera 2D image in response to the obstacle tracking trajectory information; A matching module is used to determine whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and use the matching result as the determination result of whether the obstacle is on the line; The determining whether the convex hull outer contour information matches the circumscribed contour information in the camera 2D image, and using the matching result as the determination result of whether the obstacle is on the line, includes: Traversing each obstacle tracking track, and screening whether there is an obstacle tracking track within the perception area according to the camera's perception area; If yes, aligning the obstacle tracking trajectory with the convex hull outer contour information by time stamp; aligning the obstacle tracking trajectory with the convex hull outer contour information by time stamp includes: Linearly aligning the circumscribed contour of the obstacle tracking trajectory in the UTM coordinate system; obtaining circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system through a transformation matrix; converting the circumscribed contour information of the obstacle tracking trajectory from the camera coordinate system to the pixel coordinate system based on the circumscribed contour information of the obstacle tracking trajectory in the camera coordinate system, and deleting circumscribed contour points converted outside the pixel coordinate system; Determine whether the number of remaining circumscribed contour points of the obstacle tracking trajectory after the circumscribed contour information is converted from the camera coordinate system to the pixel coordinate system is greater than or equal to 3; if so, perform similarity calculation between the convex hull circumscribed contour information and the remaining circumscribed contour points of the obstacle tracking trajectory in the pixel coordinate system; if the similarity calculation result meets the requirements, determine that the obstacle is pressed; Determine whether the number of remaining circumscribed contour points is less than 3 when the circumscribed contour information of the obstacle tracking trajectory is converted from the camera coordinate system to the pixel coordinate system; if so, calculate the distance value from each point in the convex hull outer contour information to each edge of the outer contour composed of the remaining circumscribed contour points; if the distance value meets the requirements, it is determined to be a line press.

5. An electronic device comprising: processor; as well as A memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 3.

6. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute the method according to any one of claims 1 to 3.

Images